OpenSMTPD: Don't return message body in successfull DNS reports

[Christopher Zimmermann]

Hi,

delivery success DSNs include the message body if not explicitely 
disabled by RET HDRS.
But according to rfc3461 4.3 the body should _only_ be included for 
failure DSNs:


[…] If a DSN contains no indications of delivery failure, only the 
headers of the message should be returned.


The default behaviour when neither RET HDRS nor RET FULL is requested is 
not specified:


If no RET parameter is supplied, the MTA MAY return either the headers 
of the message or the entire message for any DSN containing indication 
of failed deliveries.


To me it seems more sane to not include the whole message, which may 
well be quite large.


Therefore I propose the following change. OK ?


Christopher

Index: bounce.c
===
RCS file: /cvs/src/usr.sbin/smtpd/bounce.c,v
retrieving revision 1.87
diff -u -p -r1.87 bounce.c
--- bounce.c8 Feb 2023 08:20:54 -   1.87
+++ bounce.c20 Apr 2023 17:30:58 -
@@ -538,8 +538,8 @@ bounce_next(struct bounce_session *s)
if ((len = getline(, , s->msgfp)) == -1)
break;
if (len == 1 && line[0] == '\n' && /* end of headers */
-   s->msg->bounce.type == B_DELIVERED &&
-   s->msg->bounce.dsn_ret ==  DSN_RETHDRS) {
+   (s->msg->bounce.type != B_FAILED ||
+   s->msg->bounce.dsn_ret != DSN_RETFULL)) {
free(line);
fclose(s->msgfp);
s->msgfp = NULL;

Re: explain priority codepoints and their mapping in vlan.4

[Christopher Zimmermann]

On Mon, Dec 20, 2021 at 04:41:07PM -0700, Theo de Raadt wrote:
You say it twice.  But my eyes still glazed over it, not seeing what 
was going on the first two times.


Maybe something more like

 prio 0 and 1 are mapped out of order to PCP 1 and 0, but prio 2 to
 7 are mapped directly to PCP 2 to 7.

No that still doesn't quite capture it in a visible way. How about

 prio 2 to 7 are mapped directly to PCP 2 to 7, but prio 0 and 1
 are mapped backwards, to PCP 1 and 0, because <...>

Something which will draw the eye+brain to 'something is different here'.
The table alone doesn't do that.


I agree. How about this?

 The 802.1Q and 802.1ad protocols include a Priority Code Point (PCP).  By
 default, the 802.1p PCP in a transmitted packet is based on the priority
 of packets sent over the interface, which may be altered via pf.conf(5);
 see the prio option for more information.  Alternatively, the ifconfig(4)
 txprio option can set a specific priority for transmitted packets. On
 vlan and svlan interfaces priorities 2 to 7 will be mapped directly to
 PCP 2 to 7, but priorities 0 and 1 are mapped backwards, to PCP 1 and 0.
 This is because 802.1p defines PCP 1 as lowest priority and PCP 0 as
 second lowest priority, which is meant to be used as default (“best
 effort”).


Index: vlan.4
===
RCS file: /cvs/src/share/man/man4/vlan.4,v
retrieving revision 1.51
diff -u -p -r1.51 vlan.4
--- vlan.4  4 Oct 2020 12:44:49 -   1.51
+++ vlan.4  30 Dec 2021 20:00:38 -
@@ -83,17 +83,26 @@ interfaces by their respective protocol 
 identifiers, and decapsulated for reception on the associated virtual

 interfaces.
 .Pp
-The 802.1Q and 802.1ad protocols include a priority field.
-By default, the 802.1p priority in a transmitted packet is based on the
+The 802.1Q and 802.1ad protocols include a Priority Code Point (PCP).
+By default, the 802.1p PCP in a transmitted packet is based on the
 priority of packets sent over the interface, which may
 be altered via
 .Xr pf.conf 5 ;
 see the
 .Cm prio
 option for more information.
-Alternatively,
+Alternatively, the
+.Xr ifconfig 4
 .Cm txprio
-can set a specific priority for transmitted packets.
+option can set a specific priority for transmitted packets. On
+.Nm vlan
+and
+.Nm svlan
+interfaces priorities 2 to 7 will be mapped directly to PCP 2 to 7, but
+priorities 0 and 1 are mapped backwards, to PCP 1 and 0. This is because
+802.1p defines PCP 1 as lowest priority and PCP 0 as second lowest
+priority, which is meant to be used as default
+.Pq Dq best effort .
 .Pp
 .Nm vlan
 and

Re: explain priority codepoints and their mapping in vlan.4

[Christopher Zimmermann]

On Tue, Dec 21, 2021 at 07:12:31AM +1000, David Gwynne wrote:
are you able to find a reference for this in any of the current specs? 
or in documentation from other vendors?


https://en.wikipedia.org/wiki/IEEE_P802.1p

i've faithfully maintained this behaviour, but the only place i've seen 
in with my own eyes is in our code. if we're doing it different to 
literally everyone else, then maybe we shouldnt?


We are not doing it different. But in my mind especially the mapping of 
PCP / prio 0 / 1 is not intuitive and may be confusing (but is 
correct!). That's why I think this should be documented in detail.



On 21 Dec 2021, at 00:54, Christopher Zimmermann  wrote:

Hi,

it might be helpful for others to understand the meaning of codepoints 0 and 1 
of the vlan priority field. OK?

Christopher

Here is the formatted content:

The 802.1Q and 802.1ad protocols include a 3-bit priority code point
(PCP):
   PCP 1 is defined as the lowest priority (“background”)
   PCP 0 is the default (“best effort”) - second lowest priority.
   PCPs 2-7 are increasing priority above best effort.
The priority may be altered via pf.conf(5); see the prio option for more
information.
   “prio 0” is mapped to PCP 1.
   “prio 1” is mapped to PCP 0.
   “prio 2-7” are mapped to PCP 2-7.
Alternatively, the txprio property of an interface can set a specific
priority for transmitted packets.


And here's the diff:

Index: vlan.4
===
RCS file: /cvs/src/share/man/man4/vlan.4,v
retrieving revision 1.51
diff -u -p -r1.51 vlan.4
--- vlan.4  4 Oct 2020 12:44:49 -   1.51
+++ vlan.4  20 Dec 2021 14:51:31 -
@@ -83,16 +83,38 @@ interfaces by their respective protocol  identifiers, and 
decapsulated for reception on the associated virtual
interfaces.
.Pp
-The 802.1Q and 802.1ad protocols include a priority field.
-By default, the 802.1p priority in a transmitted packet is based on the
-priority of packets sent over the interface, which may
-be altered via
+The 802.1Q and 802.1ad protocols include a 3-bit priority code point
+(PCP):
+.Bl -item -compact -offset MMM
+.It
+PCP 1 is defined as the lowest priority
+.Pq Dq background
+.It
+PCP 0 is the default
+.Pq Dq best effort
+- second lowest priority.
+.It
+PCPs 2-7 are increasing priority above best effort.
+.El
+The priority may be altered via
.Xr pf.conf 5 ;
see the
.Cm prio
option for more information.
-Alternatively,
+.Bl -item -compact -offset MMM
+.It
+.Dq prio 0
+is mapped to PCP 1.
+.It
+.Dq prio 1
+is mapped to PCP 0.
+.It
+.Dq prio 2-7
+are mapped to PCP 2-7.
+.El
+Alternatively, the
.Cm txprio
+property of an interface
can set a specific priority for transmitted packets.
.Pp
.Nm vlan





--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

document patch(1) not supporting binary diffs

[Christopher Zimmermann]

Hi,

base patch cannot work with diffs of binary files. It might help to say 
so in the manpage since other implementations do support this (ab)use of 
patch. OK?



Christopher


Index: patch.1
===
RCS file: /cvs/src/usr.bin/patch/patch.1,v
retrieving revision 1.33
diff -u -p -r1.33 patch.1
--- patch.1 9 Nov 2021 16:13:40 -   1.33
+++ patch.1 20 Dec 2021 15:00:52 -
@@ -655,6 +655,8 @@ However, the results are guaranteed to b
 applied to exactly the same version of the file that the patch was
 generated from.
 .Sh BUGS
+No support for patching binary files.
+.Pp
 Could be smarter about partial matches, excessively deviant offsets and
 swapped code, but that would take an extra pass.
 .Pp

explain priority codepoints and their mapping in vlan.4

[Christopher Zimmermann]

Hi,

it might be helpful for others to understand the meaning of codepoints 0 
and 1 of the vlan priority field. OK?


Christopher

Here is the formatted content:

The 802.1Q and 802.1ad protocols include a 3-bit priority code point
 (PCP):
PCP 1 is defined as the lowest priority (“background”)
PCP 0 is the default (“best effort”) - second lowest priority.
PCPs 2-7 are increasing priority above best effort.
 The priority may be altered via pf.conf(5); see the prio option for more
 information.
“prio 0” is mapped to PCP 1.
“prio 1” is mapped to PCP 0.
“prio 2-7” are mapped to PCP 2-7.
 Alternatively, the txprio property of an interface can set a specific
 priority for transmitted packets.


And here's the diff:

Index: vlan.4
===
RCS file: /cvs/src/share/man/man4/vlan.4,v
retrieving revision 1.51
diff -u -p -r1.51 vlan.4
--- vlan.4  4 Oct 2020 12:44:49 -   1.51
+++ vlan.4  20 Dec 2021 14:51:31 -
@@ -83,16 +83,38 @@ interfaces by their respective protocol 
 identifiers, and decapsulated for reception on the associated virtual

 interfaces.
 .Pp
-The 802.1Q and 802.1ad protocols include a priority field.
-By default, the 802.1p priority in a transmitted packet is based on the
-priority of packets sent over the interface, which may
-be altered via
+The 802.1Q and 802.1ad protocols include a 3-bit priority code point
+(PCP):
+.Bl -item -compact -offset MMM
+.It
+PCP 1 is defined as the lowest priority
+.Pq Dq background
+.It
+PCP 0 is the default
+.Pq Dq best effort
+- second lowest priority.
+.It
+PCPs 2-7 are increasing priority above best effort.
+.El
+The priority may be altered via
 .Xr pf.conf 5 ;
 see the
 .Cm prio
 option for more information.
-Alternatively,
+.Bl -item -compact -offset MMM
+.It
+.Dq prio 0
+is mapped to PCP 1.
+.It
+.Dq prio 1
+is mapped to PCP 0.
+.It
+.Dq prio 2-7
+are mapped to PCP 2-7.
+.El
+Alternatively, the
 .Cm txprio
+property of an interface
 can set a specific priority for transmitted packets.
 .Pp
 .Nm vlan

xhci uhub on arm64: handle device in SS_INACTIVE state

[Christopher Zimmermann]

Hi,

on my RK3399, a usb device connected to the USB 3 port is not detected 
during boot because it is in SS_INACTIVE (0x00c0) state:


uhub3 at usb3 configuration 1 interface 0 "Generic xHCI root hub" rev 
3.00/1.00 addr 1

uhub3: uhub_attach
uhub3: 2 ports with 2 removable, self powered
uhub3: intr status=0
usb_needs_explore: usb3: not exploring before first explore
uhub3: uhub_explore
uhub3: port 1 status=0x02a0 change=0x
uhub3: port 2 status=0x02c0 change=0x0040
usb_explore: usb3: first explore done
xhci1: port=2 change=0x04
uhub3: intr status=0
uhub3: uhub_explore
uhub3: port 2 status=0x02c0 change=0x0040
xhci1: port=2 change=0x04
uhub3: intr status=0
uhub3: uhub_explore
uhub3: port 2 status=0x02c0 change=0x0040

[...]

[turn the usb device off and on again]

uhub3: intr status=0
uhub3: uhub_explore
uhub3: port 2 status=0x0203 change=0x0001
usbd_reset_port: port 2 reset done
xhci1: port=2 change=0x04
uhub3: intr status=0
uhub3: port 2 status=0x0203 change=0x
umass0 at uhub3 port 2 configuration 1 interface 0 "ATEC Dual Disk Drive" rev 
3.00/1.08 addr 2

This might be because u-boot-aarch64-2021.10 from packages left it in 
that state.

I added this code to reset a device locked in such a state:

Index: uhub.c
===
RCS file: /cvs/src/sys/dev/usb/uhub.c,v
retrieving revision 1.95
diff -u -p -r1.95 uhub.c
--- uhub.c  31 Jul 2020 10:49:33 -  1.95
+++ uhub.c  17 Oct 2021 06:44:14 -
@@ -414,6 +414,24 @@ uhub_explore(struct usbd_device *dev)
change |= UPS_C_CONNECT_STATUS;
}

+   if (change & UPS_C_PORT_LINK_STATE &&
+   UPS_PORT_LS_GET(status) == UPS_PORT_LS_SS_INACTIVE &&
+   ! (status & UPS_CURRENT_CONNECT_STATUS)) {
+   DPRINTF("%s: port %d is in in SS_INACTIVE.Quiet state. "
+ "Reset port.\n",
+ sc->sc_dev.dv_xname, port);
+   usbd_clear_port_feature(sc->sc_hub, port,
+   UHF_C_PORT_RESET);
+
+   if (usbd_reset_port(sc->sc_hub, port)) {
+   printf("%s: port %d reset failed\n",
+ DEVNAME(sc), port);
+   return (-1);
+   }
+
+   change |= UPS_C_CONNECT_STATUS;
+   }
+
if (change & UPS_C_BH_PORT_RESET &&
sc->sc_hub->speed == USB_SPEED_SUPER) {
usbd_clear_port_feature(sc->sc_hub, port,


Now the device attaches during boot. A redundant second reset of the 
device is performed during uhub_port_connect():


uhub3 at usb3 configuration 1 interface 0 "Generic xHCI root hub" rev 
3.00/1.00 addr 1

uhub3: uhub_attach
uhub3: 2 ports with 2 removable, self powered
xhci1: port=2 change=0x04
uhub3: intr status=0
usb_needs_explore: usb3: not exploring before first explore
uhub3: uhub_explore
uhub3: port 1 status=0x02a0 change=0x
uhub3: port 2 status=0x02c0 change=0x0040
uhub3: port 2 is in in SS_INACTIVE.Quiet state. Reset port.
usbd_reset_port: port 2 reset done
usb_explore: usb3: first explore done
xhci1: port=2 change=0x04
uhub3: intr status=0
uhub3: uhub_explore
uhub3: port 2 status=0x0203 change=0x0031
uhub3: uhub_port_connect
usbd_reset_port: port 2 reset done
xhci1: port=2 change=0x04
uhub3: intr status=0
uhub3: port 2 status=0x0203 change=0x
umass0 at uhub3 port 2 configuration 1 interface 0 "ATEC Dual Disk Drive" rev 
3.00/1.08 addr 2


OK to commit this diff? Or should this be done some other way?


Christopher

Re: /bin/cp: overwrite symlink with file pointed to by symlink

[Christopher Zimmermann]

Hi Quentin,

Thank you for having a look.
I know I'm very late, but I still like your patch. So ok chrisz@ if you 
want to commit it.


Christopher


On Fri, Nov 01, 2019 at 08:46:51PM +0100, Quentin Rameau wrote:

If you're interested in adapting to POSIX, here's a proposition patch:

Index: bin/cp//cp.1
===
RCS file: /var/cvs/src/bin/cp/cp.1,v
retrieving revision 1.40
diff -u -r1.40 cp.1
--- bin/cp//cp.128 Jan 2019 18:58:42 -  1.40
+++ bin/cp//cp.11 Nov 2019 19:41:26 -
@@ -79,9 +79,8 @@
Same as
.Fl RpP .
.It Fl f
-For each existing destination pathname, remove it and
-create a new file, without prompting for confirmation,
-regardless of its permissions.
+For each existing destination pathname which cannot be opened, remove
+it and create a new file, without prompting for confirmation.
The
.Fl f
option overrides any previous
Index: bin/cp//utils.c
===
RCS file: /var/cvs/src/bin/cp/utils.c,v
retrieving revision 1.47
diff -u -r1.47 utils.c
--- bin/cp//utils.c 28 Jan 2019 18:58:42 -  1.47
+++ bin/cp//utils.c 1 Nov 2019 19:25:02 -
@@ -55,7 +55,7 @@
static char *buf;
static char *zeroes;
struct stat to_stat, *fs;
-   int from_fd, rcount, rval, to_fd, wcount;
+   int from_fd, rcount, rval, to_fd, wcount, create = 1;
#ifdef VM_AND_BUFFER_CACHE_SYNCHRONIZED
char *p;
#endif
@@ -79,26 +79,24 @@
fs = entp->fts_statp;

/*
-* In -f (force) mode, we always unlink the destination first
-* if it exists.  Note that -i and -f are mutually exclusive.
-*/
-   if (exists && fflag)
-   (void)unlink(to.p_path);
-
-   /*
 * If the file DNE, set the mode to be the from file, minus setuid
 * bits, modified by the umask; arguably wrong, but it makes copying
 * executables work right and it's been that way forever.  (The
 * other choice is 666 or'ed with the execute bits on the from file
 * modified by the umask.)
 */
-   if (exists && !fflag) {
-   if (!copy_overwrite()) {
+   if (exists) {
+   /* Note that -i and -f are mutually exclusive. */
+   if (!fflag && !copy_overwrite()) {
(void)close(from_fd);
return 2;
}
to_fd = open(to.p_path, O_WRONLY | O_TRUNC, 0);
-   } else
+   if (to_fd != -1 || fflag && unlink(to.p_path) == -1)
+   create = 0;
+   }
+
+   if (create)
to_fd = open(to.p_path, O_WRONLY | O_TRUNC | O_CREAT,
fs->st_mode & ~(S_ISTXT | S_ISUID | S_ISGID));




--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Re: Unbound: add support for pf tables to ipset module

[Christopher Zimmermann]

On Sun, Feb 07, 2021 at 04:20:26PM +, Stuart Henderson wrote:

On 2021/02/07 17:04, Christopher Zimmermann wrote:

Hi,

a year ago I added support for our pf tables to the unbound ipset module.
Upstream does not seem eager to merge it:
https://github.com/NLnetLabs/unbound/pull/144

Implementing pf tables support was pretty straightforward. It has been more
work to adjust module's privilege management to allow the modules to open
privileget files like /dev/pf and keep them open across reloads.
This is also what upstream was unsure about.

So below you find the diff against our base unbound.

Should this go in? Continue to wait for upstream?
Suggestions for improvement?


I would not be happy about including this in base unbound. Partly
because it is a large diff to carry, partly unbound is a much more
complex process than I'd be happy with having direct access to
reconfigure PF.

The whole approach (including for linux ipset) doesn't seem ideal to
me. It would seem much better to have this done out-of-process with a
communication mechanism to allow sending the addresses across, then
unbound wouldn't need firewall-specific knowledge in the code, and
there's a clear separation of privilege.


Hi Stuart and Florian,

thanks for giving a thought. I agree to your analysis.
It seems to me a more sane approach would be to change / create an 
"ipset" module that doesn't talk a specific ipset / pf protocol, but 
simply dumps raw ipv4 /ipv6 addresses in a file / fifo / pipe, which can 
then be consumed by a thin privileged translator process that adds those 
ips to pf / ipset / whatever. This would also get rid of the privilege 
management issues I encountered.
Now I'm just wondering whether the filtering feature of ipset should 
remain in unbound or be moved to another process. I would tend to keep 
it within unbound, since parsing queries is what it is built to do after 
all.



Thanks,

Christopher



--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Unbound: add support for pf tables to ipset module

[Christopher Zimmermann]

Hi,

a year ago I added support for our pf tables to the unbound ipset 
module. Upstream does not seem eager to merge it:

https://github.com/NLnetLabs/unbound/pull/144

Implementing pf tables support was pretty straightforward. It has been 
more work to adjust module's privilege management to allow the modules 
to open privileget files like /dev/pf and keep them open across reloads.

This is also what upstream was unsure about.

So below you find the diff against our base unbound.

Should this go in? Continue to wait for upstream?
Suggestions for improvement?


Christopher



Index: Makefile.bsd-wrapper
===
RCS file: /cvs/src/usr.sbin/unbound/Makefile.bsd-wrapper,v
retrieving revision 1.20
diff -u -p -r1.20 Makefile.bsd-wrapper
--- Makefile.bsd-wrapper24 Aug 2020 15:33:33 -  1.20
+++ Makefile.bsd-wrapper7 Feb 2021 16:02:51 -
@@ -17,6 +17,7 @@ CONFIGURE_OPTS_UNBOUND=   --enable-allsymb
--with-rootkey-file=/var/unbound/db/root.key \
--with-conf-file=${CHROOTDIR}/etc/unbound.conf \
--with-username=_unbound \
+   --enable-ipset \
--disable-shared \
--disable-explicit-port-randomisation
 
Index: config.h.in

===
RCS file: /cvs/src/usr.sbin/unbound/config.h.in,v
retrieving revision 1.21
diff -u -p -r1.21 config.h.in
--- config.h.in 28 Oct 2020 11:31:06 -  1.21
+++ config.h.in 7 Feb 2021 16:02:51 -
@@ -381,6 +381,9 @@
 /* Define to 1 if you have the  header file. */
 #undef HAVE_NETTLE_EDDSA_H
 
+/* Define to 1 if you have the  header file. */

+#undef HAVE_NET_PFVAR_H
+
 /* Define to 1 if you have the  header file. */
 #undef HAVE_NET_IF_H
 
Index: configure

===
RCS file: /cvs/src/usr.sbin/unbound/configure,v
retrieving revision 1.41
diff -u -p -r1.41 configure
--- configure   10 Dec 2020 21:44:57 -  1.41
+++ configure   7 Feb 2021 16:02:52 -
@@ -21520,7 +21520,22 @@ $as_echo "#define USE_IPSET 1" >>confdef
IPSET_OBJ="ipset.lo"
 
 
-		# mnl

+   # BSD's pf
+   for ac_header in net/pfvar.h
+do :
+  ac_fn_c_check_header_compile "$LINENO" "net/pfvar.h" "ac_cv_header_net_pfvar_h" 
"
+ #include 
+ #include 
+
+"
+if test "x$ac_cv_header_net_pfvar_h" = xyes; then :
+  cat >>confdefs.h <<_ACEOF
+#define HAVE_NET_PFVAR_H 1
+_ACEOF
+
+else
+
+ # mnl
 
 # Check whether --with-libmnl was given.

 if test "${with_libmnl+set}" = set; then :
@@ -21529,28 +21544,34 @@ else
withval="yes"
 fi
 
-		found_libmnl="no"

-   { $as_echo "$as_me:${as_lineno-$LINENO}: checking for libmnl" 
>&5
+ found_libmnl="no"
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for libmnl" 
>&5
 $as_echo_n "checking for libmnl... " >&6; }
-   if test x_$withval = x_ -o x_$withval = x_yes; then
-   withval="/usr/local /opt/local /usr/lib /usr/pkg /usr/sfw 
/usr"
-   fi
-   for dir in $withval ; do
-   if test -f "$dir/include/libmnl/libmnl.h"; then
-   found_libmnl="yes"
-   if test "$dir" != 
"/usr"; then
-   CPPFLAGS="$CPPFLAGS -I$dir/include"
-   LDFLAGS="$LDFLAGS -L$dir/lib"
-   fi
-   { $as_echo "$as_me:${as_lineno-$LINENO}: result: found in 
$dir" >&5
+ if test x_$withval = x_ -o x_$withval = x_yes; then
+ withval="/usr/local /opt/local /usr/lib /usr/pkg /usr/sfw 
/usr"
+ fi
+ for dir in $withval ; do
+ if test -f "$dir/include/libmnl/libmnl.h"; then
+ found_libmnl="yes"
+ if test "$dir" != 
"/usr"; then
+ CPPFLAGS="$CPPFLAGS -I$dir/include"
+ LDFLAGS="$LDFLAGS -L$dir/lib"
+ fi
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: found in 
$dir" >&5
 $as_echo "found in $dir" >&6; }
-   LIBS="$LIBS -lmnl"
-   break;
-   fi
-   done
-   if test x_$found_libmnl != x_yes; then
-   as_fn_error $? "Could not find libmnl, libmnl.h" 
"$LINENO" 5
-   fi
+ LIBS="$LIBS -lmnl"
+ break;
+ fi
+ 

add OpenType bitmap fonts to xenocara

[Christopher Zimmermann]

Hi,

this makes the xenocara bitmap fonts available for pango apps like gvim, 
too.
Is it ok to call fonttosfnt directly in the Makefile or is there a 
reason to use a macro like done for $(BDFTOPCF) ? OK?



Christopher


--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1
Index: adobe-100dpi/Makefile.am
===
RCS file: /cvs/xenocara/font/adobe-100dpi/Makefile.am,v
retrieving revision 1.4
diff -u -p -r1.4 Makefile.am
--- adobe-100dpi/Makefile.am	2 Oct 2016 09:34:35 -	1.4
+++ adobe-100dpi/Makefile.am	25 Oct 2020 10:54:00 -
@@ -127,6 +127,7 @@ FONT_FILES = $(BDF2UCS_FONT_FILES) $(EXT
 
 BDF_FILES = $(FONT_FILES:%=%.bdf)
 PCF_FILES = $(FONT_FILES:%=%.pcf$(COMPRESS_SUFFIX))
+OTB_FILES = $(FONT_FILES:%=%.otb)
 
 if ISO8859_1
 ISO8859_1_BDF_FILES = $(BDF2UCS_FONT_FILES:%=%-ISO8859-1.bdf)
@@ -190,6 +191,10 @@ SUFFIXES = .bdf .pcf$(COMPRESS_SUFFIX) \
 	@rm -f $@
 	$(AM_V_GEN)$(BDFTOPCF) -t $< | $(COMPRESS) > $@
 
+.bdf.otb:
+	@rm -f $@
+	$(AM_V_GEN)fonttosfnt -o $@ $<
+
 .bdf-ISO8859-1.bdf:
 	@rm -f $@
 	$(AM_V_GEN)$(UCS2ANY) $< $(UTIL_DIR)/map-ISO8859-1 ISO8859-1
@@ -229,6 +234,7 @@ SUFFIXES = .bdf .pcf$(COMPRESS_SUFFIX) \
 fontdir = @FONTDIR@
 font_DATA = \
 	$(PCF_FILES) \
+	$(OTB_FILES) \
 	$(ISO8859_1_PCF_FILES) \
 	$(ISO8859_2_PCF_FILES) \
 	$(ISO8859_3_PCF_FILES) \
Index: adobe-100dpi/Makefile.in
===
RCS file: /cvs/xenocara/font/adobe-100dpi/Makefile.in,v
retrieving revision 1.6
diff -u -p -r1.6 Makefile.in
--- adobe-100dpi/Makefile.in	2 Oct 2016 09:36:26 -	1.6
+++ adobe-100dpi/Makefile.in	25 Oct 2020 10:54:00 -
@@ -272,6 +272,7 @@ pdfdir = @pdfdir@
 prefix = @prefix@
 program_transform_name = @program_transform_name@
 psdir = @psdir@
+runstatedir = @runstatedir@
 sbindir = @sbindir@
 sharedstatedir = @sharedstatedir@
 srcdir = @srcdir@
@@ -389,6 +390,7 @@ EXTRA_FONT_FILES = \
 FONT_FILES = $(BDF2UCS_FONT_FILES) $(EXTRA_FONT_FILES)
 BDF_FILES = $(FONT_FILES:%=%.bdf)
 PCF_FILES = $(FONT_FILES:%=%.pcf$(COMPRESS_SUFFIX))
+OTB_FILES = $(FONT_FILES:%=%.otb)
 @ISO8859_1_TRUE@ISO8859_1_BDF_FILES = $(BDF2UCS_FONT_FILES:%=%-ISO8859-1.bdf)
 @ISO8859_1_TRUE@ISO8859_1_PCF_FILES = $(BDF2UCS_FONT_FILES:%=%-ISO8859-1.pcf$(COMPRESS_SUFFIX))
 @ISO8859_2_TRUE@ISO8859_2_BDF_FILES = $(BDF2UCS_FONT_FILES:%=%-ISO8859-2.bdf)
@@ -422,6 +424,7 @@ SUFFIXES = .bdf .pcf$(COMPRESS_SUFFIX) \
 fontdir = @FONTDIR@
 font_DATA = \
 	$(PCF_FILES) \
+	$(OTB_FILES) \
 	$(ISO8859_1_PCF_FILES) \
 	$(ISO8859_2_PCF_FILES) \
 	$(ISO8859_3_PCF_FILES) \
@@ -449,7 +452,7 @@ MAINTAINERCLEANFILES = ChangeLog INSTALL
 all: all-am
 
 .SUFFIXES:
-.SUFFIXES: .bdf .pcf$(COMPRESS_SUFFIX) -ISO8859-1.bdf -ISO8859-2.bdf -ISO8859-3.bdf -ISO8859-4.bdf -ISO8859-9.bdf -ISO8859-10.bdf -ISO8859-13.bdf -ISO8859-14.bdf -ISO8859-15.bdf
+.SUFFIXES: .bdf .pcf$(COMPRESS_SUFFIX) -ISO8859-1.bdf -ISO8859-2.bdf -ISO8859-3.bdf -ISO8859-4.bdf -ISO8859-9.bdf -ISO8859-10.bdf -ISO8859-13.bdf -ISO8859-14.bdf -ISO8859-15.bdf .otb
 am--refresh: Makefile
 	@:
 $(srcdir)/Makefile.in: @MAINTAINER_MODE_TRUE@ $(srcdir)/Makefile.am  $(am__configure_deps)
@@ -784,6 +787,10 @@ uninstall-am: uninstall-fontDATA
 .bdf.pcf$(COMPRESS_SUFFIX):
 	@rm -f $@
 	$(AM_V_GEN)$(BDFTOPCF) -t $< | $(COMPRESS) > $@
+
+.bdf.otb:
+	@rm -f $@
+	$(AM_V_GEN)fonttosfnt -o $@ $<
 
 .bdf-ISO8859-1.bdf:
 	@rm -f $@
Index: adobe-75dpi/Makefile.am
===
RCS file: /cvs/xenocara/font/adobe-75dpi/Makefile.am,v
retrieving revision 1.4
diff -u -p -r1.4 Makefile.am
--- adobe-75dpi/Makefile.am	2 Oct 2016 09:34:35 -	1.4
+++ adobe-75dpi/Makefile.am	25 Oct 2020 10:54:00 -
@@ -127,6 +127,7 @@ FONT_FILES = $(BDF2UCS_FONT_FILES) $(EXT
 
 BDF_FILES = $(FONT_FILES:%=%.bdf)
 PCF_FILES = $(FONT_FILES:%=%.pcf$(COMPRESS_SUFFIX))
+OTB_FILES = $(FONT_FILES:%=%.otb)
 
 if ISO8859_1
 ISO8859_1_BDF_FILES = $(BDF2UCS_FONT_FILES:%=%-ISO8859-1.bdf)
@@ -190,6 +191,10 @@ SUFFIXES = .bdf .pcf$(COMPRESS_SUFFIX) \
 	@rm -f $@
 	$(AM_V_GEN)$(BDFTOPCF) -t $< | $(COMPRESS) > $@
 
+.bdf.otb:
+	@rm -f $@
+	$(AM_V_GEN)fonttosfnt -o $@ $<
+
 .bdf-ISO8859-1.bdf:
 	@rm -f $@
 	$(AM_V_GEN)$(UCS2ANY) $< $(UTIL_DIR)/map-ISO8859-1 ISO8859-1
@@ -229,6 +234,7 @@ SUFFIXES = .bdf .pcf$(COMPRESS_SUFFIX) \
 fontdir = @FONTDIR@
 font_DATA = \
 	$(PCF_FILES) \
+	$(OTB_FILES) \
 	$(ISO8859_1_PCF_FILES) \
 	$(ISO8859_2_PCF_FILES) \
 	$(ISO8859_3_PCF_FILES) \
Index: adobe-75dpi/Makefile.in
===
RCS file: /cvs/xenocara/font/adobe-75dpi/Makefile.in,v
retrieving revision 1.6
diff -u -p -r1.6 Makefile.in
--- adobe-75dpi/Makefile.in	2 Oct 2016 09:36:26 -	1.6
+++ adobe-75dpi/Makefile.in	25 Oct 2020 10:54:00 -
@@ -272,6 +272,7 @@ pdfdir = @pdfdir@
 prefix = @prefix@
 program_transform_name = @program_transform_name@
 psdir = @psdir@

fonttosfnt: merge changes to fix fonts for new pango

[Christopher Zimmermann]

Hi Matthieu,

you already gave an ok some time ago to merge changes to fonttosfnt. Now 
that my changes have been tested and merged upstream I'd like to 
integrate them into xenocara. May I ask for another ok?


Christopher


Index: fonttosfnt.c
===
RCS file: /cvs/xenocara/app/fonttosfnt/fonttosfnt.c,v
retrieving revision 1.3
diff -u -p -r1.3 fonttosfnt.c
--- fonttosfnt.c14 Jan 2020 20:52:17 -  1.3
+++ fonttosfnt.c9 Oct 2020 07:27:19 -
@@ -106,6 +106,12 @@ main(int argc, char **argv)
 
 font = makeFont();
 
+if(argc - i > 1)

+   fprintf(stderr,
+   "You are requesting to put more than one font into a single OpenType 
font.\n"
+   "This is not recommended. The global font metrics will not match every 
font face.\n"
+   "The creation of an OpenType font collection is recommended.\n");
+
 if(i == argc) {
 rc = readFile(NULL, font);
 if(rc != 0)
Index: fonttosfnt.h
===
RCS file: /cvs/xenocara/app/fonttosfnt/fonttosfnt.h,v
retrieving revision 1.3
diff -u -p -r1.3 fonttosfnt.h
--- fonttosfnt.h15 Aug 2019 16:57:57 -  1.3
+++ fonttosfnt.h9 Oct 2020 07:27:19 -
@@ -29,6 +29,7 @@ THE SOFTWARE.
 #endif
 
 #include 

+#include 
 #include 
 #include FT_FREETYPE_H
 
@@ -69,17 +70,15 @@ extern int reencode_flag;
 
 #define UNITS_PER_EM 2048
 
-#define EPSILON 0.1

-#define FLOOR(x) ((x) < 0.0 ? -(int)(-(x)) : (x))
-#define CEIL(x) FLOOR((x) + 1.0 - EPSILON)
+#define UNDEF 0x8000
 
 /* Convert a fixed-point value into FUnits */

 #define FONT_UNITS(x) \
-  FLOOR(((double)(x)) / TWO_SIXTEENTH * UNITS_PER_EM + 0.5)
+  round(((double)(x)) / TWO_SIXTEENTH * UNITS_PER_EM)
 #define FONT_UNITS_FLOOR(x) \
-  FLOOR(((double)(x)) / TWO_SIXTEENTH * UNITS_PER_EM)
+  floor(((double)(x)) / TWO_SIXTEENTH * UNITS_PER_EM)
 #define FONT_UNITS_CEIL(x) \
-  CEIL(((double)(x)) / TWO_SIXTEENTH * UNITS_PER_EM)
+  ceil(((double)(x)) / TWO_SIXTEENTH * UNITS_PER_EM)
 
 typedef struct _FontNameEntry {

 int nid;/* name id */
@@ -87,6 +86,23 @@ typedef struct _FontNameEntry {
 char *value;
 } FontNameEntryRec, *FontNameEntryPtr;
 
+typedef struct _Metrics {

+int height;
+int size;
+int maxX;
+int minX;
+int maxY;
+int minY;
+int xHeight;
+int capHeight;
+int maxAwidth;
+int awidth;
+int ascent;
+int descent;
+int underlinePosition;
+int underlineThickness;
+} MetricsRec, *MetricsPtr;
+
 typedef struct _Font {
 int numNames;
 struct _FontNameEntry *names;
@@ -94,8 +110,8 @@ typedef struct _Font {
 int weight; /* as in the OS/2 table */
 int width;  /* as in the OS/2 table */
 int italicAngle;/* degrees c-clockwise from the vertical */
-int underlinePosition;
-int underlineThickness;
+MetricsRec pxMetrics;
+MetricsRec metrics;
 unsigned foundry;
 struct _Strike *strikes;
 } FontRec, *FontPtr;
@@ -152,9 +168,9 @@ CmapPtr makeCmap(FontPtr);
 int findIndex(CmapPtr, int);
 int findCode(CmapPtr, int);
 BitmapPtr strikeBitmapIndex(StrikePtr, CmapPtr, int);
-void strikeMetrics(StrikePtr, int*, int*, int*, int*, int*);
+int strikeMaxWidth(StrikePtr);
 int glyphMetrics(FontPtr, int, int*, int*, int*, int*, int*);
-void fontMetrics(FontPtr, int*, int*, int*, int*, int*);
+void fontMetrics(FontPtr);
 int maxIndex(CmapPtr);
 
 int readFile(char *filename, FontPtr);

@@ -174,6 +190,7 @@ int macTime(int *, unsigned *);
 unsigned faceFoundry(FT_Face);
 char *faceEncoding(FT_Face);
 int faceFlags(FT_Face);
+int faceIntProp(FT_Face, const char *);
 int faceWeight(FT_Face);
 int faceWidth(FT_Face);
 int faceItalicAngle(FT_Face);
Index: read.c
===
RCS file: /cvs/xenocara/app/fonttosfnt/read.c,v
retrieving revision 1.4
diff -u -p -r1.4 read.c
--- read.c  14 Jan 2020 20:52:17 -  1.4
+++ read.c  9 Oct 2020 07:27:20 -
@@ -29,6 +29,7 @@ THE SOFTWARE.
 #include FT_FREETYPE_H
 #include FT_MODULE_H
 #include FT_BDF_H
+#include FT_FONT_FORMATS_H
 #include "X11/Xos.h"
 #include "fonttosfnt.h"
 #include "X11/fonts/fontenc.h"
@@ -102,7 +103,7 @@ readFile(char *filename, FontPtr font)
 BitmapPtr bitmap;
 int symbol = 0;
 int force_unicode = 1;
-char *encoding_name = NULL;
+const char *encoding_name, *file_format;
 FontMapPtr mapping = NULL;
 FontMapReversePtr reverse = NULL;
 
@@ -133,6 +134,15 @@ readFile(char *filename, FontPtr font)

 return -1;
 }
 
+file_format = FT_Get_Font_Format(face);

+if(strcmp(file_format, "BDF") != 0)
+   fprintf(stderr,
+   "font file %s is of format %s.\n"
+   "It's recommended to convert directly from a BDF font.\n"
+   "Some font properties may get lost 

Re: OpenSMTPD: unprivileged mode - now with diff

[Christopher Zimmermann]

On Sun, Apr 26, 2020 at 08:55:14AM +, gil...@poolp.org wrote:

April 26, 2020 10:34 AM, "Christopher Zimmermann"  wrote:


Hi,

I further developed my approach to allow running smtpd with fewer privileges. 
This diff does two
things:

- always run lmtp deliveries as SMTPD_USER. The change to mda_unpriv.c is 
needed, because otherwise
all mails would be delivered to SMTPD_USER.

- add two internal flags NOPRIV and NEEDPRIV. NOPRIV can be configured by the 
simple directive
"no-priv". NEEDPRIV gets set on all delivery methods / options requiring 
setuid() to run as the
receipient user.
A configuration error is produced on any conflict betweed NEEDPRIV and NOPRIV.
In case of a NOPRIV run smtpd will drop root privileges.
This will break .forward and alias filters.

The change to the lmtp delivery has benefits even without the second change. 
With the second change
my smtpd now runs without root privileges.
The NEEDPRIV/NOPRIV options are meant to allow restricting of the privileges of 
other delivery
methods.

I am now looking for OKs on the first change to do unprivileged lmtp deliveries 
and feedback on the
general approach of the second change.



The LMTP change seems interesting to me, it means that a broken LMTP delivery
will fail with _smtpd privileges instead of the (unprivileged) recipient user
so I think it's a good move.



Ok to commit the below change?

Christopher


Index: mda_unpriv.c
===
RCS file: /cvs/src/usr.sbin/smtpd/mda_unpriv.c,v
retrieving revision 1.6
diff -u -p -r1.6 mda_unpriv.c
--- mda_unpriv.c2 Feb 2020 22:13:48 -   1.6
+++ mda_unpriv.c23 May 2020 13:43:39 -
@@ -69,8 +69,8 @@ mda_unpriv(struct dispatcher *dsp, struc
xasprintf(_environ[idx++], "RECIPIENT=%s@%s", deliver->dest.user, 
deliver->dest.domain);
xasprintf(_environ[idx++], "SHELL=/bin/sh");
xasprintf(_environ[idx++], "LOCAL=%s", deliver->rcpt.user);
-   xasprintf(_environ[idx++], "LOGNAME=%s", pw_name);
-   xasprintf(_environ[idx++], "USER=%s", pw_name);
+   xasprintf(_environ[idx++], "LOGNAME=%s", 
deliver->userinfo.username);
+   xasprintf(_environ[idx++], "USER=%s", deliver->userinfo.username);

if (deliver->sender.user[0])
xasprintf(_environ[idx++], "SENDER=%s@%s",
Index: parse.y
===
RCS file: /cvs/src/usr.sbin/smtpd/parse.y,v
retrieving revision 1.277
diff -u -p -r1.277 parse.y
--- parse.y 24 Feb 2020 23:54:27 -  1.277
+++ parse.y 23 May 2020 13:43:40 -
@@ -690,10 +690,12 @@ MBOX {
 | LMTP STRING {
asprintf(>u.local.command,
"/usr/libexec/mail.lmtp -d %s -u", $2);
+   dispatcher->u.local.user = SMTPD_USER;
 } dispatcher_local_options
 | LMTP STRING RCPT_TO {
asprintf(>u.local.command,
"/usr/libexec/mail.lmtp -d %s -r", $2);
+   dispatcher->u.local.user = SMTPD_USER;
 } dispatcher_local_options
 | MDA STRING {
asprintf(>u.local.command,



--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Re: carp: send only IPv4 carp packets on dual stack interface

[Christopher Zimmermann]

On Sat, May 23, 2020 at 01:55:54PM +1000, David Gwynne wrote:




On 23 May 2020, at 8:44 am, Christopher Zimmermann  wrote:

On Sun, Jan 19, 2020 at 01:32:17PM +, Stuart Henderson wrote:

On 2020/01/19 00:11, Sebastian Benoit wrote:

chr...@openbsd.org(chr...@openbsd.org) on 2020.01.18 06:18:21 +0100:
> On Wed, Jan 15, 2020 at 12:47:28PM +0100, Sebastian Benoit wrote:
> >Christopher Zimmermann(chr...@openbsd.org) on 2020.01.15 11:55:43 +0100:
> >>Hi,
> >>
> >>as far as I can see a dual stack carp interface does not care whether it
> >>receives advertisements addressed to IPv4 or IPv6. Any one will do.
> >>So I propose to send IPv6 advertisements only when IPv4 is not possible.
> >>
> >>Why?
> >>
> >>- Noise can be reduced by using unicast advertisements.
> >>  This is only possible for IPv4 by ``ifconfig carppeer``.
> >>  I don't like flooding the whole network with carp advertisements when
> >>  I may also unicast them.
> >
> >Maybe i'm getting confused, but in the problem description you were talking
> >about v6 vs v4, and here you argue about unicast (vs multicast?) being
> >better. Thats orthogonal, isnt it?
>
> Yes, kind of. The point is we support ``carppeer`` for IPv4, but not for
> IPv6.
>
> >>- breaking IPv6 connectivity (for example by running iked without -6)
> >>  will start a preempt-war, because failing ip6_output will cause the
> >>  demote counter to be increased. That's what hit me.
> >
> >But the whole point of carp is to notice broken connectivity. If you run v6
> >on an interface, you want to know if its working, no?
>
> I grant you that much. But what kind of failures do you hope to detect
> on the _sending_ carp master, that would not also affect the backup?

sure: misconfigured pf. Missing routes. Buggy switch.


misconfigured mac address filter on switch.


I'm afraid you guys haven't yet got the point I'm trying to make.

Current behaviour is that in a dual-stack carp setup failover only happens when 
advertisements on _both_ AFs fail to reach the backup.
A node in backup state will stay in backup state as long as it receives _any_ 
advertisements.
In my mind this is the only sensible way for a backup node to react.

If a backup node that fails to receive advertisements of only one AF would 
transition to master it would in most cases start a preempt war.
So why do we even send dual-stack advertisements?
The only effect those dual-stack ipv6 advertisements currently have is that 
they prevent failover when ipv4 connectivity breaks.

I would propose to choose one "sentinel" AF (in this case ipv4) and failover 
whenever advertisements of this AF fail to reach the backup.

Monitoring multiple AFs is not helpful, because there is no good way in which 
to react to a failure that affects only one AF.


I don't know if this helps, but at work we use separate carp interfaces for v4 
and v6. It ends up looking a bit like this:

# cat /etc/hostname.vlan871:
parent aggr0 vnetid 871
inet alias 192.0.2.2/24
inet6 alias 2001:db8:871::2/64
up

# cat /etc/hostname.carp40871
carpdev vlan871 vhid 47
-inet6
-group carp
group ipv4g
inet alias 192.0.2.1/24
up

# cat /etc/hostname.carp60871
carpdev vlan871 vhid 61
-group carp
group ipv6g
inet6 alias 2001:db8:871::1/64
up

This let's us run a pair of firewalls one active for v4 and the other for v6. 
We don't do any af-to in PF, so it works pretty well. But yeah, it means v4 and 
v6 fail separately.


Yes that is a sensible and coherent setup where ipv4 and ipv6 are kept 
completely separate.
But what I'm proposing to change is the dual stack mode of carp where 
there is only one interface. There is no use for advertisements on two 
address families, because in case of failure of only one address family 
there is no sensible way to respond to that failure.


> >At the very least, this needs some more thought and testing in 
> >all > >the ways

> >carp can be configured.
>
> Anyway, my main concern indeed is the broadcast noise generated by carp
> and I would be equally happy if we had a ``carppeer6`` option. Would
> that be considered?

of course carppeer should work with v6, and as claudio says without an extra
keyword in ifconfig, but thats a trivial detail.



Currently carp only handles one address per af, setting carppeer twice
changes the current peer address rather than adding another. A trivial
implementation that sets the v4 peer address if a v4 address is passed
in, and sets the v6 peer address if a v6 address is passed in, that
would mean things work differently with

ifconfig carp1 carppeer $foo
ifconfig carp1 carppeer $bar

depending on whether foo/bar are v4 or v6. Also removing a configured
carppeer address to reset to multicast is just done with -carppeer
with no way to indicate the 

Re: carp: send only IPv4 carp packets on dual stack interface

[Christopher Zimmermann]

On Sun, Jan 19, 2020 at 01:32:17PM +, Stuart Henderson wrote:

On 2020/01/19 00:11, Sebastian Benoit wrote:

chr...@openbsd.org(chr...@openbsd.org) on 2020.01.18 06:18:21 +0100:
> On Wed, Jan 15, 2020 at 12:47:28PM +0100, Sebastian Benoit wrote:
> >Christopher Zimmermann(chr...@openbsd.org) on 2020.01.15 11:55:43 +0100:
> >>Hi,
> >>
> >>as far as I can see a dual stack carp interface does not care whether it
> >>receives advertisements addressed to IPv4 or IPv6. Any one will do.
> >>So I propose to send IPv6 advertisements only when IPv4 is not possible.
> >>
> >>Why?
> >>
> >>- Noise can be reduced by using unicast advertisements.
> >>  This is only possible for IPv4 by ``ifconfig carppeer``.
> >>  I don't like flooding the whole network with carp advertisements when
> >>  I may also unicast them.
> >
> >Maybe i'm getting confused, but in the problem description you were talking
> >about v6 vs v4, and here you argue about unicast (vs multicast?) being
> >better. Thats orthogonal, isnt it?
>
> Yes, kind of. The point is we support ``carppeer`` for IPv4, but not for
> IPv6.
>
> >>- breaking IPv6 connectivity (for example by running iked without -6)
> >>  will start a preempt-war, because failing ip6_output will cause the
> >>  demote counter to be increased. That's what hit me.
> >
> >But the whole point of carp is to notice broken connectivity. If you run v6
> >on an interface, you want to know if its working, no?
>
> I grant you that much. But what kind of failures do you hope to detect
> on the _sending_ carp master, that would not also affect the backup?

sure: misconfigured pf. Missing routes. Buggy switch.


misconfigured mac address filter on switch.


I'm afraid you guys haven't yet got the point I'm trying to make.

Current behaviour is that in a dual-stack carp setup failover only 
happens when advertisements on _both_ AFs fail to reach the backup.
A node in backup state will stay in backup state as long as it receives 
_any_ advertisements.

In my mind this is the only sensible way for a backup node to react.

If a backup node that fails to receive advertisements of only one AF 
would transition to master it would in most cases start a preempt war. 


So why do we even send dual-stack advertisements?
The only effect those dual-stack ipv6 advertisements currently have is 
that they prevent failover when ipv4 connectivity breaks.


I would propose to choose one "sentinel" AF (in this case ipv4) and 
failover whenever advertisements of this AF fail to reach the backup.


Monitoring multiple AFs is not helpful, because there is no good way in 
which to react to a failure that affects only one AF.


> >At the very least, this needs some more thought and testing in all 
> >the ways

> >carp can be configured.
>
> Anyway, my main concern indeed is the broadcast noise generated by carp
> and I would be equally happy if we had a ``carppeer6`` option. Would
> that be considered?

of course carppeer should work with v6, and as claudio says without an extra
keyword in ifconfig, but thats a trivial detail.



Currently carp only handles one address per af, setting carppeer twice
changes the current peer address rather than adding another. A trivial
implementation that sets the v4 peer address if a v4 address is passed
in, and sets the v6 peer address if a v6 address is passed in, that
would mean things work differently with

ifconfig carp1 carppeer $foo
ifconfig carp1 carppeer $bar

depending on whether foo/bar are v4 or v6. Also removing a configured
carppeer address to reset to multicast is just done with -carppeer
with no way to indicate the af.

It would work pretty nicely if you could set multiple carppeer addresses
(of whatever af) and remove them individually. That's a more complex
change (carp would need to keep a list of peers per af rather than a
single address) but without something like that they can't really be
equals and it feels like shoehorning both afs into the same keyword
will just be confusing.



--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Re: ADMtec aue(4) interface supporting VLAN_MTU ?

[Christopher Zimmermann]

On Tue, Apr 21, 2020 at 04:12:16PM -0700, Chris Cappuccio wrote:

Tom Smyth [tom.sm...@wirelessconnect.eu] wrote:

Hi Chrisz,

4 bytes for the vlan header .. have you tried increasing the parent
intetface mtu by 4bytes



IFCAP_VLAN_MTU is a direct bypass for this. "hardmtu" on the parent interface
is perhaps more interesting as it will limit everything including these
encapsulations


IFAP_VLAN_MTU will affect how the hardmtu is passed on to vlan child 
interfaces. The vlan interfaces won't care for the parent's "soft" mtu 
AFAICS:


hardmtu = ifp0->if_hardmtu;
if (!ISSET(ifp0->if_capabilities, IFCAP_VLAN_MTU))
hardmtu -= EVL_ENCAPLEN;

Linux uses a MTU of 1536 for Pegasus chips. We always default to a 
hardmtu of 1500. So the hardmtu can't be the cause for my interface not 
managing full-size vlan.


Christopher


--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Re: OpenSMTPD: unprivileged mode - now with diff

[Christopher Zimmermann]

On Sun, Apr 26, 2020 at 08:55:14AM +, gil...@poolp.org wrote:

April 26, 2020 10:34 AM, "Christopher Zimmermann"  wrote:

- always run lmtp deliveries as SMTPD_USER. The change to 
  mda_unpriv.c is needed, because otherwise

all mails would be delivered to SMTPD_USER.

- add two internal flags NOPRIV and NEEDPRIV. NOPRIV can be configured by the 
simple directive
"no-priv". NEEDPRIV gets set on all delivery methods / options requiring 
setuid() to run as the
receipient user.
A configuration error is produced on any conflict betweed NEEDPRIV and NOPRIV.
In case of a NOPRIV run smtpd will drop root privileges.
This will break .forward and alias filters.



Hi, thanks for your fast reply.


The LMTP change seems interesting to me, it means that a broken LMTP delivery
will fail with _smtpd privileges instead of the (unprivileged) recipient user
so I think it's a good move.


That's great. Is there anything that needs to change or be clarified 
before you can give an ok?



I'm less convinced by the other change and it doesn't only break .forward and
alias, it also breaks authentication, tables reloading, and probably stuff my
mind is not yet awake enough to think of.


Thanks for giving it a thought. I'm not entirely convinced either. But 
believe some thought should be given to it.
In your opinion would it be generaly a bad idea to try run smtpd without 
root privileges?

What exectly will cease to work?

- .forward and alias _filtering_ will break for sure.
  Forwarding won't break.

- Authentication: Authentication of system users will break. So I would 
  mark auth without auth-table as need_priv() in parse.y.


- tables reloading: this is a problem only for tables with restricted 
  read permissions, isn't it?
  I could try to figure out whether it is a problem in parse.y and mark 
  it as need_priv() if necessary, but one could also just document the 
  no-priv option with its limitations.


- other stuff: Can it be dealt with need_priv()? Will it lead to 
  unpleasant surprises for users?


The general idea is to allow running smtpd without root priviliges where 
possible and maybe try to change it to support running without root for 
more use-cases over time. Like try to pick the low-hanging fruits first.


One less low-hanging fruits would be to integrate mail.lmtp(8) into 
smtpd and move other mail.* functionality into a privileged lmtpd 
daemon.
This would remove those vulnerable mda-exec lines from the envelope 
files. That's just one idea how one could proceed. The current proposal 
is to start stripping privileges where possible.



Christopher


 
-- http://gmerlin.de

OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

OpenSMTPD: unprivileged mode - now with diff

[Christopher Zimmermann]

Hi,

I further developed my approach to allow running smtpd with fewer 
privileges. This diff does two things:


- always run lmtp deliveries as SMTPD_USER. The change to mda_unpriv.c 
  is needed, because otherwise all mails would be delivered to 
  SMTPD_USER.
- add two internal flags NOPRIV and NEEDPRIV. NOPRIV can be configured 
  by the simple directive "no-priv". NEEDPRIV gets set on all delivery 
  methods / options requiring setuid() to run as the receipient user.
  A configuration error is produced on any conflict betweed NEEDPRIV and 
  NOPRIV.

  In case of a NOPRIV run smtpd will drop root privileges.
  This will break .forward and alias filters.

The change to the lmtp delivery has benefits even without the second 
change. With the second change my smtpd now runs without root 
privileges.
The NEEDPRIV/NOPRIV options are meant to allow restricting of the 
privileges of other delivery methods.


I am now looking for OKs on the first change to do unprivileged lmtp 
deliveries and feedback on the general approach of the second change.



Christopher



Index: mda_unpriv.c
===
RCS file: /cvs/src/usr.sbin/smtpd/mda_unpriv.c,v
retrieving revision 1.6
diff -u -p -r1.6 mda_unpriv.c
--- mda_unpriv.c2 Feb 2020 22:13:48 -   1.6
+++ mda_unpriv.c26 Apr 2020 05:27:34 -
@@ -69,8 +69,8 @@ mda_unpriv(struct dispatcher *dsp, struc
xasprintf(_environ[idx++], "RECIPIENT=%s@%s", deliver->dest.user, 
deliver->dest.domain);
xasprintf(_environ[idx++], "SHELL=/bin/sh");
xasprintf(_environ[idx++], "LOCAL=%s", deliver->rcpt.user);
-   xasprintf(_environ[idx++], "LOGNAME=%s", pw_name);
-   xasprintf(_environ[idx++], "USER=%s", pw_name);
+   xasprintf(_environ[idx++], "LOGNAME=%s", 
deliver->userinfo.username);
+   xasprintf(_environ[idx++], "USER=%s", deliver->userinfo.username);
 
 	if (deliver->sender.user[0])

xasprintf(_environ[idx++], "SENDER=%s@%s",
Index: parse.y
===
RCS file: /cvs/src/usr.sbin/smtpd/parse.y,v
retrieving revision 1.277
diff -u -p -r1.277 parse.y
--- parse.y 24 Feb 2020 23:54:27 -  1.277
+++ parse.y 26 Apr 2020 05:27:35 -
@@ -154,6 +154,7 @@ static int  host_v4(struct listen_opts *)
 static int host_v6(struct listen_opts *);
 static int host_dns(struct listen_opts *);
 static int interface(struct listen_opts *);
+static voidneed_priv(const char *);
 
 int		 delaytonum(char *);

 int is_if_in_group(const char *, const char *);
@@ -186,7 +187,7 @@ typedef struct {
 %token KEY
 %token LIMIT LISTEN LMTP LOCAL
 %token MAIL_FROM MAILDIR MASK_SRC MASQUERADE MATCH MAX_MESSAGE_SIZE 
MAX_DEFERRED MBOX MDA MTA MX
-%token NO_DSN NO_VERIFY NOOP
+%token NO_DSN NO_PRIV NO_VERIFY NOOP
 %token ON
 %token PHASE PKI PORT PROC PROC_EXEC PROXY_V2
 %token QUEUE QUIT
@@ -212,6 +213,7 @@ grammar : /* empty */
| grammar ca '\n'
| grammar mda '\n'
| grammar mta '\n'
+   | grammar privs '\n'
| grammar pki '\n'
| grammar proc '\n'
| grammar queue '\n'
@@ -379,6 +381,20 @@ MTA MAX_DEFERRED NUMBER  {
 ;
 
 
+privs:

+NO_PRIV {
+   if (conf->sc_opts & SMTPD_OPT_NEEDPRIV) {
+   yyerror("Unprivileged operation is not possible.");
+   YYERROR;
+   }
+   else {
+   log_warnx("Unprivileged operation requested.");
+   conf->sc_opts |= SMTPD_OPT_NOPRIV;
+   }
+}
+;
+
+
 pki:
 PKI STRING {
char buf[HOST_NAME_MAX+1];
@@ -566,6 +582,8 @@ SRS KEY STRING {
 
 dispatcher_local_option:

 USER STRING {
+   need_priv("with user");
+
if (dispatcher->u.local.is_mbox) {
yyerror("user may not be specified for this dispatcher");
YYERROR;
@@ -662,16 +680,20 @@ dispatcher_local_option dispatcher_local
 
 dispatcher_local:

 MBOX {
+   need_priv("mbox");
dispatcher->u.local.is_mbox = 1;
asprintf(>u.local.command, "/usr/libexec/mail.local -f 
%%{mbox.from} -- %%{user.username}");
 } dispatcher_local_options
 | MAILDIR {
+   need_priv("maildir");
asprintf(>u.local.command, "/usr/libexec/mail.maildir");
 } dispatcher_local_options
 | MAILDIR JUNK {
+   need_priv("maildir");
asprintf(>u.local.command, "/usr/libexec/mail.maildir -j");
 } dispatcher_local_options
 | MAILDIR STRING {
+   need_priv("maildir");
if (strncmp($2, "~/", 2) == 0)
asprintf(>u.local.command,
"/usr/libexec/mail.maildir \"%%{user.directory}/%s\"", 
$2+2);
@@ -680,6 +702,7 @@ MBOX {
"/usr/libexec/mail.maildir \"%s\"", $2);
 } dispatcher_local_options
 | MAILDIR STRING JUNK {
+   need_priv("maildir");
if (strncmp($2, "~/", 2) == 0)
asprintf(>u.local.command,
  

OpenSMTPD: unprivileged mode

[Christopher Zimmermann]

Hi,

I further developed my approach to allow running smtpd with fewer 
privileges. This diff does two things:


- always run lmtp deliveries as SMTPD_USER. The change to mda_unpriv.c 
  is needed, because otherwise all mails would be delivered to 
  SMTPD_USER.
- add two internal flags NOPRIV and NEEDPRIV. NOPRIV can be configured 
  by the simple directive "no-priv". NEEDPRIV gets set on all delivery 
  methods / options requiring setuid() to run as the receipient user.
  A configuration error is produced on any conflict betweed NEEDPRIV and 
  NOPRIV.

  In case of a NOPRIV run smtpd will drop root privileges.
  This will break .forward and alias filters.

The change to the lmtp delivery has benefits even without the second 
change. With the second change my smtpd now runs without root 
privileges.
The NEEDPRIV/NOPRIV options are meant to allow restricting of the 
privileges of other delivery methods.


I am now looking for OKs on the first change to do unprivileged lmtp 
deliveries and feedback on the general approach of the second change.



Christopher


--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Re: ADMtec aue(4) interface supporting VLAN_MTU ?

[Christopher Zimmermann]

On Sun, Jul 07, 2019 at 01:26:55PM -0700, Chris Cappuccio wrote:

Christopher Zimmermann [chr...@openbsd.org] wrote:

This works:

doas ifconfig vlan67 mtu 1496

this doesn't:

doas ifconfig vlan67 mtu 1497


Should we therefore disable VLAN_MTU on this chipset?

-   ifp->if_capabilities = IFCAP_VLAN_MTU;
-



Yes absolutely. If IFCAP_VLAN_MTU actually worked for some chip versions,
more work is needed to distinguish the working chipsets.


Hi,

I'm looking for reponts of aue(4) chipsets actually supporting VLAN_MTU. 
Mine does not. It identifies as:


aue0 at uhub9 port 5 configuration 1 interface 0 "ADMtek USB To LAN Converter" 
rev 2.00/1.01 addr 5
aue0: address 00:05:1b:b2:96:02
ukphy0 at aue0 phy 1: Generic IEEE 802.3u media interface, rev. 1: OUI 
0x000749, model 0x0001

In case there are none, I would like to drop above mentioned 
IFCAP_VLAN_MTU flag for the aue(4) driver and look for OKs.



Christopher


--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

Re: carp: send only IPv4 carp packets on dual stack interface

[Christopher Zimmermann]

On January 15, 2020 12:04:56 PM GMT+01:00, Janne Johansson 
 wrote:
>Den ons 15 jan. 2020 kl 11:57 skrev Christopher Zimmermann <
>chr...@openbsd.org>:
>
>> So I propose to send IPv6 advertisements only when IPv4 is not
>possible.
>>
>> Why?
>>
>> - Noise can be reduced by using unicast advertisements.
>>This is only possible for IPv4 by ``ifconfig carppeer``.
>>I don't like flooding the whole network with carp advertisements
>when
>>I may also unicast them.
>>
>
>This sentence seems to assume people would only ever use two members in
>a
>carp group

Well, even though it may have seemed like it, I did not assume this.

My concern is more about sending out through two network stacks, one of which 
(IPv6) may fail and produce unwanted effects in carp's demote logic.
Is there something to gain by sending out though both stacks?
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

carp: send only IPv4 carp packets on dual stack interface

[Christopher Zimmermann]

Hi,

as far as I can see a dual stack carp interface does not care whether it 
receives advertisements addressed to IPv4 or IPv6. Any one will do.

So I propose to send IPv6 advertisements only when IPv4 is not possible.

Why?

- Noise can be reduced by using unicast advertisements.
  This is only possible for IPv4 by ``ifconfig carppeer``.
  I don't like flooding the whole network with carp advertisements when 
  I may also unicast them.
- breaking IPv6 connectivity (for example by running iked without -6) 
  will start a preempt-war, because failing ip6_output will cause the 
  demote counter to be increased. That's what hit me.


I would suggest a change like this:

Index: ip_carp.c
===
RCS file: /cvs/src/sys/netinet/ip_carp.c,v
retrieving revision 1.342
diff -u -p -r1.342 ip_carp.c
--- ip_carp.c   8 Nov 2019 07:51:41 -   1.342
+++ ip_carp.c   15 Jan 2020 10:45:56 -
@@ -1175,7 +1175,7 @@ carp_send_ad(struct carp_vhost_entry *vh
}
}
 #ifdef INET6
-   if (sc->sc_naddrs6) {
+   else if (sc->sc_naddrs6) {
struct ip6_hdr *ip6;
 
 		MGETHDR(m, M_DONTWAIT, MT_HEADER);



one could also use a slightly smaller hammer and only avoid sending IPv6 
if the user requested an IPv4 unicast address:


-   if (sc->sc_naddrs6) {
+   if (sc->sc_naddrs6 &&
+   (! sc->sc_naddrs ||
+   sc->sc_peer.s_addr == INADDR_CARP_GROUP) ) {


Christopher


--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

/usr/bin/ftp: 308 Permanent Redirect

[Christopher Zimmermann]

Hi,

to me it seems that since our ftp does only support GET requests anyway, 
a 308 Redirect response can be handled identical to a 301 Redirect. OK?


Christopher


Index: fetch.c
===
RCS file: /cvs/src/usr.bin/ftp/fetch.c,v
retrieving revision 1.184
diff -u -p -r1.184 fetch.c
--- fetch.c 9 Dec 2019 19:05:06 -   1.184
+++ fetch.c 14 Dec 2019 11:10:09 -
@@ -834,6 +834,7 @@ noslash:
case 302:   /* Found */
case 303:   /* See Other */
case 307:   /* Temporary Redirect */
+   case 308:   /* Moved Permanently */
isredirect++;
if (redirect_loop++ > 10) {
warnx("Too many redirections requested");


--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1


signature.asc
Description: PGP signature

/bin/cp: overwrite symlink with file pointed to by symlink

[Christopher Zimmermann]

Hi,

I admit this is a very special case, but anyway this is what I hit:

$ touch blub; ln -s blub blab; ls -l; cp -f blub blab
total 0
lrwxr-xr-x  1 madroach  wheel  4 Oct 30 17:39 blab -> blub
-rw-r--r--  1 madroach  wheel  0 Oct 30 17:39 blub
cp: blab and blub are identical (not copied).

Now I had a look at our cp.c, our cp(1) and at POSIX:

 -f For each existing destination pathname, remove it and create a
new file, without prompting for confirmation, regardless of its
permissions.  The -f option overrides any previous -i options.

POSIX says cp must always first try to open(2)(O_TRUNC) the destination
and only if this fails AND -f was specified should it try unlink(2).

At least regarding this special case our cp behaves according to
POSIX, not according to our manpage.

As long as we cannot change the POSIX standard I don't know whether our
manpage should mention this or cp.c be changed to use lstat(), too.
Maybe this is such a special case we could just ignore it...


Christopher


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1


signature.asc
Description: PGP signature

Re: apmd battery emergency message

[Christopher Zimmermann]

On Fri, 20 Sep 2019 17:00:39 +0200
Alexander Bluhm  wrote:

> Hi,
> 
> sometimes my laptop was running out of battery while I was working.
> To avoid that, I patched apmd(8) to write a emergency message to
> syslog(3).  Then with this line in syslog.conf I receive a warning
> in every xterm.
> 
> # Everyone gets emergency messages.
> *.emerg *
> 
> So I have enough time to find a power supply.

isn't this a job for sensorsd ?

Christopher

> 
> Index: usr.sbin/apmd/apmd.c
> ===
> RCS file: /data/mirror/openbsd/cvs/src/usr.sbin/apmd/apmd.c,v
> retrieving revision 1.88
> diff -u -p -r1.88 apmd.c
> --- usr.sbin/apmd/apmd.c  25 Jul 2019 07:29:44 -  1.88
> +++ usr.sbin/apmd/apmd.c  26 Jul 2019 08:09:44 -
> @@ -145,7 +145,7 @@ power_status(int fd, int force, struct a
>  {
>   struct apm_power_info bstate;
>   static struct apm_power_info last;
> - int acon = 0;
> + int acon = 0, priority = LOG_NOTICE;
> 
>   if (fd == -1) {
>   if (pinfo) {
> @@ -164,6 +164,9 @@ power_status(int fd, int force, struct a
>* enough since last report, or asked to force a print */
>   if (bstate.ac_state == APM_AC_ON)
>   acon = 1;
> + if (bstate.battery_state == APM_BATT_CRITICAL &&
> + bstate.battery_state != last.battery_state)
> + priority = LOG_EMERG;
>   if (force ||
>   bstate.ac_state != last.ac_state ||
>   bstate.battery_state != last.battery_state ||
> @@ -182,7 +185,7 @@ power_status(int fd, int force, struct a
>  #else
>   if ((int)bstate.minutes_left > 0)
>  #endif
> - logmsg(LOG_NOTICE, "battery status:
> %s. "
> + logmsg(priority, "battery status:
> %s. " "external power status: %s. "
>   "estimated battery life %d%% (%u
> minutes)", battstate(bstate.battery_state),
> @@ -190,7 +193,7 @@ power_status(int fd, int force, struct a
>   bstate.battery_life,
>   bstate.minutes_left);
>   else
> - logmsg(LOG_NOTICE, "battery status:
> %s. "
> + logmsg(priority, "battery status:
> %s. " "external power status: %s. "
>   "estimated battery life %d%%",
>   battstate(bstate.battery_state),
> 



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1


pgpUG0ZvriH3E.pgp
Description: OpenPGP digital signature

grep -ob: behave like ggrep, adapt documentation

[Christopher Zimmermann]

Hi,

I just tried to find the byte position of a string within a binary file
using grep. Our base grep -bo let me down because it will only display
the position of the '\n' delimited line, not the position of the
pattern.

That's what our grep(1) says:
-b  The offset in bytes of a matched pattern is displayed in
front of the respective matched line.

what it actually does is displaying the offset of the line, not the
pattern. I think it should read as following:

-b  Each output line is preceded by its position (in bytes) in the
file.  If option -o is also specified, the offset in bytes of
the actual matched pattern is displayed.


and here's the diff. Regression tests still pass. OK?

Christopher



Index: grep.1
===
RCS file: /cvs/src/usr.bin/grep/grep.1,v
retrieving revision 1.45
diff -u -p -r1.45 grep.1
--- grep.1  10 Dec 2017 09:17:24 -  1.45
+++ grep.1  16 Jul 2019 21:16:18 -
@@ -131,8 +131,11 @@ and
 .Fl C
 options.
 .It Fl b
-The offset in bytes of a matched pattern is
-displayed in front of the respective matched line.
+Each output line is preceded by its position (in bytes) in the file.
+If option
+.Fl o
+is also specified, the offset in bytes of the actual matched pattern is
+displayed.
 .It Fl C Ns Oo Ar num Oc , Fl -context Ns Op = Ns Ar num
 Print
 .Ar num
Index: util.c
===
RCS file: /cvs/src/usr.bin/grep/util.c,v
retrieving revision 1.59
diff -u -p -r1.59 util.c
--- util.c  23 Jan 2019 23:00:54 -  1.59
+++ util.c  16 Jul 2019 21:16:18 -
@@ -250,9 +250,9 @@ print:
if (Bflag > 0)
printqueue();
linesqueued = 0;
-   printline(l, ':', oflag ?  : NULL);
+   printline(l, ':', );
} else {
-   printline(l, '-', oflag ?  : NULL);
+   printline(l, '-', );
tail--;
}
}
@@ -651,12 +651,14 @@ printline(str_t *line, int sep, regmatch
if (bflag) {
if (n)
putchar(sep);
-   printf("%lld", (long long)line->off);
+   printf("%lld",
+   (long long)line->off
+   + (pmatch && oflag ? pmatch->rm_so : 0));
++n;
}
if (n)
putchar(sep);
-   if (pmatch)
+   if (oflag && pmatch)
fwrite(line->dat + pmatch->rm_so,
pmatch->rm_eo - pmatch->rm_so, 1, stdout);
else


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1

-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1


pgp36UztoKX1k.pgp
Description: OpenPGP digital signature

ADMtec aue interface does not work with full 1500 VLAN_MTU

[Christopher Zimmermann]

Hi,

I own this old USB ethernet controller:

aue0 at uhub9 port 5 configuration 1 interface 0 "ADMtek USB To LAN Converter" 
rev 2.00/1.01 addr 4
aue0: address 00:05:1b:b2:96:02
ukphy0 at aue0 phy 1: Generic IEEE 802.3u media interface, rev. 1: OUI 
0x000749, model 0x0001

This works:

doas ifconfig vlan67 mtu 1496

this doesn't:

doas ifconfig vlan67 mtu 1497


Should we therefore disable VLAN_MTU on this chipset?


Christopher



Index: if_aue.c
===
RCS file: /cvs/src/sys/dev/usb/if_aue.c,v
retrieving revision 1.109
diff -u -p -r1.109 if_aue.c
--- if_aue.c2 Oct 2018 19:49:10 -   1.109
+++ if_aue.c4 Jul 2019 18:25:06 -
@@ -781,8 +781,6 @@ aue_attach(struct device *parent, struct
ifp->if_watchdog = aue_watchdog;
strlcpy(ifp->if_xname, sc->aue_dev.dv_xname, IFNAMSIZ);
 
-   ifp->if_capabilities = IFCAP_VLAN_MTU;
-
/* Initialize MII/media info. */
mii = >aue_mii;
mii->mii_ifp = ifp;


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
CB07 DA40 B0B6 571D 35E2  0DEF 87E2 92A7 13E5 DEE1


pgphJJK9uAJdt.pgp
Description: OpenPGP digital signature

Re: Don't send icmp redirect to the same interface a forwarded packet came in on

[Christopher Zimmermann]

On 2018-05-05 Martin Pieuchot <m...@openbsd.org> wrote:
> On 03/05/18(Thu) 17:19, Christopher Zimmermann wrote:
> > On 2018-05-03 Martin Pieuchot <m...@openbsd.org> wrote:  
> > > On 02/05/18(Wed) 14:45, Christopher Zimmermann wrote:  
> > > > On 2018-05-02 Martin Pieuchot <m...@openbsd.org> wrote:    
> > > > > On 02/05/18(Wed) 11:47, Christopher Zimmermann wrote:
> > > > > > I just want to bring this up again. Can some network guru give me 
> > > > > > an ok
> > > > > > or some feedback please?  
> > > > > 
> > > > > Can you explain with words why we shouldn't send a redirect?  The
> > > > > comment above your diff states clearly:
> > > > > 
> > > > >   "If forwarding packet using same interface that it came in on,
> > > > >   perhaps should send a redirect to sender to shortcut a hop."
> > > > > 
> > > > > So you're suggesting no to do that, why?
> > > > 
> > > > That's not exactly what I'm suggesting.
> > > > 
> > > > In this setting:
> > > > 
> > > > A 192.168.4.7 <--> 192.168.4.1 Gateway 192.168.1.1 <--> 192.168.1.2 B
> > > > 
> > > > I observed this senseless redirect:
> > > > 
> > > > 192.168.4.1 > 192.168.4.7: icmp: redirect 192.168.1.2 to host 
> > > > 192.168.4.1
> > > > in plain language it means:
> > > > "Hi 192.168.4.7, I'm 192.168.4.1. You sent me a packet for 192.168.1.2.
> > > > I'm not the best route, next time send it to 192.168.4.1."
> > > > So the gateway is instructing host 192.168.4.7 to use gateway
> > > > 192.168.4.1 instead of 192.168.4.1. (this is not a typo!)
> > > 
> > > How does your routing table looks like?  
> > 
> > % doas ifconfig vlan2   
> >   
> > vlan2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500  
> > 
> >   
> > index 7 priority 0 llprio 3
> > encap: vnetid 2 parent em2
> > status: active
> > inet 192.168.4.1 netmask 0xfff0 broadcast 192.168.4.15
> > % doas route -n show -inet
> > Routing tables
> > 
> > Internet:
> > DestinationGatewayFlags   Refs  Use   Mtu  Prio 
> > Iface
> > default62.27.93.143   UGS563324 - 8 
> > pppoe0
> > 224/4  127.0.0.1  URS00 32768 8 lo0
> > 62.27.93.143   85.212.225.8   UHh11 - 8 
> > pppoe0
> > 85.212.225.8   85.212.225.8   UHl0 1589 - 1 
> > pppoe0
> > 127/8  127.0.0.1  UGRS   00 32768 8 lo0
> > 127.0.0.1  127.0.0.1  UHhl   9 1131 32768 1 lo0
> > 192.168.0/22   192.168.4.1UGS0  314 - 8 
> > vlan2 <- this is the culprit
> > 192.168.4.0/28 192.168.4.1UCn20 - 4 
> > vlan2
> > 
> > You'll note I manually (Flag S) added this rather senseless route to gateway
> > 192.268.4.1 (Flag G), which is the very same machine.  
> 
> See, that's why the redirect message doesn't make sense.  Because it
> uses the info from this route.  Now What I'm not getting is where did
> you configured 192.168.1.1?

It's the IP of the other end of the IPsec tunnel.


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
2779 7F73 44FD 0736 B67A  C410 69EC 7922 34B4 2566

Re: Don't send icmp redirect to the same interface a forwarded packet came in on

[Christopher Zimmermann]

On 2018-05-03 Martin Pieuchot <m...@openbsd.org> wrote:
> On 02/05/18(Wed) 14:45, Christopher Zimmermann wrote:
> > On 2018-05-02 Martin Pieuchot <m...@openbsd.org> wrote:  
> > > On 02/05/18(Wed) 11:47, Christopher Zimmermann wrote:  
> > > > I just want to bring this up again. Can some network guru give me an ok
> > > > or some feedback please?
> > > 
> > > Can you explain with words why we shouldn't send a redirect?  The
> > > comment above your diff states clearly:
> > > 
> > >   "If forwarding packet using same interface that it came in on,
> > >   perhaps should send a redirect to sender to shortcut a hop."
> > > 
> > > So you're suggesting no to do that, why?  
> > 
> > That's not exactly what I'm suggesting.
> > 
> > In this setting:
> > 
> > A 192.168.4.7 <--> 192.168.4.1 Gateway 192.168.1.1 <--> 192.168.1.2 B
> > 
> > I observed this senseless redirect:
> > 
> > 192.168.4.1 > 192.168.4.7: icmp: redirect 192.168.1.2 to host 192.168.4.1
> > in plain language it means:
> > "Hi 192.168.4.7, I'm 192.168.4.1. You sent me a packet for 192.168.1.2.
> > I'm not the best route, next time send it to 192.168.4.1."
> > So the gateway is instructing host 192.168.4.7 to use gateway
> > 192.168.4.1 instead of 192.168.4.1. (this is not a typo!)  
> 
> How does your routing table looks like?

% doas ifconfig vlan2   
  
vlan2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500  
  
index 7 priority 0 llprio 3
encap: vnetid 2 parent em2
status: active
inet 192.168.4.1 netmask 0xfff0 broadcast 192.168.4.15
% doas route -n show -inet
Routing tables

Internet:
DestinationGatewayFlags   Refs  Use   Mtu  Prio Iface
default62.27.93.143   UGS563324 - 8 pppoe0
224/4  127.0.0.1  URS00 32768 8 lo0
62.27.93.143   85.212.225.8   UHh11 - 8 pppoe0
85.212.225.8   85.212.225.8   UHl0 1589 - 1 pppoe0
127/8  127.0.0.1  UGRS   00 32768 8 lo0
127.0.0.1  127.0.0.1  UHhl   9 1131 32768 1 lo0
192.168.0/22   192.168.4.1UGS0  314 - 8 vlan2   
  <- this is the culprit
192.168.4.0/28 192.168.4.1UCn20 - 4 vlan2

You'll note I manually (Flag S) added this rather senseless route to gateway
192.268.4.1 (Flag G), which is the very same machine.
Why did I do this?
To prevent leakage of packets which should be routed via IPsec. Without this
pseudo-route packets from localhost to 192.168.0/22 would be routed via default
route on pppoe0 and have the src IP masqueraded by nat-to 85.212.225.8,
which would not be caught by the IPsec flow.

If I'm doing something wrong here and should have solved this another way
please tell me.
Using a "flow esp in from 0.0.0.0/0" would work on the local side, but the
remote side would be required to accept packets from arbitrary IPs, too.


Christopher


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
2779 7F73 44FD 0736 B67A  C410 69EC 7922 34B4 2566

Re: Don't send icmp redirect to the same interface a forwarded packet came in on

[Christopher Zimmermann]

On 2018-05-02 Martin Pieuchot <m...@openbsd.org> wrote:
> On 02/05/18(Wed) 11:47, Christopher Zimmermann wrote:
> > I just want to bring this up again. Can some network guru give me an ok
> > or some feedback please?  
> 
> Can you explain with words why we shouldn't send a redirect?  The
> comment above your diff states clearly:
> 
>   "If forwarding packet using same interface that it came in on,
>   perhaps should send a redirect to sender to shortcut a hop."
> 
> So you're suggesting no to do that, why?

That's not exactly what I'm suggesting.

In this setting:

A 192.168.4.7 <--> 192.168.4.1 Gateway 192.168.1.1 <--> 192.168.1.2 B

I observed this senseless redirect:

192.168.4.1 > 192.168.4.7: icmp: redirect 192.168.1.2 to host 192.168.4.1
in plain language it means:
"Hi 192.168.4.7, I'm 192.168.4.1. You sent me a packet for 192.168.1.2.
I'm not the best route, next time send it to 192.168.4.1."
So the gateway is instructing host 192.168.4.7 to use gateway
192.168.4.1 instead of 192.168.4.1. (this is not a typo!)

I'm suggesting to skip the redirect when we would redirect the sender
to an(other) address of the very interface the original packet came in
on.

I hope this explanation is reasonable.


Christopher


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
2779 7F73 44FD 0736 B67A  C410 69EC 7922 34B4 2566

Re: Don't send icmp redirect to the same interface a forwarded packet came in on

[Christopher Zimmermann]

Hi,

I just want to bring this up again. Can some network guru give me an ok
or some feedback please?

Christopher


On 2017-12-01 Christopher Zimmermann <chr...@openbsd.org> wrote:
> Hi,
> 
> by accident I discovered this rather senseless redirect:
> 
> $ doas tcpdump -eptni vlan2 icmp 
> tcpdump: listening on vlan2, link-type EN10MB
> 11:11:11:11:11:11 22:22:22:22:22 0800 98: 192.168.1.2 > 192.168.4.7: icmp: 
> echo request
> 22:22:22:22:22 11:11:11:11:11:11 0800 98: 192.168.4.7 > 192.168.1.2: icmp: 
> echo reply
> 11:11:11:11:11:11 22:22:22:22:22 0800 70: 192.168.4.1 > 192.168.4.7: icmp: 
> redirect 192.168.1.2 to host 192.168.4.1
> ^C
> 110 packets received by filter
> 0 packets dropped by kernel


[...]


> In any case I'd propose the following diff which will check whether the
> redirect target is an address of the interface the forwarded packet came in 
> on.
> 
> 
> Christopher
> 
> 
> Index: ip_input.c
> ===
> RCS file: /cvs/src/sys/netinet/ip_input.c,v
> retrieving revision 1.322
> diff -u -p -r1.322 ip_input.c
> --- ip_input.c7 Sep 2017 10:54:49 -   1.322
> +++ ip_input.c1 Dec 2017 18:00:53 -
> @@ -1514,16 +1514,27 @@ ip_forward(struct mbuf *m, struct ifnet 
>   (rt->rt_flags & (RTF_DYNAMIC|RTF_MODIFIED)) == 0 &&
>   satosin(rt_key(rt))->sin_addr.s_addr != 0 &&
>   ipsendredirects && !srcrt &&
> - !arpproxy(satosin(rt_key(rt))->sin_addr, m->m_pkthdr.ph_rtableid)) {
> - if ((ip->ip_src.s_addr & ifatoia(rt->rt_ifa)->ia_netmask) ==
> - ifatoia(rt->rt_ifa)->ia_net) {
> - if (rt->rt_flags & RTF_GATEWAY)
> + !arpproxy(satosin(rt_key(rt))->sin_addr, m->m_pkthdr.ph_rtableid) &&
> + (ip->ip_src.s_addr & ifatoia(rt->rt_ifa)->ia_netmask) ==
> + ifatoia(rt->rt_ifa)->ia_net) {
> + struct ifaddr *ifa;
> +
> + if (rt->rt_flags & RTF_GATEWAY)
>   dest = satosin(rt->rt_gateway)->sin_addr.s_addr;
> - else
> + else
>   dest = ip->ip_dst.s_addr;
> - /* Router requirements says to only send host redirects */
> - type = ICMP_REDIRECT;
> - code = ICMP_REDIRECT_HOST;
> +
> + /* don't redirect to the interface the packet came in on. */
> + TAILQ_FOREACH(ifa, >if_addrlist, ifa_list) {
> + if (ifa->ifa_addr->sa_family == AF_INET &&
> + satosin(ifa->ifa_addr)->sin_addr.s_addr == dest)
> + dest = 0;
> + }
> +
> + /* Router requirements says to only send host redirects */
> + if (dest != 0) {
> + type = ICMP_REDIRECT;
> + code = ICMP_REDIRECT_HOST;
>   }
>   }
> 
>  



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
2779 7F73 44FD 0736 B67A  C410 69EC 7922 34B4 2566

Don't send icmp redirect to the same interface a forwarded packet came in on

[Christopher Zimmermann]

Hi,

by accident I discovered this rather senseless redirect:

$ doas tcpdump -eptni vlan2 icmp 
tcpdump: listening on vlan2, link-type EN10MB
11:11:11:11:11:11 22:22:22:22:22 0800 98: 192.168.1.2 > 192.168.4.7: icmp: echo 
request
22:22:22:22:22 11:11:11:11:11:11 0800 98: 192.168.4.7 > 192.168.1.2: icmp: echo 
reply
11:11:11:11:11:11 22:22:22:22:22 0800 70: 192.168.4.1 > 192.168.4.7: icmp: 
redirect 192.168.1.2 to host 192.168.4.1
^C
110 packets received by filter
0 packets dropped by kernel


The routing table on 192.168.4.1 tells you why this redirect was sent.
Have a look at the second last route.

% doas ifconfig vlan2   
  
vlan2: flags=8843 mtu 1500  
  
lladdr 11:11:11:11:11:11
index 7 priority 0 llprio 3
encap: vnetid 2 parent em2
status: active
inet 192.168.4.1 netmask 0xfff0 broadcast 192.168.4.15
% doas route -n show -inet
Routing tables

Internet:
DestinationGatewayFlags   Refs  Use   Mtu  Prio Iface
default62.27.93.143   UGS563324 - 8 pppoe0
224/4  127.0.0.1  URS00 32768 8 lo0
62.27.93.143   85.212.225.8   UHh11 - 8 pppoe0
85.212.225.8   85.212.225.8   UHl0 1589 - 1 pppoe0
127/8  127.0.0.1  UGRS   00 32768 8 lo0
127.0.0.1  127.0.0.1  UHhl   9 1131 32768 1 lo0
192.168.0/22   192.168.4.1UGS0  314 - 8 vlan2   
  <- this is the culprit
192.168.4.0/28 192.168.4.1UCn20 - 4 vlan2


Network 192.268.0/22 is remote and connected via IPsec:
$ ipsecctl -s flow
flow esp in from 192.168.0.0/22 to 192.168.4.0/24 type use  
  
flow esp out from 192.168.224.0/24 to 192.168.0.0/17 type require   


You'll note I manually (Flag S) added this rather senseless route to gateway
192.268.4.1 (Flag G), which is the very same machine.
Why did I do this?
To prevent leakage of packets which should be routed via IPsec. Without this
pseudo-route packets from localhost to 192.168.0/22 would be routed via default
route on pppoe0 and have a src IP of 85.212.225.8,
which would not be caught by the IPsec flow.

If I'm doing something wrong here and should have solved this an another way
please tell me.
Using a "flow esp in from 0.0.0.0/0" would work on the local side, but the
remote side would be required to accept packets from arbitrary IPs, too.

In any case I'd propose the following diff which will check whether the
redirect target is an address of the interface the forwarded packet came in on.


Christopher


Index: ip_input.c
===
RCS file: /cvs/src/sys/netinet/ip_input.c,v
retrieving revision 1.322
diff -u -p -r1.322 ip_input.c
--- ip_input.c  7 Sep 2017 10:54:49 -   1.322
+++ ip_input.c  1 Dec 2017 18:00:53 -
@@ -1514,16 +1514,27 @@ ip_forward(struct mbuf *m, struct ifnet 
(rt->rt_flags & (RTF_DYNAMIC|RTF_MODIFIED)) == 0 &&
satosin(rt_key(rt))->sin_addr.s_addr != 0 &&
ipsendredirects && !srcrt &&
-   !arpproxy(satosin(rt_key(rt))->sin_addr, m->m_pkthdr.ph_rtableid)) {
-   if ((ip->ip_src.s_addr & ifatoia(rt->rt_ifa)->ia_netmask) ==
-   ifatoia(rt->rt_ifa)->ia_net) {
-   if (rt->rt_flags & RTF_GATEWAY)
+   !arpproxy(satosin(rt_key(rt))->sin_addr, m->m_pkthdr.ph_rtableid) &&
+   (ip->ip_src.s_addr & ifatoia(rt->rt_ifa)->ia_netmask) ==
+   ifatoia(rt->rt_ifa)->ia_net) {
+   struct ifaddr *ifa;
+
+   if (rt->rt_flags & RTF_GATEWAY)
dest = satosin(rt->rt_gateway)->sin_addr.s_addr;
-   else
+   else
dest = ip->ip_dst.s_addr;
-   /* Router requirements says to only send host redirects */
-   type = ICMP_REDIRECT;
-   code = ICMP_REDIRECT_HOST;
+
+   /* don't redirect to the interface the packet came in on. */
+   TAILQ_FOREACH(ifa, >if_addrlist, ifa_list) {
+   if (ifa->ifa_addr->sa_family == AF_INET &&
+   satosin(ifa->ifa_addr)->sin_addr.s_addr == dest)
+   dest = 0;
+   }
+
+   /* Router requirements says to only send host redirects */
+   if (dest != 0) {
+   type = ICMP_REDIRECT;
+   code = ICMP_REDIRECT_HOST;
}
}

 
-- 
http://gmerlin.de
OpenPGP: 

Re: Could not savecore after reproducible crash - solved

[Christopher Zimmermann]

This 'crash' can be prevented by turning ddb.console off. Sorry for the noise.
I'm still wondering  why I could not recover the dump.

--
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
F190 D013 8F01 AA53 E080  3F3C F17F B0A1 D44E 4FEE

Could not savecore after reproducible crash

[Christopher Zimmermann]

Hi,

I can reliably reproduce a panic in the isa serial com port driver, but
cannot recover the dumped core -> savecore: no core dump. The serial console 
log is attached
below. What else can I try?

Christopher



ddb> show registers
ds  0x10
es  0x10
fs  0x20
gs 0
edi0x3f8
esi   0xd18500acend+0xb280ac
ebp   0xf35e96fc
ebx   0xd184c0f9end+0xb240f9
edx   0xd184c000end+0xb24000
ecx   0xd0b77634i386_bus_space_io_ops
eax   0xc7ff0100 
eip   0xd060ae74db_enter+0x4
cs  0x50
eflags  0x200202start_phys+0x12e
esp   0xf35e96fc
ss  0x10
db_enter+0x4:   popl%ebp
ddb> trace  
db_enter() at db_enter+0x4
comintr(d184c000) at comintr+0x14e
intr_handler(f35e9748,d184b0c0) at intr_handler+0x13
Xintr_legacy4() at Xintr_legacy4+0x7c   
--- interrupt ---
cpu_idle_cycle(d0c92900) at cpu_idle_cycle+0xf
ddb> boot crash
   
dumping to dev 1, offset 519945
dump 255 254 253 252 251 250 249 248 247 246 245 244 243 242 241 240 239 238 
237 236 235 234 233 232 231 230 229 228 227 226 225 224 223 222 221 220 219 218 
217 216 215 214 213 212 211 210 209 208 207 206 205 204 203 202 201 200 199 198 
197 196 195 194 193 192 191 190 189 188 187 186 185 184 183 182 181 180 179 178 
177 176 175 174 173 172 171 170 169 168 167 166 165 164 163 162 161 160 159 158 
157 156 155 154 153 152 151 150 149 148 147 146 145 144 143 142 141 140 139 138 
137 136 135 134 133 132 131 130 129 128 127 126 125 124 123 122 121 120 119 118 
117 116 115 114 113 112 111 110 109 108 107 106 105 104 103 102 101 100 99 98 
97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 
71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 
45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 
19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 succeeded

   
rebooting...
PC Engines ALIX.2 v0.99h
640 KB Base Memory
261120 KB Extended Memory

01F0 Master 848A SDCFHS-016G 
Phys C/H/S 31045/16/63 Log C/H/S 1947/255/63 LBA
Using drive 0, partition 3.
Loading..
probing: pc0 com0 com1 pci mem[640K 255M a20=on] 
disk: hd0+   
>> OpenBSD/i386 BOOT 3.31
switching console to com0
>> OpenBSD/i386 BOOT 3.31
boot> booting hd0a:/bsd: 8156968+2282500+163320+0+1097728 
[674684+82+489520+501323]=0xcc0c10
entry point at 0x2000d4 

   
[ using 1666064 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993 
The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2017 OpenBSD. All rights reserved.  https://www.OpenBSD.org
  
OpenBSD 6.2-stable (GENERIC) #3: Sun Nov 26 09:30:31 CET 2017
madroach@localhost:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Geode(TM) Integrated Processor by AMD PCS ("AuthenticAMD" 586-class) 499 
MHz
cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX,MMXX,3DNOW2,3DNOW  
  
real mem  = 267931648 (255MB) 
avail mem = 248762368 (237MB)
mpath0 at root   
scsibus0 at mpath0: 256 targets
mainbus0 at root   
bios0 at mainbus0: date 11/05/08, BIOS32 rev. 0 @ 0xfd088
pcibios0 at bios0: rev 2.1 @ 0xf/0x1 
pcibios0: pcibios_get_intr_routing - function not supported
pcibios0: PCI IRQ Routing information unavailable. 
pcibios0: PCI bus #0 is the last bus  
bios0: ROM list: 0xe/0xa800 
cpu0 at mainbus0: (uniprocessor)
mtrr: K6-family MTRR support (2 registers)
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 1 function 0 "AMD Geode LX" rev 0x33
glxsb0 at pci0 dev 1 function 2 "AMD Geode LX Crypto" rev 0x00: RNG AES
vr0 at pci0 dev 9 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 10, address 
00:0d:b9:24:60:40
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 0x004063, 
model 0x0034   
vr1 at pci0 dev 10 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 11, address 
00:0d:b9:24:60:41
ukphy1 at vr1 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 0x004063, 
model 0x0034
vr2 at pci0 dev 11 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 15, address 
00:0d:b9:24:60:42
ukphy2 at vr2 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 0x004063, 
model 0x0034
athn0 at pci0 dev 12 function 0 "Atheros 

Re: Add quirks to support M-Audio FastTrack Pro (uaudio)

[Christopher Zimmermann]

Hi, 

thanks for your patience.
If some things are still unclear I'll be glad to clarify.


Christopher


On 2017-02-02 Martin Pieuchot <m...@openbsd.org> wrote:
> On 31/01/17(Tue) 14:05, Christopher Zimmermann wrote:
> > On 2017-01-29 Martin Pieuchot <m...@openbsd.org> wrote:  
> > > On 29/01/17(Sun) 19:33, Christopher Zimmermann wrote:  
> > > > [...]
> > > > @@ -444,6 +447,11 @@ uaudio_match(struct device *parent, void
> > > > if (uaa->iface == NULL || uaa->device == NULL)
> > > > return (UMATCH_NONE);
> > > >  
> > > > +   if (uaa->vendor == USB_VENDOR_MAUDIO &&
> > > > +   uaa->product == USB_PRODUCT_MAUDIO_FASTTRACKPRO &&
> > > > +   uaa->configno != 2)
> > > > +   return UMATCH_NONE;
> > > 
> > > Why to you need this?  
> > 
> > This device exposes only a very limited set of features in
> > configuration 1.  
> 
> But configuration 1 is midi not audio right?  So it won't match, so
> why do you need that?

In config 1 the device exposes these ifaces:
0 audio control
1 midi
2 audio play
3 audio rec

in config 2 the device exposes these ifaces:
0 audio control
1 midi
2 audio analog out
3 audio analog / digital out
4 audio analog in (with mic preamps)
5 audio digital in

if any iface in config 1 is claimed the device cannot switch to config 2.

> > > > @@ -3312,6 +3340,9 @@ uaudio_set_params(void *addr, int setmod
> > > > }
> > > > break;
> > > > }
> > > > +
> > > > +   if (sc->sc_quirks & UAUDIO_FLAG_BE)
> > > > +   p->encoding =
> > > > AUDIO_ENCODING_SLINEAR_BE;
> > > 
> > > Why do you need this chunk and we don't need to set
> > > AUDIO_ENCODING_SLINEAR_LE in the other case?  
> > 
> > We probably should set it to _LE in the other case. I moved this
> > piece of code into uaudio_match_alt() and made it agnostic about
> > the specific endianness. 
> > > > @@ -152,6 +153,11 @@ umidi_match(struct device *parent, void 
> > > > DPRINTFN(1,("umidi_match\n"));
> > > >  
> > > > if (uaa->iface == NULL)
> > > > +   return UMATCH_NONE;
> > > > +
> > > > +   if (uaa->vendor == USB_VENDOR_MAUDIO &&
> > > > +   uaa->product == USB_PRODUCT_MAUDIO_FASTTRACKPRO &&
> > > > +   uaa->configno != 2)
> > > > return UMATCH_NONE;
> > > 
> > > I'd leave the configno check out and add a comment explaining why
> > > we want to force this driver to attach to uaudio(4).  
> > 
> > See my comment above about the same piece of code in uaudio.c.
> > If I don't add this condition umidi will attach to configuration 1
> > and configuration 2 wouldn't be tried, uaudio wouldn't attach.  
> 
> Please re-read my comment.  I'm talking about the "configno" check.
> Think about somebody reading your code in 5 years, it will just looks
> like black magic.

What about adding this comment:

/*
 * This combined audio / midi device exposes its
 * full set of features only in configuration 2.
 */

> What you're doing is working around our stupid USB detection mechanism
> to use a specific configuration.  So you don't want your device to
> attach to umidi(4) no matter with which configuration.
> 
> > +   if (p->precision > 8)
> > +   /*
> > +* Don't search for matching encoding.
> > +* Just tell the upper layers which one we support.
> > +*/
> > +   p->encoding = sc->sc_alts[i].encoding;  
> 
> Why do you need that?

- The audio device is big endian.
- The uaudio driver was written for little endian devices.
- The audio driver assumes _host_ endianness.
- The uaudio driver will reject AUDIO_SETPAR (audio(4)) requests which don't
  exactly match the device encoding. 
- I change uaudio to communicate the supported endianness back to audio(4), so
  that userspace will receive the supported parameters via AUDIO_GETPAR and
  will use a supported encoding.

> > +   else if (p->encoding != sc->sc_alts[i].encoding)
> > +   continue;
> > +
> > alts_eh |= 1 << i;
> > DPRINTFN(6,("%s: matched %s alt %d for enc/pre\n",
> > __func__, mode == AUMODE_RECORD ? "rec" : "play", i));  



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
2779 7F73 44FD 0736 B67A  C410 69EC 7922 34B4 2566


pgpwepgVnSMc9.pgp
Description: OpenPGP digital signature

Re: Add quirks to support M-Audio FastTrack Pro (uaudio)

[Christopher Zimmermann]

Hi,

below you find an updated diff. OK?


On 2017-01-29 Martin Pieuchot <m...@openbsd.org> wrote:
> On 29/01/17(Sun) 19:33, Christopher Zimmermann wrote:
> > [...]
> > @@ -444,6 +447,11 @@ uaudio_match(struct device *parent, void
> > if (uaa->iface == NULL || uaa->device == NULL)
> > return (UMATCH_NONE);
> >  
> > +   if (uaa->vendor == USB_VENDOR_MAUDIO &&
> > +   uaa->product == USB_PRODUCT_MAUDIO_FASTTRACKPRO &&
> > +   uaa->configno != 2)
> > +   return UMATCH_NONE;  
> 
> Why to you need this?

This device exposes only a very limited set of features in
configuration 1.
To force usb_subr.c:usbd_probe_and_attach() to try configuration 2
both uaudio and umidi must attach to the device only configuration 2.
Otherwise one will claim the device and other configurations won't be
tried.

> > @@ -531,8 +539,21 @@ uaudio_attach(struct device *parent, str
> > found = 1;
> > }
> > }
> > -   if (found)
> > +   if (found) {
> > usbd_claim_iface(sc->sc_udev, i);
> > +   if (uaa->vendor == USB_VENDOR_MAUDIO &&
> > +   uaa->product == USB_PRODUCT_MAUDIO_FASTTRACKPRO) {
> > +   /*
> > +* temporarily switch every iface to 24bit.
> > +* Causes the device to be big endian even
> > +* for 16bit samples.
> > +* using 16bits first will cause the device
> > +* to break when we later switch to 24bit.
> > +*/
> > +   usbd_set_interface(sc->sc_alts[i].ifaceh, 2);
> > +   usbd_set_interface(sc->sc_alts[i].ifaceh, 0);  
> 
> How did you figure that out?  Is this behavior documented somehwere?

It's not documented. I did not find a similar hack in the linux driver.
Still its a reproducable behaviour.

> > @@ -3312,6 +3340,9 @@ uaudio_set_params(void *addr, int setmod
> > }
> > break;
> > }
> > +
> > +   if (sc->sc_quirks & UAUDIO_FLAG_BE)
> > +   p->encoding = AUDIO_ENCODING_SLINEAR_BE;  
> 
> Why do you need this chunk and we don't need to set
> AUDIO_ENCODING_SLINEAR_LE in the other case?

We probably should set it to _LE in the other case. I moved this piece
of code into uaudio_match_alt() and made it agnostic about the specific 
endianness.

> > @@ -152,6 +153,11 @@ umidi_match(struct device *parent, void 
> > DPRINTFN(1,("umidi_match\n"));
> >  
> > if (uaa->iface == NULL)
> > +   return UMATCH_NONE;
> > +
> > +   if (uaa->vendor == USB_VENDOR_MAUDIO &&
> > +   uaa->product == USB_PRODUCT_MAUDIO_FASTTRACKPRO &&
> > +   uaa->configno != 2)
> > return UMATCH_NONE;  
> 
> I'd leave the configno check out and add a comment explaining why we
> want to force this driver to attach to uaudio(4).

See my comment above about the same piece of code in uaudio.c.
If I don't add this condition umidi will attach to configuration 1 and
configuration 2 wouldn't be tried, uaudio wouldn't attach.


Here's the updated diff:



Index: uaudio.c
===
RCS file: /cvs/src/sys/dev/usb/uaudio.c,v
retrieving revision 1.122
diff -u -p -r1.122 uaudio.c
--- uaudio.c3 Jan 2017 06:45:58 -   1.122
+++ uaudio.c31 Jan 2017 12:37:46 -
@@ -172,6 +172,7 @@ struct chan {
 #define UAUDIO_FLAG_VENDOR_CLASS 0x0010/* claims vendor class but 
works */
 #define UAUDIO_FLAG_DEPENDENT   0x0020 /* play and record params must equal */
 #define UAUDIO_FLAG_EMU0202 0x0040
+#define UAUDIO_FLAG_BE  0x0080
 
 struct uaudio_devs {
struct usb_devno uv_dev;
@@ -223,7 +224,9 @@ struct uaudio_devs {
{ { USB_VENDOR_LOGITECH, USB_PRODUCT_LOGITECH_QUICKCAMZOOM },
UAUDIO_FLAG_BAD_AUDIO },
{ { USB_VENDOR_TELEX, USB_PRODUCT_TELEX_MIC1 },
-   UAUDIO_FLAG_NO_FRAC }
+   UAUDIO_FLAG_NO_FRAC },
+   { { USB_VENDOR_MIDIMAN, USB_PRODUCT_MIDIMAN_FASTTRACKPRO },
+   UAUDIO_FLAG_BE | UAUDIO_FLAG_DEPENDENT }
 };
 #define uaudio_lookup(v, p) \
((struct uaudio_devs *)usb_lookup(uaudio_devs, v, p))
@@ -444,6 +447,11 @@ uaudio_match(struct device *parent, void
if (uaa->iface == NULL || uaa->device == NULL)
return (UMATCH

Re: Add quirks to support M-Audio FastTrack Pro (uaudio)

[Christopher Zimmermann]

On 2017-01-30 Martin Pieuchot <m...@openbsd.org> wrote:
> On 29/01/17(Sun) 14:45, Christopher Zimmermann wrote:
> 
> > You are talking about usb_subr.c? Done.  
> 
> No I'm not :)

> Yes please, do not touch usb_subr.c.  

> Also make sure your comment below respect style(9).

ok. I got it. I added conditions to uaudio.c and umidi.c to accept the
device only in configuration 2.
The quirks in uaudio.c stay the same.

Christopher


Index: uaudio.c
===
RCS file: /cvs/src/sys/dev/usb/uaudio.c,v
retrieving revision 1.122
diff -u -p -r1.122 uaudio.c
--- uaudio.c3 Jan 2017 06:45:58 -   1.122
+++ uaudio.c29 Jan 2017 17:13:28 -
@@ -172,6 +172,7 @@ struct chan {
 #define UAUDIO_FLAG_VENDOR_CLASS 0x0010/* claims vendor class but 
works */
 #define UAUDIO_FLAG_DEPENDENT   0x0020 /* play and record params must equal */
 #define UAUDIO_FLAG_EMU0202 0x0040
+#define UAUDIO_FLAG_BE  0x0080
 
 struct uaudio_devs {
struct usb_devno uv_dev;
@@ -223,7 +224,9 @@ struct uaudio_devs {
{ { USB_VENDOR_LOGITECH, USB_PRODUCT_LOGITECH_QUICKCAMZOOM },
UAUDIO_FLAG_BAD_AUDIO },
{ { USB_VENDOR_TELEX, USB_PRODUCT_TELEX_MIC1 },
-   UAUDIO_FLAG_NO_FRAC }
+   UAUDIO_FLAG_NO_FRAC },
+   { { USB_VENDOR_MAUDIO, USB_PRODUCT_MAUDIO_FASTTRACKPRO },
+   UAUDIO_FLAG_BE | UAUDIO_FLAG_DEPENDENT }
 };
 #define uaudio_lookup(v, p) \
((struct uaudio_devs *)usb_lookup(uaudio_devs, v, p))
@@ -444,6 +447,11 @@ uaudio_match(struct device *parent, void
if (uaa->iface == NULL || uaa->device == NULL)
return (UMATCH_NONE);
 
+   if (uaa->vendor == USB_VENDOR_MAUDIO &&
+   uaa->product == USB_PRODUCT_MAUDIO_FASTTRACKPRO &&
+   uaa->configno != 2)
+   return UMATCH_NONE;
+
quirk = uaudio_lookup(uaa->vendor, uaa->product);
if (quirk)
flags = quirk->flags;
@@ -531,8 +539,21 @@ uaudio_attach(struct device *parent, str
found = 1;
}
}
-   if (found)
+   if (found) {
usbd_claim_iface(sc->sc_udev, i);
+   if (uaa->vendor == USB_VENDOR_MAUDIO &&
+   uaa->product == USB_PRODUCT_MAUDIO_FASTTRACKPRO) {
+   /*
+* temporarily switch every iface to 24bit.
+* Causes the device to be big endian even
+* for 16bit samples.
+* using 16bits first will cause the device
+* to break when we later switch to 24bit.
+*/
+   usbd_set_interface(sc->sc_alts[i].ifaceh, 2);
+   usbd_set_interface(sc->sc_alts[i].ifaceh, 0);
+   }
+   }
}
 
for (j = 0; j < sc->sc_nalts; j++) {
@@ -1662,7 +1683,10 @@ uaudio_process_as(struct uaudio_softc *s
} else if (prec == 24) {
sc->sc_altflags |= HAS_24;
}
-   enc = AUDIO_ENCODING_SLINEAR_LE;
+   if (sc->sc_quirks & UAUDIO_FLAG_BE)
+   enc = AUDIO_ENCODING_SLINEAR_BE;
+   else
+   enc = AUDIO_ENCODING_SLINEAR_LE;
format_str = "pcm";
break;
case UA_FMT_PCM8:
@@ -1687,9 +1711,13 @@ uaudio_process_as(struct uaudio_softc *s
return (USBD_NORMAL_COMPLETION);
}
 #ifdef UAUDIO_DEBUG
-   printf("%s: %s: %d-ch %d-bit %d-byte %s,", sc->sc_dev.dv_xname,
+   printf("%s: %s: alt %d(%d) for interface %d %d-ch %d-bit %d-byte %s enc 
%d,",
+  sc->sc_dev.dv_xname,
   dir == UE_DIR_IN ? "recording" : "playback",
-  chan, prec, bps, format_str);
+  id->bAlternateSetting,
+  sc->sc_nalts,
+  id->bInterfaceNumber,
+  chan, prec, bps, format_str, enc);
if (asf1d->bSamFreqType == UA_SAMP_CONTNUOUS) {
printf(" %d-%dHz\n", UA_SAMP_LO(asf1d), UA_SAMP_HI(asf1d));
} else {
@@ -3312,6 +3340,9 @@ uaudio_set_params(void *addr, int setmod
}
break;
}
+
+   if (sc->sc_quirks & UAUDIO_FLAG_BE)
+   p->encoding = AUDIO_ENCODING_SLINEAR_BE;
 
i = uaudio_match_alt(sc, p, mode);
if (i < 0) {
Index: umidi.c
===
RCS file: /cvs/src/sys/dev/u

Add quirks to support M-Audio FastTrack Pro (uaudio)

[Christopher Zimmermann]

Hi,

I needed to add some quirks to support the M-Audio FastTrack Pro USB
audio interface.

* The device needs to be switched to the configuration 2 to show all
  its capabilities. This is done by a new quirk in usb_subr.c
* It claims to consume little-endian samples, but actually expects
  big-endian samples.


OK?

Christopher


Index: uaudio.c
===
RCS file: /cvs/src/sys/dev/usb/uaudio.c,v
retrieving revision 1.122
diff -u -p -r1.122 uaudio.c
--- uaudio.c3 Jan 2017 06:45:58 -   1.122
+++ uaudio.c28 Jan 2017 11:53:17 -
@@ -172,6 +172,7 @@ struct chan {
 #define UAUDIO_FLAG_VENDOR_CLASS 0x0010/* claims vendor class but 
works */
 #define UAUDIO_FLAG_DEPENDENT   0x0020 /* play and record params must equal */
 #define UAUDIO_FLAG_EMU0202 0x0040
+#define UAUDIO_FLAG_FASTTRACKPRO 0x0080
 
 struct uaudio_devs {
struct usb_devno uv_dev;
@@ -223,7 +224,9 @@ struct uaudio_devs {
{ { USB_VENDOR_LOGITECH, USB_PRODUCT_LOGITECH_QUICKCAMZOOM },
UAUDIO_FLAG_BAD_AUDIO },
{ { USB_VENDOR_TELEX, USB_PRODUCT_TELEX_MIC1 },
-   UAUDIO_FLAG_NO_FRAC }
+   UAUDIO_FLAG_NO_FRAC },
+   { { USB_VENDOR_MIDIMAN, USB_PRODUCT_MIDIMAN_FASTTRACKPRO },
+   UAUDIO_FLAG_FASTTRACKPRO | UAUDIO_FLAG_DEPENDENT }
 };
 #define uaudio_lookup(v, p) \
((struct uaudio_devs *)usb_lookup(uaudio_devs, v, p))
@@ -531,8 +534,18 @@ uaudio_attach(struct device *parent, str
found = 1;
}
}
-   if (found)
+   if (found) {
usbd_claim_iface(sc->sc_udev, i);
+   if (sc->sc_quirks & UAUDIO_FLAG_FASTTRACKPRO) {
+   /* temporarily switch every iface to 24bit.
+* Causes the device to be big endian even
+* for 16bit samples.
+* using 16bits first will cause the device
+* to break when we later switch to 24bit. */
+   usbd_set_interface(sc->sc_alts[i].ifaceh, 2);
+   usbd_set_interface(sc->sc_alts[i].ifaceh, 0);
+   }
+   }
}
 
for (j = 0; j < sc->sc_nalts; j++) {
@@ -1662,7 +1675,10 @@ uaudio_process_as(struct uaudio_softc *s
} else if (prec == 24) {
sc->sc_altflags |= HAS_24;
}
-   enc = AUDIO_ENCODING_SLINEAR_LE;
+   if (sc->sc_quirks & UAUDIO_FLAG_FASTTRACKPRO)
+   enc = AUDIO_ENCODING_SLINEAR_BE;
+   else
+   enc = AUDIO_ENCODING_SLINEAR_LE;
format_str = "pcm";
break;
case UA_FMT_PCM8:
@@ -1687,9 +1703,13 @@ uaudio_process_as(struct uaudio_softc *s
return (USBD_NORMAL_COMPLETION);
}
 #ifdef UAUDIO_DEBUG
-   printf("%s: %s: %d-ch %d-bit %d-byte %s,", sc->sc_dev.dv_xname,
+   printf("%s: %s: alt %d(%d) for interface %d %d-ch %d-bit %d-byte %s enc 
%d,",
+  sc->sc_dev.dv_xname,
   dir == UE_DIR_IN ? "recording" : "playback",
-  chan, prec, bps, format_str);
+  id->bAlternateSetting,
+  sc->sc_nalts,
+  id->bInterfaceNumber,
+  chan, prec, bps, format_str, enc);
if (asf1d->bSamFreqType == UA_SAMP_CONTNUOUS) {
printf(" %d-%dHz\n", UA_SAMP_LO(asf1d), UA_SAMP_HI(asf1d));
} else {
@@ -3312,6 +3332,9 @@ uaudio_set_params(void *addr, int setmod
}
break;
}
+
+   if (sc->sc_quirks & UAUDIO_FLAG_FASTTRACKPRO)
+   p->encoding = AUDIO_ENCODING_SLINEAR_BE;
 
i = uaudio_match_alt(sc, p, mode);
if (i < 0) {
Index: usb_quirks.c
===
RCS file: /cvs/src/sys/dev/usb/usb_quirks.c,v
retrieving revision 1.74
diff -u -p -r1.74 usb_quirks.c
--- usb_quirks.c27 Nov 2015 10:59:32 -  1.74
+++ usb_quirks.c28 Jan 2017 11:53:17 -
@@ -67,6 +67,8 @@ const struct usbd_quirk_entry {
0x001, { UQ_ASSUME_CM_OVER_DATA }},
  { USB_VENDOR_EICON, USB_PRODUCT_EICON_DIVA852,
0x100, { UQ_ASSUME_CM_OVER_DATA }},
+ { USB_VENDOR_MIDIMAN, USB_PRODUCT_MIDIMAN_FASTTRACKPRO,
+   0x100, { 1 << UQ_CONFIG_SHIFT }},
  /* YAMAHA router's ucdDevice is the version of firmware and often changes. */
  { USB_VENDOR_YAMAHA, USB_PRODUCT_YAMAHA_RTA54I,
ANY, { UQ_ASSUME_CM_OVER_DATA }},
Index: usb_quirks.h
===
RCS file: /cvs/src/sys/dev/usb/usb_quirks.h,v
retrieving revision 1.16
diff -u -p -r1.16 usb_quirks.h
--- usb_quirks.h

Re: httpd: $DOCUMENT_URI macro fix for FastCGI

[Christopher Zimmermann]

On 2016-04-17 Tim Baumgard  wrote:
> 
> Below is an updated diff for the man page to address your feedback. I
> also included another small diff that fixes the QUERY_STRING variable
> when no query string is given. The RFC states:
> 
> The server MUST set this variable; if the Script-URI does not include
> a query component, the QUERY_STRING MUST be defined as an empty string
> ("").
> 
> (https://tools.ietf.org/html/rfc3875#section-4.1.7)
> 
> nginx follows the RFC, so I'm assuming that httpd's behavior is a
> bug. The man page diff takes this change into account.


OK chrisz@, please wait for florian@ or reyk@, too.


> Index: httpd.conf.5
> ===
> RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
> retrieving revision 1.68
> diff -u -p -r1.68 httpd.conf.5
> --- httpd.conf.5  19 Jul 2015 05:17:27 -  1.68
> +++ httpd.conf.5  18 Apr 2016 01:26:18 -
> @@ -274,6 +274,62 @@ root directory of
>  .Xr httpd 8
>  and defaults to
>  .Pa /run/slowcgi.sock .
> +.Pp
> +The FastCGI handler will be given the following variables:
> +.Pp
> +.Bl -tag -width GATEWAY_INTERFACE -offset indent -compact
> +.It Ic DOCUMENT_ROOT
> +The document root in which the script is located as configured by the
> +.Ic root
> +option for the server or location that matches the request.
> +.It Ic GATEWAY_INTERFACE
> +The revision of the CGI specification used.
> +.It Ic HTTP_*
> +Additional HTTP headers the connected client sent in the request, if
> +any.
> +.It Ic HTTPS
> +A variable that is set to
> +.Qq on
> +when the server has been configured to use TLS. This variable is
> +omitted otherwise.
> +.It Ic REQUEST_URI
> +The path and optional query string as requested by the connected
> client. +.It Ic DOCUMENT_URI
> +The canonicalized URI for the script, possibly with a slash or
> +directory index file name appended.
> +.It Ic SCRIPT_NAME
> +The virtual URI path to the script.
> +.It Ic PATH_INFO
> +The optional path appended after the script name in the request path.
> +This variable is an empty string if no path is appended after the
> +script name.
> +.It Ic SCRIPT_FILENAME
> +The absolute, physical path to the script within the
> +.Xr chroot 2
> +directory.
> +.It Ic QUERY_STRING
> +The optional query string of the request. This variable is an empty
> +string if there is no query string in the request.
> +.It Ic REMOTE_ADDR
> +The IP address of the connected client.
> +.It Ic REMOTE_PORT
> +The TCP source port of the connected client.
> +.It Ic REMOTE_USER
> +The remote user when using HTTP authentication.
> +.It Ic REQUEST_METHOD
> +The HTTP method the connected client used when making the request.
> +.It Ic SERVER_ADDR
> +The configured IP address of the server.
> +.It Ic SERVER_NAME
> +The name of the server.
> +.It Ic SERVER_PORT
> +The configured TCP server port of the server.
> +.It Ic SERVER_PROTOCOL
> +The revision of the HTTP specification used.
> +.It Ic SERVER_SOFTWARE
> +The server software name of
> +.Xr httpd 8 .
> +.El
>  .It Ic hsts Oo Ar option Oc
>  Enable HTTP Strict Transport Security.
>  Valid options are:
> 
> 
> Index: server_fcgi.c
> ===
> RCS file: /cvs/src/usr.sbin/httpd/server_fcgi.c,v
> retrieving revision 1.67
> diff -u -p -r1.67 server_fcgi.c
> --- server_fcgi.c 23 Nov 2015 20:56:15 -  1.67
> +++ server_fcgi.c 18 Apr 2016 01:26:49 -
> @@ -242,12 +242,16 @@ server_fcgi(struct httpd *env, struct cl
>   goto fail;
>   }
>  
> - if (desc->http_query)
> + if (desc->http_query) {
>   if (fcgi_add_param(, "QUERY_STRING",
> desc->http_query, clt) == -1) {
>   errstr = "failed to encode param";
>   goto fail;
>   }
> + } else if (fcgi_add_param(, "QUERY_STRING", "", clt)
> == -1) {
> + errstr = "failed to encode param";
> + goto fail;
> + }
>  
>   if (fcgi_add_param(, "DOCUMENT_ROOT", srv_conf->root,
>   clt) == -1) {
> 



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
2779 7F73 44FD 0736 B67A  C410 69EC 7922 34B4 2566


pgpggwfN2z0vD.pgp
Description: OpenPGP digital signature

Re: httpd: $DOCUMENT_URI macro fix for FastCGI

[Christopher Zimmermann]

On 2016-04-14 Tim Baumgard <open...@bmgrd.com> wrote:
> > Christopher Zimmermann wrote:
> > Tim Baumgard wrote:  
> >> According to convention and httpd.conf(5), the $DOCUMENT_URI macro
> >> for FastCGI calls should expand to the request path instead of the
> >> path alias.  
> > 
> > FastCGI / CGI is poorly documented. As far as convention goes, at
> > least nginx uses the rewritten url, too:
> > 
> > $document_uri
> >same as $uri 
> > 
> > $uri
> >current URI in request, normalized
> > 
> >The value of $uri may change during request processing, e.g. when
> >doing internal redirects, or when using index files.
> > 
> > This behavior has the effect that the CGI script will know the
> > "canonical" url of the requested ressource, not an alias that the
> > client happened to request. We call the rewritten, canonical url
> > "alias" in our code, so that might be confusing.  
> 
> Indeed, nginx works as you described. Also, it appears lighttpd works
> in a similar way.
> 
> "Convention" was a bad word choice. I meant that it's what Apache and
> many implementations of SSI do. It's also what I was finding when
> searching using queries along the lines of "CGI DOCUMENT_URI", which I
> was doing because the CGI RFC doesn't include DOCUMENT_URI.
> 
> The current behavior is redundant since SCRIPT_NAME, which is in the
> RFC, and DOCUMENT_URI are always set to the same thing.

That's not always the case. DOCUMENT_URI is SCRIPT_NAME ^ PATH_INFO.
If you want the original URI, you can always use REQUEST_URI and strip
the query string, which is easy to do.

> Also, the man
> page says that the $DOCUMENT_URI macro is "the request path" and the
> $REQUEST_URI macro is "the request path and optional query string." I
> suppose it's OK to use different meanings for the same term in these
> two different places since they're serving different purposes, but
> it's a little misleading.
> 
> Anyway, while I now think the current behavior is probably best in
> terms of the semantics of "DOCUMENT_URI," the point I just tried to
> make with my drawn-out explanation is that it's pretty easy to come
> to the same incorrect conclusion that I did.

CGI variables are poorly documented, not really standardised and
confusing.

> So, instead of
> considering what I sent previously, here's a diff for the
> httpd.conf(5) man page that adds a list of the variables (and their
> descriptions) that are given to the FastCGI handler. Making the
> behavior for these values explicit should avoid any similar
> confusion, and I'm sure it would be helpful in other ways as well.

Thanks for the effort. This will indeed be helpfull for many users.

> 
> Index: httpd.conf.5
> ===
> RCS file: /cvs/src/usr.sbin/httpd/httpd.conf.5,v
> retrieving revision 1.68
> diff -u -p -r1.68 httpd.conf.5
> --- httpd.conf.5  19 Jul 2015 05:17:27 -  1.68
> +++ httpd.conf.5  15 Apr 2016 00:55:25 -
> @@ -274,6 +274,59 @@ root directory of
>  .Xr httpd 8
>  and defaults to
>  .Pa /run/slowcgi.sock .
> +.Pp
> +The FastCGI handler will be given the following variables:
> +.Pp
> +.Bl -tag -width GATEWAY_INTERFACE -offset indent -compact
> +.It Ic DOCUMENT_ROOT
> +The document root in which the script is located as configured by the
> +.Ic root
> +option for the server or location that matches the request.
> +.It Ic DOCUMENT_URI
> +The URI path to the script.

Rather the canonicalised URI, possibly with '/' and/or index appended.
NOT necessarily the path to the script; neither virtual nor physical (see 
above).

> +.It Ic GATEWAY_INTERFACE
> +The revision of the CGI specification used.
> +.It Ic HTTP_*
> +Additional HTTP headers the connected client sent in the request, if
> +any.
> +.It Ic HTTPS
> +A variable that is set to
> +.Qq on
> +when the server has been configured to use TLS. This variable is not
> +given otherwise.

Put REQUEST_PATH, DOCUMENT_URI, SCRIPT_NAME, PATH_INFO and
SCRIPT_FILENAME here, they are best understood in context of each other.

> +.It Ic PATH_INFO
> +The optional path appended after the script name in the request path.
> +This variable is empty if no path is appended after the script name.
> +.It Ic QUERY_STRING
> +The optional query string of the request.
> +.It Ic REMOTE_ADDR
> +The IP address of the connected client.
> +.It Ic REMOTE_PORT
> +The TCP source port of the connected client
> +.It Ic REMOTE_USER
> +The remote user when using HTTP authentication.
> +.It Ic REQUEST_METHOD
> +The HTTP method the connected clien

Re: httpd: $DOCUMENT_URI macro fix for FastCGI

[Christopher Zimmermann]

On 2016-04-14 Tim Baumgard  wrote:
> According to convention and httpd.conf(5), the $DOCUMENT_URI macro for
> FastCGI calls should expand to the request path instead of the path
> alias.

FastCGI / CGI is poorly documented. As far as convention goes, at least
nginx uses the rewritten url, too:

$document_uri
same as $uri 

$uri
current URI in request, normalized

The value of $uri may change during request processing, e.g. when
doing internal redirects, or when using index files.

This behavior has the effect that the CGI script will know the
"canonical" url of the requested ressource, not an alias that the
client happened to request. We call the rewritten, canonical url "alias"
in our code, so that might be confusing.

If you still think httpd is showing unwanted behaviour, please provide
an example.

> This isn't a major issue because only "block return code uri"
> options make use of path aliases at the moment, and those use a
> separate macro-expansion function, server_expand_http(), which
> correctly expands $DOCUMENT_URI.
> 
> Index: server_fcgi.c
> ===
> RCS file: /cvs/src/usr.sbin/httpd/server_fcgi.c,v
> retrieving revision 1.67
> diff -u -p -r1.67 server_fcgi.c
> --- server_fcgi.c 23 Nov 2015 20:56:15 -  1.67
> +++ server_fcgi.c 14 Apr 2016 03:41:30 -
> @@ -254,7 +254,7 @@ server_fcgi(struct httpd *env, struct cl
>   errstr = "failed to encode param";
>   goto fail;
>   }
> - if (fcgi_add_param(, "DOCUMENT_URI", alias,
> + if (fcgi_add_param(, "DOCUMENT_URI", desc->http_path,
>   clt) == -1) {
>   errstr = "failed to encode param";
>   goto fail;
> 



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
2779 7F73 44FD 0736 B67A  C410 69EC 7922 34B4 2566


pgphn3GpSMWJ_.pgp
Description: OpenPGP digital signature

Re: spamd - DNS whitelist - with prototype

[Christopher Zimmermann]

I forgot to attach my prototype. Here it is.

On 2016-03-29 Bob Beck <b...@openbsd.org> wrote:
> No.  DNS based whitelisting does not belong in there. because it is
> slow and DOS'able
> 
> spamd is designed to be high speed low drag. If you want to do a DNS
> based whitelist, write a little co-thing that spits one
> into a file or into your nospamd table that then spamd *does not even
> see*.

That's kind of what my prototype implementation does.

> In short *spamd* is the wrong place to do this.  put your dns based
> whitelist in a table periodically

I put the ips into the table on demand since I cannot simply download a
dnswl. I already suspected relayd might be a better place to do this.
Or keep it in a separate program. Sadly I cannot reinject the diverted
SYN into pf.

> On Tue, Mar 29, 2016 at 1:11 PM, Christopher Zimmermann
> <chr...@openbsd.org> wrote:
> > Hi,
> >
> > I want to use a DNS white list to skip greylisting delays for known
> > good addresses, which would pass the greylist anyway.
> > To do this with spamd and OpenSMTPd I wrote a prototype which
> > intercepts the initial SYN packet from any non-whitelisted ip. It
> > then queries DNS whitelists and on any positive reply it whitelists
> > the ip. The SYN packet is dropped. Any sane smtp server will very
> > shortly resend the SYN and get through to OpenSMTPd.
> > This program is only a proof-of-concept. I think the same
> > functionality could be integrated into spamd or as transparent
> > relay into relayd. Is this a sensible approach?
> >
> > Christopher
> >
> >
> > On 2016-03-15 Stuart Henderson <s...@spacehopper.org> wrote:  
> >> On 2016/03/15 12:55, Craig Skinner wrote:  
> >> > Generally, everything has changed from file feeds to DNS.  
> >>
> >> Yep, because for the more actively maintained ones 1) new entries
> >> show up more quickly than any sane rsync interval, this is quite
> >> important for good blocking these days 2) DNS is less resource
> >> intensive and more easily distributed than rsync, and 3)
> >> importantly for the rbl providers, it gives additional input to
> >> them about new mail sources (if an rbl suddenly starts seeing
> >> queries from all over the world for a previously unseen address,
> >> it's probably worth investigation - I am sure this is why some of
> >> the commercial antispam operators provide free DNS-based lookups
> >> for smaller orgs).
> >>
> >> A more flexible approach would be to skip the PF table integration
> >> completely and do DNS lookups in spamd (or, uh, relayd, or
> >> something new) and based on that it could choose whether to
> >> tarpit, greylist or transparent-forward the connection to the real
> >> mail server. This would also give a way to use dnswl.org's
> >> whitelist to avoid greylisting for those hosts where it just
> >> doesn't work well (gmail, office365 etc).


#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 


#define DEBUG 0

#define DIVERT_PORT 25

#define NSTATES 10

struct dns_header {
uint16_tid;
uint16_tflags;
#define QR 0x8000
#define OPCODE_MASK 0x7800
#define OPCODE_SHIFT 11
#define AA 0x0400
#define TC 0x0200
#define RD 0x0100
#define RA 0x0080
#define AD 0x0020
#define CD 0x0010
#define RCODE_MASK 0x000f
#define RCODE_SHIFT 0
uint16_tqdcount;
uint16_tancount;
uint16_tnscount;
uint16_tarcount;
};

struct dns_record {
uint16_ttype;
uint16_tclass;
uint32_tttl;
uint16_tlength;
};

struct state {
union {
struct in_addr in4;
struct in6_addr in6;
uint8_t octets[sizeof(struct in6_addr)];
} addr;
struct timespec timeout;
int af;
uint16_t dnskey;
} states[NSTATES];

void send_query(struct state *state, const char *question);
void process_response();

enum color { white, grey };
void enlist(struct state *state, enum color color);

int dnssock, pfdev;

const char *const whitelists[] = {
"list.dnswl.org",
"swl.spamhaus.org"
};

int main(int argc, char *argv[])
{
int i, ret, timeout;
time_t t;
struct sockaddr_in sin4;
struct sockaddr_in6 sin6;
struct group *group;
struct passwd *passwd;
struct pollfd fds[3];

pfdev = open("/dev/pf", O_RDWR);
if (pfdev == -1) err(1, "open(\"/dev/pf\") failed");

ret = IPPROTO_DIVERT_INIT;
setsockopt(fds[1].fd, IPPROTO_IP, IP_DIVERTFL, , sizeof(ret));
setsockopt(fds

Re: spamd - DNS whitelist

[Christopher Zimmermann]

Hi,

I want to use a DNS white list to skip greylisting delays for known
good addresses, which would pass the greylist anyway.
To do this with spamd and OpenSMTPd I wrote a prototype which intercepts
the initial SYN packet from any non-whitelisted ip. It then queries DNS
whitelists and on any positive reply it whitelists the ip. The SYN
packet is dropped. Any sane smtp server will very shortly resend the
SYN and get through to OpenSMTPd.
This program is only a proof-of-concept. I think the same functionality
could be integrated into spamd or as transparent relay into relayd. Is
this a sensible approach? 

Christopher


On 2016-03-15 Stuart Henderson  wrote:
> On 2016/03/15 12:55, Craig Skinner wrote:
> > Generally, everything has changed from file feeds to DNS.  
> 
> Yep, because for the more actively maintained ones 1) new entries show
> up more quickly than any sane rsync interval, this is quite important
> for good blocking these days 2) DNS is less resource intensive and
> more easily distributed than rsync, and 3) importantly for the rbl
> providers, it gives additional input to them about new mail sources
> (if an rbl suddenly starts seeing queries from all over the world for
> a previously unseen address, it's probably worth investigation - I am
> sure this is why some of the commercial antispam operators provide
> free DNS-based lookups for smaller orgs).
> 
> A more flexible approach would be to skip the PF table integration
> completely and do DNS lookups in spamd (or, uh, relayd, or something
> new) and based on that it could choose whether to tarpit, greylist or
> transparent-forward the connection to the real mail server. This
> would also give a way to use dnswl.org's whitelist to avoid
> greylisting for those hosts where it just doesn't work well (gmail,
> office365 etc).
> 



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
2779 7F73 44FD 0736 B67A  C410 69EC 7922 34B4 2566


pgp3n09YtGV91.pgp
Description: OpenPGP digital signature

Re: httpd: multiple addresses for one server

[Christopher Zimmermann]

Hi,


On Sat, 3 Jan 2015 14:42:18 +0100 Reyk Floeter r...@openbsd.org wrote:

 On Thu, Jan 01, 2015 at 11:54:46PM -0500, Geoff Steckel wrote:
  Is there any way todo the equivalent of:
  
  server an.example.com
  listen on 192.168.2.99
  listen on 2001.fefe.1.1::99
 
 I used include directives to avoid duplications (see previous reply)
 but the following diff allows to add aliases and multiple listen
 statements.
 
 Reyk

As I understand your diff you will duplicate the entire struct
server_config for each combination of hostname (alias), listen address
and location section. Isn't this overkill?
To me it seems like server_config is serving too much purposes here. A
clean design should split up the struct server_config into one struct
for connection settings, which will contain a TAILQ of hostnames, a
TAILQ of listen addresses and a TAILQ of structs with location settings:

TAILQ_HEAD(hostnames, char*);

struct server_config {
charlocation[NAME_MAX];
charroot[MAXPATHLEN];
[...]
}
TAILQ_HEAD(server_configs, server_config);

struct serverhost {
struct sockaddr_storages*srv_ss;
char*tls_cert;
int  tcpbufsize;
[... other connection settings ...]
struct hostnames*srv_names;
struct server_configs   *srv_confs;
}


For now you could simply loop over a TAILQ of hostnames in
server_privinit() and add an inner LOOP in server_response() searching
for the hostname/aliases.


Christopher



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
F190 D013 8F01 AA53 E080  3F3C F17F B0A1 D44E 4FEE

sed: allow d and y functions in { } function list

[Christopher Zimmermann]

Hi

$ sed -e { y/o/u/ }
sed: 1: { y/o/u/ }: extra text at the end of a transform command

but this is allowed according to the manual:

 Functions can be combined to form a function list, a list of sed
 functions separated by newlines, as follows:

   { function
 function
 ...
 function
   }

 The `{' can be preceded or followed by whitespace.  The function can be
 preceded by whitespace as well.  The terminating `}' must be preceded by
 a newline or optional whitespace.


I tried to fix it by adding special cases for '}' to the relevant parts
of the compile_stream() function. Is this correct? OK?


Christopher


Index: compile.c
===
RCS file: /cvs/src/usr.bin/sed/compile.c,v
retrieving revision 1.36
diff -u -p -r1.36 compile.c
--- compile.c   8 Oct 2014 04:19:08 -   1.36
+++ compile.c   22 Oct 2014 15:32:55 -
@@ -234,14 +234,19 @@ nonsel:   /* Now parse the command */
case EMPTY: /* d D g G h H l n N p P q x = \0 */
p++;
EATSPACE();
-   if (*p == ';') {
+   switch (*p) {
+   case ';':
p++;
link = cmd-next;
+   /* FALLTHROUGH */
+   case '}':
goto semicolon;
+   case '\0':
+   break;
+   default:
+   err(COMPILE, extra characters at the end of 
+   %c command, cmd-code);
}
-   if (*p)
-   err(COMPILE,
-extra characters at the end of %c command, cmd-code);
break;
case TEXT:  /* a c i */
p++;
@@ -323,14 +328,19 @@ nonsel:   /* Now parse the command */
p++;
p = compile_tr(p, (char **)cmd-u.y);
EATSPACE();
-   if (*p == ';') {
+   switch (*p) {
+   case ';':
p++;
link = cmd-next;
+   /* FALLTHROUGH */
+   case '}':
goto semicolon;
-   }
-   if (*p)
+   case '\0':
+   break;
+   default:
err(COMPILE, extra text at the end of a
 transform command);
+   }
break;
}
}


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
F190 D013 8F01 AA53 E080  3F3C F17F B0A1 D44E 4FEE

signature.asc
Description: PGP signature

Re: sed: allow d and y functions in { } function list

[Christopher Zimmermann]

On Wed, 22 Oct 2014 10:46:43 -0700 Philip Guenther guent...@gmail.com
wrote:

 On Wed, Oct 22, 2014 at 8:37 AM, Christopher Zimmermann
 chr...@openbsd.org wrote:
  $ sed -e { y/o/u/ }
  sed: 1: { y/o/u/ }: extra text at the end of a transform command
 
  but this is allowed according to the manual:
 
   Functions can be combined to form a function list, a list of
  sed functions separated by newlines, as follows:
 
 { function
   function
   ...
   function
 }
 
   The `{' can be preceded or followed by whitespace.  The
  function can be preceded by whitespace as well.  The terminating
  `}' must be preceded by a newline or optional whitespace.
 
 That looks like a documentation bug to me.  To quote the POSIX spec:
 --
 [2addr] {editing command
 editing command
 ...
 } Execute a list of sed editing commands only when the pattern space
 is selected. The
 
   list of sed editing commands shall be surrounded by
 braces and separated by
   newline characters, and conform to the following
 rules. The braces can be
   preceded or followed by blank characters. The
 editing commands can be
   preceded by blank characters, but shall not be
 followed by blank characters.
   The right-brace shall be preceded by a newline
 and can be preceded or
   followed by blank characters.
 --
 
 So the newline before the close-brace is required.  Since the code
 matches the spec, I think we should change the doc to match both of
 them.  Or is there some reason this extension is required?

I just patched sysutils/findlib:

$OpenBSD: patch-src_findlib_Makefile,v 1.5 2014/10/22 14:56:42 chrisz Exp $
--- src/findlib/Makefile.orig   Wed Oct 15 13:07:40 2014
+++ src/findlib/MakefileWed Oct 22 16:54:22 2014
@@ -74,7 +74,7 @@ topfind.ml: topfind.ml.in
if [ $(ENABLE_TOPFIND_PPXOPT) = true ]; then \
cp topfind.ml.in topfind.ml; \
else \
-   sed -e '/PPXOPT_BEGIN/,/PPXOPT_END/{d}' topfind.ml.in\
+   sed -e '/PPXOPT_BEGIN/,/PPXOPT_END/ d' topfind.ml.in\
 topfind.ml ;   \
fi
 
gnu sed is more permissive here:

{ commands }
A group of commands may be enclosed between { and } characters.
This is particularly useful when you want a group of commands
to be triggered by a single address (or address-range) match. 

But I don't expect to stumble over non portable sed code like that
often.

Christopher


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
F190 D013 8F01 AA53 E080  3F3C F17F B0A1 D44E 4FEE

signature.asc
Description: PGP signature

Re: sed: allow d and y functions in { } function list

[Christopher Zimmermann]

On Wed, 22 Oct 2014 21:57:14 +0200 Ingo Schwarze schwa...@usta.de
wrote:

  So the newline before the close-brace is required.  Since the code
  matches the spec, I think we should change the doc to match both of
  them.  Or is there some reason this extension is required?
 
 That would be the following patch:
   Ingo
 
 
 Index: sed.1
 ===
 RCS file: /cvs/src/usr.bin/sed/sed.1,v
 retrieving revision 1.43
 diff -u -p -r1.43 sed.1
 --- sed.1 27 May 2014 17:45:02 -  1.43
 +++ sed.1 22 Oct 2014 19:36:54 -
 @@ -268,7 +268,7 @@ Functions can be combined to form a
  .Em function list ,
  a list of
  .Nm
 -functions separated by newlines, as follows:
 +functions each followed by a newline, as follows:
  .Bd -literal -offset indent
  { function
function
 @@ -277,13 +277,8 @@ functions separated by newlines, as foll
  }
  .Ed
  .Pp
 -The
 -.Ql {
 -can be preceded or followed by whitespace.
 -The function can be preceded by whitespace as well.
 -The terminating
 -.Ql }
 -must be preceded by a newline or optional whitespace.
 +The braces can be preceded and followed by whitespace.
 +The functions can be preceded by whitespace as well.
  .Pp
  Functions and function lists may be preceded by an exclamation mark,
  in which case they are applied only to lines that are


OK chrisz@

-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
F190 D013 8F01 AA53 E080  3F3C F17F B0A1 D44E 4FEE

signature.asc
Description: PGP signature

half-duplex netcat with closed stdout / stdin

[Christopher Zimmermann]

Hi,

I need a shell utility for sending udp datagrams. Of course netcat is
the tool of choice for this purpose.
But our netcat won't ever terminate when used for sending an udp
datagram:

$ echo 'Hello, World!' |nc -Nu 8.8.8.8 53

also our netcat shows undefined behavior when started with closed
stdin or stdout:

$ nc -l 2000 - - 
[1] 30593
$ echo 'Hello, World!' |nc localhost 2000
Hello, World!

that's bizarre. The nc -l is dual-using its network socket as stdout or
stdin by accident.
I think the most natural behavior for netcat would be to terminate when
stdout and stdin are closed and to switch to half-duplex when only one
of them is closed.
I believe the two diffs I provide will accomplish this without changing
the current defined behavior of netcat.
I split the diff in two. The first one contains all the code changes
and will apply and compile just fine. The second one contains a
reordering of the main loop which would have made the first diff
unreadable. As a cosmetic change reordered the indices on pfd to match
the filedescriptors. Don't let this confuse you.
I'm not 100% sure whether the new close(lfd); I added is without bad
sideeffects. I added it because the shutdown(lfd, SHUT_WR) only works
for sockets.

OK? comments?

Christopher


--- netcat.c.orig   Sun Mar 30 23:05:14 2014
+++ netcat.cMon Mar 31 19:21:53 2014
@@ -66,7 +66,8 @@
 #define UNIX_DG_TMP_SOCKET_SIZE19
 
 /* Command Line Options */
-intdflag;  /* detached, no stdin */
+intnostdin;/* detached, no stdin */
+intnostdout;   /* no stdout */
 intFflag;  /* fdpass sock to stdout */
 unsigned int iflag;/* Interval Flag */
 intkflag;  /* More than one connect */
@@ -157,7 +158,7 @@
errx(1, unsupported proxy protocol);
break;
case 'd':
-   dflag = 1;
+   nostdin = 1;
break;
case 'F':
Fflag = 1;
@@ -286,6 +287,13 @@
if (!lflag  kflag)
errx(1, must use -l with -k);
 
+   if (nostdin) Nflag = 0;
+   /* check if stdin / stdout are open */
+   if (dup2(STDIN_FILENO, STDIN_FILENO) == -1  errno == EBADF)
+   nostdin = 1;
+   if (dup2(STDOUT_FILENO, STDOUT_FILENO) == -1  errno == EBADF)
+   nostdout = 1;
+
/* Get name of temporary socket for unix datagram client */
if ((family == AF_UNIX)  uflag  !lflag) {
if (sflag) {
@@ -732,27 +740,34 @@
 void
 readwrite(int nfd)
 {
-   struct pollfd pfd[2];
+   struct pollfd pfd[3];
unsigned char buf[16384];
-   int n, wfd = fileno(stdin);
+   int n, wfd = nostdin ? -1 : STDIN_FILENO;
-   int lfd = fileno(stdout);
+   int lfd = nostdout ? -1 : STDOUT_FILENO;
int plen;
 
plen = 2048;
 
/* Setup Network FD */
-   pfd[0].fd = nfd;
+   pfd[2].fd = nfd;
-   pfd[0].events = POLLIN;
+   pfd[2].events = POLLIN;
 
/* Set up STDIN FD. */
-   pfd[1].fd = wfd;
+   pfd[0].fd = wfd;
-   pfd[1].events = POLLIN;
+   pfd[0].events = POLLIN;
 
+   /* Set up STDOUT FD. */
+   pfd[1].fd = lfd;
+   pfd[1].events = POLLHUP;
+
+   if (nostdin  Nflag) shutdown(nfd, SHUT_WR);
+   if (nostdout) shutdown(nfd, SHUT_RD);
+
-   while (pfd[0].fd != -1) {
+   while (pfd[0].fd != -1 || pfd[1].fd != -1) {
if (iflag)
sleep(iflag);
 
-   if ((n = poll(pfd, 2 - dflag, timeout))  0) {
+   if ((n = poll(pfd, 3, timeout))  0) {
close(nfd);
err(1, Polling Error);
}
@@ -760,13 +775,16 @@
if (n == 0)
return;
 
-   if (pfd[0].revents  POLLIN) {
+   if (pfd[2].revents  POLLIN) {
if ((n = read(nfd, buf, plen))  0)
return;
else if (n == 0) {
shutdown(nfd, SHUT_RD);
+   shutdown(lfd, SHUT_WR);
+   close(lfd);
+   /*if (!Nflag)*/ return;
-   pfd[0].fd = -1;
-   pfd[0].events = 0;
+   pfd[1].fd = -1;
+   pfd[2].events = POLLHUP;
} else {
if (tflag)
atelnet(nfd, buf, n);
@@ -775,18 +793,30 @@
}
}
 
-   if (!dflag  pfd[1].revents  POLLIN) {
+   if (pfd[0].revents  POLLIN) {
if ((n = 

Re: half-duplex netcat with closed stdout / stdin

[Christopher Zimmermann]

On Mon, 31 Mar 2014 20:21:17 +0100 Stuart Henderson st...@openbsd.org
wrote:

 On 2014/03/31 20:33, Christopher Zimmermann wrote:
  But our netcat won't ever terminate when used for sending an udp
  datagram:
  
  $ echo 'Hello, World!' |nc -Nu 8.8.8.8 53
 
 this seems to work though ...
 
 $ echo 'Hello, World!' | nc -N -w0 -u 10.0.0.1 53


I tried that, too. It's not reliable due to a race between stdin and
0ms timeout:

dd if=/dev/zero bs=1K count=1K |md5 |./nc -u 8.8.8.8 53 -

if it still works, crank the count further up. I know I can use -w1
instead (that's what I currently do). But that's just a workaround, not
a fix. Also it is vulnerable by forged replies.


Christopher


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
F190 D013 8F01 AA53 E080  3F3C F17F B0A1 D44E 4FEE

ATM packet size quantization for newqueue

[Christopher Zimmermann]

Hello,

For several releases I used a similar patch for altq to crank up the
bandwidth of my uplink traffic shaper without compromising its
reliability with varying packet sizes over an ATM DSL link.
Now that newqueue is committed I took the time and prepared a proper
diff with pf.conf syntax and manpage. If you want to read about why
this is useful, you can read for example:
https://en.wikipedia.org/wiki/Point-to-point_protocol_over_Ethernet#Protocol_overhead

For me it is useful because I have an asymmetric bandwidth of
223kbit up / 2000kbit down. Subsequently the proportion of very
small TCP ack and VoIP packets on my uplink is large. Those packages 
are underestimated without correction for the PPPoEoA and ATM
fragmentation overhead.



Christopher


Index: sbin/pfctl/parse.y
===
RCS file: /cvs/src/sbin/pfctl/parse.y,v
retrieving revision 1.631
diff -u -p -r1.631 parse.y
--- sbin/pfctl/parse.y  22 Jan 2014 00:21:16 -  1.631
+++ sbin/pfctl/parse.y  25 Feb 2014 16:23:31 -
@@ -317,6 +317,9 @@ struct queue_opts {
struct node_sc   upperlimit;
char*parent;
int  flags;
+   int16_t  overhead;
+   u_int16_tpayload;
+   u_int16_tsize;
u_intqlimit;
 } queue_opts;
 
@@ -481,8 +484,8 @@ int parseport(char *, struct range *r, i
 %token BITMASK RANDOM SOURCEHASH ROUNDROBIN LEASTSTATES STATICPORT PROBABILITY
 %token WEIGHT
 %token ALTQ CBQ PRIQ HFSC BANDWIDTH TBRSIZE LINKSHARE REALTIME UPPERLIMIT
-%token QUEUE OLDQUEUE PRIORITY QLIMIT RTABLE RDOMAIN MINIMUM BURST PARENT
-%token LOAD RULESET_OPTIMIZATION RTABLE RDOMAIN PRIO ONCE DEFAULT
+%token QUANTIZE QUEUE OLDQUEUE PRIORITY QLIMIT RTABLE RDOMAIN MINIMUM BURST
+%token PARENT LOAD RULESET_OPTIMIZATION RTABLE RDOMAIN PRIO ONCE DEFAULT
 %token STICKYADDRESS MAXSRCSTATES MAXSRCNODES SOURCETRACK GLOBAL RULE
 %token MAXSRCCONN MAXSRCCONNRATE OVERLOAD FLUSH SLOPPY PFLOW
 %token TAGGED TAG IFBOUND FLOATING STATEPOLICY STATEDEFAULTS ROUTE
@@ -1366,6 +1369,16 @@ queue_opt: BANDWIDTH scspec optscs   
{
queue_opts.marker |= QOM_QLIMIT;
queue_opts.qlimit = $2;
}
+   | QUANTIZE '(' NUMBER comma NUMBER comma NUMBER ')' {
+   if (queue_opts.flags  HFSC_QUANTIZE) {
+   yyerror(quantize cannot be respecified);
+   YYERROR;
+   }
+   queue_opts.flags |= HFSC_QUANTIZE;
+   queue_opts.overhead = $3;
+   queue_opts.payload = $5;
+   queue_opts.size = $7;
+   }
;
 
 optscs : /* nada */{
@@ -4745,6 +4758,12 @@ expand_queue(char *qname, struct node_if
qspec.flags = opts-flags;
qspec.qlimit = opts-qlimit;
 
+   if (qspec.flags  HFSC_QUANTIZE) {
+   qspec.overhead = opts-overhead;
+   qspec.payload = opts-payload;
+   qspec.size = opts-size;
+   }
+
if (pfctl_add_queue(pf, qspec)) {
yyerror(cannot add queue);
return (1);
@@ -5484,6 +5503,7 @@ lookup(char *s)
{ probability,PROBABILITY},
{ proto,  PROTO},
{ qlimit, QLIMIT},
+   { quantize,   QUANTIZE},
{ queue,  QUEUE},
{ quick,  QUICK},
{ random, RANDOM},
Index: share/man/man5/pf.conf.5
===
RCS file: /cvs/src/share/man/man5/pf.conf.5,v
retrieving revision 1.536
diff -u -p -r1.536 pf.conf.5
--- share/man/man5/pf.conf.521 Jan 2014 03:15:46 -  1.536
+++ share/man/man5/pf.conf.525 Feb 2014 16:23:35 -
@@ -1501,6 +1501,29 @@ The parent queue must exist.
 .It Ar qlimit Aq Ar limit
 The maximum number of packets held in the queue.
 The default is 50.
+.It Xo Ar quantize
+.No ( Aq Ar overhead ,
+.Aq Ar payload ,
+.Aq Ar size )
+.Xc
+This option is useful for traffic shaping on ATM links
+like DSL or cable modems.
+With this option it is counted how many units (ATM cells) of size
+.Ar payload
+(48 bytes for ATM) are needed to fit the packet size and
+.Ar overhead .
+The number of units is then multiplied with
+.Ar size
+(53 bytes for ATM). The new size is calculated using this formula:
+( original size +
+.Ar overhead
++
+.Ar payload
+- 1 ) /
+.Ar payload
+*
+.Ar size
+where '/' is integer division.
 .El
 .Pp
 Packets can be assigned to queues based on filter rules by using the
Index: sys/net/hfsc.c
===
RCS file: 

Add [-q timeout] option to netcat

[Christopher Zimmermann]

Hi,

this patch adds this option to nc(1):

-q timeout
after end-of-file on stdin, wait timeout seconds and then quit.
The default is no timeout.

This should be compatible with the -q option of the netcat-openbsd
package in debian (what the heck ?)
I use this to send simple udp datagrams:
echo 'Hello World!' |obj/nc -uq0 88.88.88.88 88

ok?

Christopher


Index: nc.1
===
RCS file: /cvs/src/usr.bin/nc/nc.1,v
retrieving revision 1.65
diff -u -p -r1.65 nc.1
--- nc.120 Aug 2013 21:05:20 -  1.65
+++ nc.15 Nov 2013 20:09:32 -
@@ -172,6 +172,11 @@ should use, subject to privilege restric
 It is an error to use this option in conjunction with the
 .Fl l
 option.
+.It Fl q Ar timeout
+after end-of-file on stdin, wait
+.Ar timeout
+seconds and then quit.
+The default is no timeout.
 .It Fl r
 Specifies that source and/or destination ports should be chosen
randomly instead of sequentially within a range or in the order that
the system Index: netcat.c
===
RCS file: /cvs/src/usr.bin/nc/netcat.c,v
retrieving revision 1.117
diff -u -p -r1.117 netcat.c
--- netcat.c26 Oct 2013 21:33:29 -  1.117
+++ netcat.c5 Nov 2013 20:09:33 -
@@ -90,6 +90,7 @@ int   Tflag =
-1; /* IP Type of Service int
rtableid = -1; 
 int timeout = -1;
+int fintime = -1;
 int family = AF_UNSPEC;
 char *portlist[PORT_MAX+1];
 char *unix_dg_tmp_socket;
@@ -136,7 +137,7 @@ main(int argc, char *argv[])
rtableid = getrtable();
 
while ((ch = getopt(argc, argv,
-   46DdFhI:i:klNnO:P:p:rSs:tT:UuV:vw:X:x:z)) != -1) {
+   46DdFhI:i:klNnO:P:p:q:rSs:tT:UuV:vw:X:x:z)) != -1) {
switch (ch) {
case '4':
family = AF_INET;
@@ -216,6 +217,12 @@ main(int argc, char *argv[])
errx(1, timeout %s: %s, errstr,
optarg); timeout *= 1000;
break;
+   case 'q':
+   fintime = strtonum(optarg, 0, INT_MAX / 1000,
errstr);
+   if (errstr)
+   errx(1, timeout %s: %s, errstr,
optarg);
+   fintime *= 1000;
+   break;
case 'x':
xflag = 1;
if ((proxy = strdup(optarg)) == NULL)
@@ -782,6 +789,8 @@ readwrite(int nfd)
else if (n == 0) {
if (Nflag)
shutdown(nfd, SHUT_WR);
+   if (fintime = 0)
+   timeout = fintime;
pfd[1].fd = -1;
pfd[1].events = 0;
} else {
@@ -1090,6 +1099,7 @@ help(void)
\t-O length TCP send buffer length\n\
\t-P proxyuser\tUsername for proxy authentication\n\
\t-p port\t Specify local port for remote connects\n\
+   \t-q secs\t Timeout for final net reads after EOF is
read on stdin\n\ \t-r   Randomize remote ports\n\
\t-SEnable the TCP MD5 signature option\n\
\t-s addr\t Local source address\n\
@@ -1113,7 +1123,7 @@ usage(int ret)
fprintf(stderr,
usage: nc [-46DdFhklNnrStUuvz] [-I length] [-i interval]
[-O length]\n \t  [-P proxy_username] [-p source_port] [-s source]
[-T ToS]\n
-   \t  [-V rtable] [-w timeout] [-X proxy_protocol]\n
+   \t  [-V rtable] [-w timeout] [-q timeout] [-X
proxy_protocol]\n \t  [-x proxy_address[:port]] [destination]
[port]\n); if (ret)
exit(1);



-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
1917 680A 723C BF3D 2CA3  0E44 7E24 D19F 34B8 2A2A

signature.asc
Description: PGP signature

Re: Add [-q timeout] option to netcat

[Christopher Zimmermann]

I just noticed my patch's bedaviour is actually different to the
behaviour of the debian version.

for my diff the man page should actually say

-q timeout
after end-of-file on stdin, timeout the connection after
timeout seconds. A timeout of 0 will close the connection after
reading all pending data.

debian version would be:

-q  after EOF on stdin, wait the specified number of seconds and then quit.
If seconds is negative, wait forever.

They implemented it using 

signal(SIGALRM, quit);
alarm(timeout);

I have no preference, since I just care for the -q 0 case.


Christopher

-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
1917 680A 723C BF3D 2CA3  0E44 7E24 D19F 34B8 2A2A

signature.asc
Description: PGP signature

Re: preparations for IP_SENDSRCADDR / race condition in6_pcbsetport()

[Christopher Zimmermann]

On Wed, 7 Aug 2013 07:11:38 +0200
Claudio Jeker cje...@diehard.n-r-g.com wrote:

 On Tue, Aug 06, 2013 at 09:24:13PM +0200, Christopher Zimmermann
 wrote:
  Hi,
  
  I'm currently working towards an IP_SENDSRCADDR implementation.
  As a first step I moved the calls to in_pcbrehash() from 
  in_pcb(dis)connect() and in_pcbbind() to the call sites of those 
  functions. This should save some calls to in_pcbrehash() in the 
  in_pcbconnect() codepath.
  Also I improved code sharing between the IPv4 and IPv6 stacks by 
  implementing a generic in_pcbsetport used by both stacks.
  The next step will be to replace in_pcbconnect() by in_selectsrc() 
  and in_pcbsetport() in udp_output() like the IPv6 stack does it.
  The splsoftnet() lock could then be removed. Also implementing
  IP_SENDSRCADDR will become trivial because in_selectsrc() won't
  modify the struct inpcb like in_pcbconnect() does at the moment.
  
  Not much testing yet. Just surfing the web and sending this mail...
  
  I'd really like some feedback on whether I'm on the right track or 
  doing something stupid here. I have no experience in the OpenBSD 
  IP-stack yet.
  
  
 
 Moving the calls of of in_pcbrehash() to outside of in_pcb.c is wrong.
 This is an internal function for the specific way fast PCB lookups are
 done at the moment.  Having that all over the place in higher level
 code where someone does a bind or connect call is not an option IMO.
 This needs to be done differently, sorry.
 
 -- 
 :wq Claudio

Fixed this. Since I won't call in_pcbconnect from udp_output() anymore
when I'm finished this is no problem at all :)

Now I'm wondering whether the call to in6_pcbsetport() in udp6_output()
should be splnet protected? Isn't it possible for two processes to
acquire the same free port when one finds it before the other has
claimed it via in_pcbrehash() ?

Christopher


-- 
http://gmerlin.de
OpenPGP: http://gmerlin.de/christopher.pub
1917 680A 723C BF3D 2CA3  0E44 7E24 D19F 34B8 2A2A

preparations for IP_SENDSRCADDR / (un)tangle pcb_bind...

[Christopher Zimmermann]

Hi,

I'm currently working towards an IP_SENDSRCADDR implementation.
As a first step I moved the calls to in_pcbrehash() from 
in_pcb(dis)connect() and in_pcbbind() to the call sites of those 
functions. This should save some calls to in_pcbrehash() in the 
in_pcbconnect() codepath.
Also I improved code sharing between the IPv4 and IPv6 stacks by 
implementing a generic in_pcbsetport used by both stacks.
The next step will be to replace in_pcbconnect() by in_selectsrc() 
and in_pcbsetport() in udp_output() like the IPv6 stack does it.
The splsoftnet() lock could then be removed. Also implementing
IP_SENDSRCADDR will become trivial because in_selectsrc() won't
modify the struct inpcb like in_pcbconnect() does at the moment.

Not much testing yet. Just surfing the web and sending this mail...

I'd really like some feedback on whether I'm on the right track or 
doing something stupid here. I have no experience in the OpenBSD 
IP-stack yet.


Christopher



Index: netinet/in_pcb.c
===
RCS file: /cvs/src/sys/netinet/in_pcb.c,v
retrieving revision 1.139
diff -u -p -r1.139 in_pcb.c
--- netinet/in_pcb.c1 Jun 2013 13:25:40 -   1.139
+++ netinet/in_pcb.c6 Aug 2013 18:25:54 -
@@ -218,7 +218,6 @@ in_pcbbind(struct inpcb *inp, struct mbu
 {
struct socket *so = inp-inp_socket;
struct inpcbtable *table = inp-inp_table;
-   u_int16_t *lastport = inp-inp_table-inpt_lastport;
struct sockaddr_in *sin;
u_int16_t lport = 0;
int wild = 0, reuseport = (so-so_options  SO_REUSEPORT);
@@ -293,71 +292,121 @@ in_pcbbind(struct inpcb *inp, struct mbu
inp-inp_laddr = sin-sin_addr;
}
if (lport == 0) {
-   u_int16_t first, last;
-   int count;
+   error = in_pcbsetport(inp, p);
+   if (error != 0)
+   return error;
+   }
+   else
+   inp-inp_lport = lport;
+   return (0);
+}
 
-   if (inp-inp_flags  INP_HIGHPORT) {
-   first = ipport_hifirstauto; /* sysctl */
-   last = ipport_hilastauto;
-   } else if (inp-inp_flags  INP_LOWPORT) {
-   if ((error = suser(p, 0)))
-   return (EACCES);
-   first = IPPORT_RESERVED-1; /* 1023 */
-   last = 600;/* not IPPORT_RESERVED/2 */
-   } else {
-   first = ipport_firstauto;   /* sysctl */
-   last  = ipport_lastauto;
-   }
+int
+in_pcbsetport(struct inpcb *inp, struct proc *p)
+{
+   struct socket *so = inp-inp_socket;
+   struct inpcbtable *table = inp-inp_table;
+   u_int16_t first, last;
+   u_int16_t *lastport = inp-inp_table-inpt_lastport;
+   u_int16_t lport = 0;
+   int count;
+   int error;
+   int wild;
+   u_int rtableid;
+   void *faddrp, *laddrp;
+#ifdef INET6
+   extern struct in6_addr zeroin6_addr;
 
-   /*
-* Simple check to ensure all ports are not used up causing
-* a deadlock here.
-*
-* We split the two cases (up and down) so that the direction
-* is not being tested on each round of the loop.
-*/
+   if (sotopf(so) == PF_INET6) {
+   wild = INPLOOKUP_IPV6;
+   rtableid = 0;
+   faddrp = zeroin6_addr;
+   laddrp = inp-inp_laddr6;
+   }
+   else
+#endif /* INET6 */
+   {
+   wild = 0;
+   rtableid = inp-inp_rtableid;
+   faddrp = zeroin_addr;
+   laddrp = inp-inp_laddr;
+   }
 
-   if (first  last) {
-   /*
-* counting down
-*/
-   count = first - last;
-   if (count)
-   *lastport = first - arc4random_uniform(count);
+   if ((so-so_options  (SO_REUSEADDR|SO_REUSEPORT)) == 0 
+   ((so-so_proto-pr_flags  PR_CONNREQUIRED) == 0 ||
+(so-so_options  SO_ACCEPTCONN) == 0))
+   wild |= INPLOOKUP_WILDCARD;
 
-   do {
-   if (count--  0)/* completely used? */
-   return (EADDRNOTAVAIL);
-   --*lastport;
-   if (*lastport  first || *lastport  last)
-   *lastport = first;
-   lport = htons(*lastport);
-   } while (in_baddynamic(*lastport, 
so-so_proto-pr_protocol) ||
-   in_pcblookup(table, zeroin_addr, 0,
-   inp-inp_laddr, lport, wild, inp-inp_rtableid));
-   } else {
- 

rare wrong routing decision

[Christopher Zimmermann]

Hi,

I'm still struggeling with my IPsec settup and routing.
This time I'm wondering how the third cloned route below was created.

172.26.153.0/28link#1 UC 00 - 4 em0  
172.26.153/24  link#5 UCS10 - 9 vether0
172.26.153.1   link#5 UHLc   3  180 - L   9 vether0

The system works fine for hours with 

172.26.153.1   00:0d:b9:24:60:40  UHLc   01 - 4 em0

as you would expect, then suddenly network connection breaks due to the
wrong routing decision you see above. Why?

Christopher

in6_unlink_ifa: interface address has no prefix

[Christopher Zimmermann]

Hi,

I have this as /etc/hostname.lo1:

inet 172.26.153.50 0xff00 NONE mtu 1398

!route add 172.26.153.0/24 172.26.153.50
!route add default 172.26.153.50 -priority 12

and see the following on boot:

in6_unlink_ifa: interface address 0x80624a00 has no prefix
in6_unlink_ifa: interface address 0x80624a00 has no prefix

by adding -inet6 to the first line I can work around this warning.


Christopher

Re: tmpfs

[Christopher Zimmermann]

On Tue, 9 Apr 2013 17:26:39 +0100
Pedro Martelletto pe...@ambientworks.net wrote:

 You will need a new kernel with option TMPFS enabled, fresh include
 files and mount_tmpfs. tmpfs is a better MFS, it is faster and can
 free unused memory.
 
 I have put up an updated diff at:
 
 http://block.io/tmp/tmpfs3.diff
 
 This diff fixes a couple of 64 vs 32-bit inconsistencies in the
 original diff, as well as a missing call to uvm_vnp_uncache() upon
 truncation. Many thanks to Chris Jackman for testing on sparc64.

Hi, 

I'd like to test this, but I'm not very experienced with building only
part of the userland. How do I tell the build system to install the new
header files?

Thanks,
Christopher

Re: vr(4) baby jumbos

[Christopher Zimmermann]

On Thu, 7 Feb 2013 17:41:12 +
Stuart Henderson s...@spacehopper.org wrote:

 This is extremely useful as it permits carrying stacked vlans
 on Alix/net5501, and also permits carrying 1500 MTU packets within
 pppoe(4) using the RFC4638 support.

So with 5.3 I can drop the max-mss 1452 on my alix router? Great!!!
Have much thanks for working this out!

Christopher

Re: em(4): enable TCP/UDP checksum offload

[Christopher Zimmermann]

On Sun, 4 Nov 2012 16:06:31 +0100
Christopher Zimmermann madro...@gmerlin.de wrote:

 On Sun, 4 Nov 2012 14:46:34 +0100
 Stefan Sperling s...@openbsd.org wrote:
 
  On Sun, Nov 04, 2012 at 01:21:38PM +, Christian Weisgerber wrote:
   Christian Weisgerber na...@mips.inka.de wrote:
   
Like bge(4), we previously couldn't enable TCP/UDP transmit checksum
offload on em(4).  We can now.

Works fine here on
  em0 at pci1 dev 1 function 0 Intel PRO/1000MT (82540EM) rev 0x02
and with VLAN on
  em0 at pci5 dev 0 function 0 Intel PRO/1000 MT (82574L) rev 0x00
   
   It does however fail on
 em0 at pci1 dev 0 function 0 Intel I350 rev 0x01
   
   (Watchdog timeouts, interface is unusable.)
  
  This one is working fine:
  em0 at pci1 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00
 
 here too:
 em0 at pci0 dev 25 function 0 Intel ICH9 IGP M AMT rev 0x03: msi


and using it actively as NFS (udp) client.

Re: bind mountd to a specified port

[Christopher Zimmermann]

On Thu, 18 Oct 2012 19:11:37 +0200
Sebastian Reitenbach sebas...@l00-bugdead-prods.de wrote:

 getting NFS through a firewall is not that trivial with mountd binding to a 
 random port each time it starts.


Hi,

here is how I solved it after my proposal for a fixed-port-option was
rejected:

/etc/rc.local:

[...]
# register (random) rpc ports to pf anchor
rpcinfo -p localhost \
|awk \
'
/tcp/ \
{printf pass in proto %s to port %d\n, $3, $4}
/udp/ \
{printf pass in proto %s to port %d\n, $3, $4; \
 printf pass out proto %s from port %d\n, $3, $4}
' \
|pfctl -a rpc -f -
[...]


/etc/pf.conf:

# NFS
# rpc daemons use more or less random ports; they will be put into this anchor
anchor rpc on {tun0 lo0} from nfs to (self)
anchor rpc on {tun0 lo0} from (self) to nfs
# local daemons need access to portmap before the rpc anchor is populated
pass in on lo0 inet proto {udp tcp} from localhost to localhost port portmap

smtpd: fatal: qwalk: opendir: No such file or directory

[Christopher Zimmermann]

I have no clue what is wrong here. maybe just the '0' before 'ab3' got
lost?

% sudo smtpd -vd
startup [debug mode]
parent_send_config: configuring smtp
parent_send_config_client_certs: configuring smtp
parent_send_config_ruleset: reloading rules and maps
parent_send_config_ruleset: reloading rules and maps
ramqueue: queue loading in progress
q-path is /queue/ab3  -- fprintf line added by me.
fatal: qwalk: opendir: No such file or directory
queue handler exiting
control process exiting
lost child: runner exited abnormally
mail filter exiting
mail delivery agent exiting
smtp server exiting
lookup agent exiting
mail transfer agent exiting
parent terminating
% sudo ls -l /var/spool/smtpd/queue
total 16
drwx--  3 _smtpd  wheel  512 May  8  2011 0151
drwx--  3 _smtpd  wheel  512 Jul 14 15:12 0425
drwx--  3 _smtpd  wheel  512 May  8  2011 0ab3
drwx--  3 _smtpd  wheel  512 May  8  2011 0f25

PPPoEoA DSL bandwidth shaping

[Christopher Zimmermann]

Hi,

Bandwidth shaping on PPPoE links is difficult, because you don't know
what the modem will do to you packets.

On my DSL PPPoE links the DSL modem sends the ethernet frames over an
ATM circuit and adds additional, but not constant overhead to each
packet in the process.

The consequence is that the raw IP throughput (which is limited by the
traffic shaper) which will saturate the link is smaller for small IP
packets and larger for large IP packets.

Now, one could determine the ATM bandwidth needed by a certain packet
by calculating the number of ATM cells needed to encapsulate the
packet.

I made some measurements on my upstream link (224 kbaud). The extreme
cases are:

261pps of very small (56 byte) TCP/IP packets.
This makes 14.27 kbps of IP traffic.
But 527 ATM cells are used per second.

17pps of the largest possible (1448 byte) TCP/IP packets.
This makes 24.04 kbps of IP traffic.
But 522 ATM cells are used per second.

So the number of used ATM cells would be a better measure for traffic
shaping on PPPoE(oA) links.

I'm thinking of adding an option to altq that would estimate bandwidth
usage by calculating the number of needed ATM cells. Is this the right
approach and is there a chance to get this into the tree?


Regards,
Christopher Zimmermann

aue(4) states half-duplex by default ?

[Christopher Zimmermann]

Hi,

I just started using a aue(4) device. Now I am a bit confused about
full-duplex / half-duplex operation.


The manpage states:

full-duplex
Force full duplex operation.  The interface will operate in
half duplex mode if this media option is not specified.

But for me ifconfig aue0 says:

aue0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500
lladdr 00:05:1b:e5:9a:02
priority: 0
groups: egress
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet6 fe80::205:1bff:fee5:9a02%aue0 prefixlen 64 scopeid 0x6
inet 192.168.1.37 netmask 0xff00 broadcast 192.168.1.255


My hostname.aue0 just says dhcp. Nothing about full-duplex.

Which one is wrong now? Manpage or ifconfig?


Christopher

Re: aue(4) states half-duplex by default ?

[Christopher Zimmermann]

On 09/27/11 20:05, Brad wrote:
 On 27/09/11 1:59 PM, Christopher Zimmermann wrote:
 Hi,

 I just started using a aue(4) device. Now I am a bit confused about
 full-duplex / half-duplex operation.


 The manpage states:

 full-duplex
  Force full duplex operation.  The interface will operate in
  half duplex mode if this media option is not specified.

 But for me ifconfig aue0 says:

 aue0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST  mtu 1500
  lladdr 00:05:1b:e5:9a:02
  priority: 0
  groups: egress
  media: Ethernet autoselect (100baseTX full-duplex)
  status: active
  inet6 fe80::205:1bff:fee5:9a02%aue0 prefixlen 64 scopeid 0x6
  inet 192.168.1.37 netmask 0xff00 broadcast 192.168.1.255


 My hostname.aue0 just says dhcp. Nothing about full-duplex.

 Which one is wrong now? Manpage or ifconfig?
 
 Neither. You're not understanding the man page correctly and took that
 piece of the page out of context. The manual duplex setting goes along
 with the two manual speed selections listed above that in the man page.
 

Thats just ridiculous. the 'media options' section is a section of its
own as well as the 'media types' section. The 'media types' section has
three items. 'autoselect' '10baseT' and '100baseTX'. It would be stupid
to assume that the explanation in the following section 'media options'
only apply to the last two(!) items in the previous section.

I think the manpage is very clear and leaves little room for
interpretation.


What about changing the 'full-duplex' section to:

full-duplex
 Force full duplex operation.

And the 'autoselect' section to:

autoselect
Enable autoselection of the media type and options.  Full
duplex mode will be selected if possible.  The user can
manually override the autoselected mode by adding media
options to the appropriate hostname.if(5) file.

Re: custom udp/tcp ports for mountd, rpc.statd, rpc.lockd

[Christopher Zimmermann]

Hi,

now here also for rpc.statd and rpc.lockd. But I could only test
mountd, because I have no clients for lockd. I would really like to see 
this committed in cvs.


Cheers,
Christopher


Index: sbin/mountd/mountd.8
===
RCS file: /cvs/src/sbin/mountd/mountd.8,v
retrieving revision 1.16
diff -u -p -r1.16 mountd.8
--- sbin/mountd/mountd.831 May 2007 19:19:46 -  1.16
+++ sbin/mountd/mountd.827 Jul 2011 21:15:12 -
@@ -63,6 +63,8 @@ Enable debugging mode.
 .Nm
 will not detach from the controlling terminal and will print
 debugging messages to stderr.
+.It Fl n Ar port
+Specifies which udp and tcp port to bind to.
 .It Fl n
 Do not require that clients make mount requests from reserved ports.
 (Normally, only mount requests from reserved ports are accepted.)
Index: sbin/mountd/mountd.c
===
RCS file: /cvs/src/sbin/mountd/mountd.c,v
retrieving revision 1.71
diff -u -p -r1.71 mountd.c
--- sbin/mountd/mountd.c22 Mar 2010 16:35:27 -  1.71
+++ sbin/mountd/mountd.c27 Jul 2011 21:15:14 -
@@ -211,11 +211,15 @@ volatile sig_atomic_t gotterm;
 int
 main(int argc, char *argv[])
 {
+   struct sockaddr_in inetaddr;
+   int port=0;
+   int udpsock, tcpsock;
SVCXPRT *udptransp, *tcptransp;
FILE *pidfile;
+   const char *errstr = NULL;
int c;

-   while ((c = getopt(argc, argv, dnr)) != -1)
+   while ((c = getopt(argc, argv, dnp:r)) != -1)
switch (c) {
case 'd':
debug = 1;
@@ -223,11 +227,18 @@ main(int argc, char *argv[])
case 'n':
resvport_only = 0;
break;
+   case 'p':
+   port = strtonum(optarg, 1, (116) - 1, errstr);
+   if (errstr) {
+   fprintf(stderr, mountd port is %s: %s\n, 
errstr, optarg);
+   return(1);
+   }
+   break;
case 'r':
/* Compatibility */
break;
default:
-   fprintf(stderr, usage: mountd [-dn] [exportsfile]\n);
+   fprintf(stderr, usage: mountd [-dn] [-p port] 
[exportsfile]\n);
exit(1);
}
argc -= optind;
@@ -273,8 +284,51 @@ main(int argc, char *argv[])
signal(SIGHUP, (void (*)(int)) new_exportlist);
signal(SIGTERM, (void (*)(int)) send_umntall);
signal(SIGSYS, SIG_IGN);
-   if ((udptransp = svcudp_create(RPC_ANYSOCK)) == NULL ||
-   (tcptransp = svctcp_create(RPC_ANYSOCK, 0, 0)) == NULL) {
+   
+   /* Create tcp/udp sockets */
+   if(port == 0)
+   udpsock = tcpsock = RPC_ANYSOCK;
+   else {
+   if ((udpsock = socket(AF_INET, SOCK_DGRAM, 0))  0) {
+   syslog(LOG_ERR, can't create udp socket);
+   return (1);
+   }
+   memset(inetaddr, 0, sizeof inetaddr);
+   inetaddr.sin_family = AF_INET;
+   inetaddr.sin_addr.s_addr = INADDR_ANY;
+   inetaddr.sin_port = htons(port);
+   inetaddr.sin_len = sizeof(inetaddr);
+   if (bind(udpsock, (struct sockaddr *)inetaddr,
+   sizeof(inetaddr))  0) {
+   syslog(LOG_ERR, can't bind udp addr);
+   return (1);
+   }
+
+
+   if ((tcpsock = socket(AF_INET, SOCK_STREAM, 0))  0) {
+   syslog(LOG_ERR, can't create tcp socket);
+   return (1);
+   }
+   memset(inetaddr, 0, sizeof inetaddr);
+   inetaddr.sin_family = AF_INET;
+   inetaddr.sin_addr.s_addr = INADDR_ANY;
+   inetaddr.sin_port = htons(port);
+   inetaddr.sin_len = sizeof(inetaddr);
+   if (bind(tcpsock, (struct sockaddr *)inetaddr,
+   sizeof (inetaddr))  0) {
+   syslog(LOG_ERR, can't bind tcp addr);
+   return (1);
+   }
+#if 0
+   if (listen(tcpsock, 5)  0) {
+   syslog(LOG_ERR, listen failed);
+   return (1);
+   }
+#endif
+   }
+
+   if ((udptransp = svcudp_create(udpsock)) == NULL ||
+   (tcptransp = svctcp_create(tcpsock, 0, 0)) == NULL) {
syslog(LOG_ERR, Can't create socket);
exit(1);
}
Index: usr.sbin/rpc.statd/rpc.statd.8
===
RCS file: /cvs/src/usr.sbin/rpc.statd/rpc.statd.8,v
retrieving revision 1.2
diff -u -p -r1.2 rpc.statd.8
--- usr.sbin/rpc.statd/rpc.statd.8  15 Jun 2008 20:42:42 -  

Re: custom udp/tcp ports for mountd, rpc.statd, rpc.lockd

[Christopher Zimmermann]

On 07/27/11 23:42, Nicholas Marriott wrote:

Do you have a particular usage that needs this?


No, I just run a local nfs server; at the moment only serving one
single, trusted client.
So I'm not in desperate need for fixed ports, but I think fixed ports
are a lot cleaner and over all easier to maintain.
For example when I run a netstat -na, tcpdump or pf I want to be able
to recognize the ports I see and use.
Of course, Theo is right. I don't get a direct gain in security by
tightening up my firewall against trusted hosts which have NFS access 
anyhow. But still the network as a whole gets more transparent and

maintainable. I know whats going on and I can easily block, allow or
inspect NFS and non-NFS traffic.

One problem I haven't run into yet, but could imagine: Could the random 
rpc services occupy a port, which services started later on, for example 
cups (631 TCP), would need?


Just out of curiosity, why those random port assignments in the first
place? Does this have or did it have any advantages? Maybe to be able
to run different versions of the same rpc server at the same time?


Cheers,
Christopher



Some minor nits on this diff

- I would use UINT16_MAX or SHORT_MAX instead of (116)-1.


done


- Spaces around = in port=0 please.


done


- Rest of main() uses exit(1) not return (1) and err/errx not fprintf
   (aside from for usage) should stay consistent.


adjusted to fit the corresponding file.


- Why do you have a bit under #if 0? It's either needed or not?


not needed - removed.


- I think you missed updating the synopsis in the man pages.


updated.


- UDP and TCP should be uppercase in man page, and I'd say comments and
   error messages too.


There already were error messages in lower case. So I left the program
output in lowercase, but changed comments and documentation to
uppercase.


- Wrap lines to below 80 columns please.


done.




On Wed, Jul 27, 2011 at 11:21:52PM +0200, Christopher Zimmermann wrote:

Hi,

now here also for rpc.statd and rpc.lockd. But I could only test
mountd, because I have no clients for lockd. I would really like to
see this committed in cvs.

Cheers,
Christopher



Index: sbin/mountd/mountd.8
===
RCS file: /cvs/src/sbin/mountd/mountd.8,v
retrieving revision 1.16
diff -u -p -r1.16 mountd.8
--- sbin/mountd/mountd.831 May 2007 19:19:46 -  1.16
+++ sbin/mountd/mountd.827 Jul 2011 23:14:31 -
@@ -41,6 +41,7 @@ mount requests
 .Sh SYNOPSIS
 .Nm mountd
 .Op Fl dn
+.Op Fl p Ar port
 .Op Ar exportsfile
 .Sh DESCRIPTION
 .Nm
@@ -63,6 +64,8 @@ Enable debugging mode.
 .Nm
 will not detach from the controlling terminal and will print
 debugging messages to stderr.
+.It Fl n Ar port
+Specifies which UDP and TCP port to bind to.
 .It Fl n
 Do not require that clients make mount requests from reserved ports.
 (Normally, only mount requests from reserved ports are accepted.)
Index: sbin/mountd/mountd.c
===
RCS file: /cvs/src/sbin/mountd/mountd.c,v
retrieving revision 1.71
diff -u -p -r1.71 mountd.c
--- sbin/mountd/mountd.c22 Mar 2010 16:35:27 -  1.71
+++ sbin/mountd/mountd.c27 Jul 2011 23:14:33 -
@@ -57,6 +57,7 @@
 #include netgroup.h
 #include pwd.h
 #include signal.h
+#include stdint.h
 #include stdio.h
 #include stdlib.h
 #include string.h
@@ -211,11 +212,15 @@ volatile sig_atomic_t gotterm;
 int
 main(int argc, char *argv[])
 {
+   struct sockaddr_in inetaddr;
+   int port = 0;
+   int udpsock, tcpsock;
SVCXPRT *udptransp, *tcptransp;
FILE *pidfile;
int c;
+   const char *errstr = NULL;

-   while ((c = getopt(argc, argv, dnr)) != -1)
+   while ((c = getopt(argc, argv, dnp:r)) != -1)
switch (c) {
case 'd':
debug = 1;
@@ -223,11 +228,20 @@ main(int argc, char *argv[])
case 'n':
resvport_only = 0;
break;
+   case 'p':
+   port = strtonum(optarg, 1, UINT16_MAX, errstr);
+   if (errstr) {
+   fprintf(stderr,
+   mountd port is %s: %s\n, errstr, optarg);
+   exit(1);
+   }
+   break;
case 'r':
/* Compatibility */
break;
default:
-   fprintf(stderr, usage: mountd [-dn] [exportsfile]\n);
+   fprintf(stderr,
+   usage: mountd [-dn] [-p port] [exportsfile]\n);
exit(1);
}
argc -= optind;
@@ -273,8 +287,44 @@ main(int argc, char *argv[])
signal(SIGHUP, (void (*)(int)) new_exportlist);
signal(SIGTERM, (void (*)(int)) send_umntall);
signal

Re: custom udp/tcp ports for mountd, rpc.statd, rpc.lockd

[Christopher Zimmermann]

On 07/28/11 01:33, Theo de Raadt wrote:

On 07/27/11 23:42, Nicholas Marriott wrote:

RPC does not work that way.  It uses the portmapper at port 111 for
discovery.  NFS at 2049 is also a known port.  The rest are supposed
to be unknown.



Unfortunately it isn't supposed to work that way.

  ^

That it is not supposed to work that way doesn't mean that it won't
work that way.
It could even work _better_ in another way. As you say yourself, ntpd
does have a fixed port number (2049). Why not allow (not enforce)
similar behaviour for mountd, too? It would have some advantages.


Except that isn't how it is supposed to work with RPC.  An extension
of where you are going would be to add such functionality to every
single program which uses RPC, either as a server or even as a client.
And that is surely over the top.


I don't think clients need this functionality, they can still ask the
portmapper as they always did. My intention was not to redesign NFSv3
(that would be NFSv4), but to improve it's maintainability where it can 
be done without breaking anything.



Just out of curiosity, why those random port assignments in the first
place? Does this have or did it have any advantages? Maybe to be able
to run different versions of the same rpc server at the same time?


We applied this as principle around 10 years ago.  Anything which does
not need to be sequential is random, just because developers make
stupid assumptions and use them as ran.


I understand this principle of randomization and its advantages for bug
hunting and security. But I don't see how this principle of
randomization could yield anything in this setting of randomized ports
for mountd, which are discoverable via portmap anyway. There are no
established ports for mountd and it doesn't look like default ports
will be established for mountd, so no stupid assumptions to catch here.


Do you see any disadvantages in using fixed ports ?

Do you see any serious enought advantages in randomizing port
assignments to RPC services to justify enforcing this behaviour?


Cheers,
Christopher

USB keyboard in boot_config(8)

[Christopher Zimmermann]

Hi,

I just installed the June 29 snapshot and noticed that my USB keyboard
no longer works in the boot_config(8) prompt. It still did work in the
June 24 snapshot. pckbd(4) works fine.


Christopher

Re: sdn allocation and umass(4)

[Christopher Zimmermann]

On 06/24/11 18:46, Joel Sing wrote:
 On Friday 24 June 2011, Benny Lofgren wrote:
 - More consistent sdn unit allocation (perhaps this is achievable
   with DUID, I haven't had time to explore that yet)
 
 sd(4) unit allocation will always be inconsistent and unpredicatable 
 - DUIDs will let you avoid this entirely.


By the way, is there a way to mount umass(4) devices without looking at
dmesg for the number of the sdn device?

Re: sdn allocation and umass(4)

[Christopher Zimmermann]

On 06/24/11 19:40, Alexander Polakov wrote:
 * Christopher Zimmermann madro...@zakweb.de [110624 21:24]:
 On 06/24/11 18:46, Joel Sing wrote:
 On Friday 24 June 2011, Benny Lofgren wrote:
 - More consistent sdn unit allocation (perhaps this is achievable
   with DUID, I haven't had time to explore that yet)

 sd(4) unit allocation will always be inconsistent and unpredicatable 
 - DUIDs will let you avoid this entirely.


 By the way, is there a way to mount umass(4) devices without looking at
 dmesg for the number of the sdn device?

 
 hotplugd(8)

That's not what I thought about, but even better - hotplugd the BSD way.
Just great :) Thanks!

Re: nc(1), GNU netcat and FIN segments after all data has been sent

[Christopher Zimmermann]

On 06/23/11 14:10, Andreas Bartelt wrote:
 Hello,
 
 I've noticed that there's a difference in behavior between nc(1) and GNU
 netcat when they talk to some daemon via TCP.
 
 The commands in the following example are basically the same:
 
 GNU netcat:
 netcat host 1234  infile
 
 nc(1):
 nc host 1234  infile
 
 nc(1) sends a FIN segment after all data has been read from stdin:
 shutdown(nfd, SHUT_WR) in netcat.c causes TCP to enter FIN-WAIT-1 state.
 GNU netcat doesn't do this. I've noticed that some daemons behave
 differently because of this, i.e., they won't return any data although
 they are still allowed to send data.

Maybe you can force nc(1) not to send a FIN segment by using something
like this:

cat infile - |nc host 1234


Christopher

 I think both variants are allowed in RFC 793. Would it make sense to add
 a further option to nc(1) which allows to toggle between both variants?
 
 Regards
 Andreas

Re: libsndio: remove support for legacy device names

[Christopher Zimmermann]

On 05/03/11 22:58, Alexandre Ratchov wrote:
 hack to support legacy audio/midi device naming scheme is almost two
 years old, ok to drop support for it?
 
 -- Alexandre


This caught me by surprise. I spent about an hour in gdb and the source
to find out what was wrong.
aucat just told me this:

% aucat -l -f /dev/audio1
aucat: /dev/audio1: can't open device
/dev/audio1: failed to open audio device

I don't know why, but the error message from sio_open about a missing
':' did not come through. This would have helped a lot.

Maybe add a note to faq/current.html?


Christopher

Re: [smtpd] diff that needs testing

[Christopher Zimmermann]

On 05/15/11 11:57, Gilles Chehade wrote:

Please test and report breakage of your existing setups :-)


No breakage for me. I use smtpd as MTA for local-to-local delivery and
to relay mail to my ISP using tls.



#   $OpenBSD: smtpd.conf,v 1.2 2009/11/03 22:32:10 gilles Exp $

# This is the smtpd server system-wide configuration file.
# See smtpd.conf(5) for more information.

listen on lo0
listen on lo0 port 587

map aliases { source db /etc/mail/aliases.db }
map secrets { source db /etc/mail/secrets.db }

accept for local alias aliases deliver to mbox
accept for all relay via smtp.1und1.de port 587 tls auth secrets

Re: Fan mode management in acpithinkpad(4)

[Christopher Zimmermann]

On 05/13/11 20:56, Alexander Polakov wrote:

* Marco Peereboomsl...@peereboom.us  [110512 17:59]:

On Thu, May 12, 2011 at 03:32:56PM +0200, Christopher Zimmermann wrote:

On 05/12/11 14:37, Vadim Zhukov wrote:

Hello all.

Here is a patch that allows for me to work on other things. :) Basically,
it makes OS choose fan mode instead of firmware. Main feature here is
enabling of disengadged mode when temperature goes critical, picking
80C as a red line. Now I can fully load CPU on my X201i -
say, make -j 4 - and it still works instead of being powering off by
acpitz(4).



User space will not be allowed to play.  I don't have a stinkpad so I
can't test this but I do encourage people to play with this diff and
report to the list.


My thinkpad is AMD-powered, so 60 degrees is *normal* for it. With this
diff the fan is always running at max speed, generating lots of noise.



I think the THINKPAD_FANMODE_MAX part could be left out. The normal 
automatic should be able to set maximum speed at a model specific 
appropriate temperature. The important part is the disengaged mode, 
because the automatic regulation of the firmware is not able to set this 
mode.


Christopher

Re: Fan mode management in acpithinkpad(4)

[Christopher Zimmermann]

On 05/12/11 14:37, Vadim Zhukov wrote:

Hello all.

Here is a patch that allows for me to work on other things. :) Basically,
it makes OS choose fan mode instead of firmware. Main feature here is
enabling of disengadged mode when temperature goes critical, picking
80C as a red line. Now I can fully load CPU on my X201i -
say, make -j 4 - and it still works instead of being powering off by
acpitz(4).

All information was taken from Linux's thinkpad_acpi.c:
http://lxr.free-electrons.com/source/drivers/platform/x86/thinkpad_acpi.c
(look at 2f register).

I also fixed a few style nits here and there.



That's a nice approach you are taking here. Switching to disengaged only 
when cpu gets too hot and letting firmware handle fan autoregulation 
otherwise. When I find the time I, maybe I will finish my patch to allow 
fan regulation from userspace, but then certeinly keep your approach as 
failsafe.



Christopher

Re: bioctl should retry passphrase

[Christopher Zimmermann]

On 01/14/11 00:51, Ted Unangst wrote:
 If I type the wrong password into bioctl at boot, disks don't exist, 
 filesystems don't get mounted, and generally lots of things go wrong.  All 
 I need is a second chance to remind me to type the right password.

In /etc/rc I simply do this:

[...]

# XXX - my own changes - enable encrypted softraid
echo -n 'wd0p '
until bioctl -c C -l /dev/wd0p softraid0
do echo -n 'try again wd0p '
done

# Check parity on raid devices.
raidctl -P all

swapctl -A -t blk



Christopher

 
 Index: bioctl.c
 ===
 RCS file: /home/tedu/cvs/src/sbin/bioctl/bioctl.c,v
 retrieving revision 1.98
 diff -u -r1.98 bioctl.c
 --- bioctl.c  1 Dec 2010 19:40:18 -   1.98
 +++ bioctl.c  13 Jan 2011 23:47:24 -
 @@ -699,6 +699,7 @@
   int rv, no_dev, fd;
   dev_t   *dt;
   u_int16_t   min_disks = 0;
 + int retry = 0;
  
   if (!dev_list)
   errx(1, no devices specified);
 @@ -738,6 +739,7 @@
   if (level == 'C'  no_dev != min_disks)
   errx(1, not exactly one partition);
  
 +again:
   memset(create, 0, sizeof(create));
   create.bc_cookie = bl.bl_cookie;
   create.bc_level = level;
 @@ -802,8 +804,14 @@
   memset(kdfinfo, 0, sizeof(kdfinfo));
   memset(create, 0, sizeof(create));
   if (rv == -1) {
 - if (errno == EPERM)
 + if (errno == EPERM) {
 + if (!retry) {
 + warnx(Incorrect passphrase. Try again.);
 + retry = 1;
 + goto again;
 + }
   errx(1, Incorrect passphrase);
 + }
   err(1, BIOCCREATERAID);
   }

acpi and sysctl

[Christopher Zimmermann]

Hi,

I've finally found some time to work on the fan control support for my
thinkpad.
But I'm having problems calling to acpiec_write() from sysctl or
timeout_set(9) context. (Assertion failure in acpiec_gpehandler()).
According to dev/acpi/acpiec.c:229 this function is meant to be called
only from acpi thread context.
Now how am I supposed to talk to the acpi controller?
One solution would be to use the aml_register_notify(..., ACPIDEV_POLL)
callback mechanism already in use by the acpithinkpad(4) driver to
somehow get into acpi thread context. The problem is that this callback
mechanism is hardcoded to 10s intervals. But I need intervals 1s to
implement a workaround described on
http://www.thinkwiki.org/wiki/How_to_control_fan_speed#Disengaged_.28full-speed.29_mode

any help is appreciated. I'm stuck here.

Christopher



this is what I've got so far:

Index: arch/i386/i386/machdep.c
===
RCS file: /cvs/src/sys/arch/i386/i386/machdep.c,v
retrieving revision 1.485
diff -u -p -r1.485 machdep.c
--- arch/i386/i386/machdep.c2 Oct 2010 23:31:34 -   1.485
+++ arch/i386/i386/machdep.c19 Dec 2010 15:11:09 -
@@ -243,6 +243,7 @@ voidvia_update_sensor(void *args);
 #endif
 int kbd_reset;
 int lid_suspend;
+int fan_control;

 /*
  * safepri is a safe priority for sleep to set for a spin-wait
@@ -3416,6 +3417,8 @@ cpu_sysctl(int *name, u_int namelen, voi
return (sysctl_rdint(oldp, oldlenp, newp, i386_has_xcrypt));
case CPU_LIDSUSPEND:
return (sysctl_int(oldp, oldlenp, newp, newlen, lid_suspend));
+   case CPU_FANCONTROL:
+   return (sysctl_int(oldp, oldlenp, newp, newlen, fan_control));
default:
return (EOPNOTSUPP);
}
Index: arch/i386/include/cpu.h
===
RCS file: /cvs/src/sys/arch/i386/include/cpu.h,v
retrieving revision 1.117
diff -u -p -r1.117 cpu.h
--- arch/i386/include/cpu.h 2 Oct 2010 23:13:28 -   1.117
+++ arch/i386/include/cpu.h 19 Dec 2010 15:11:09 -
@@ -461,7 +461,8 @@ voidvm86_gpfault(struct proc *, int);
 #define CPU_SSE2   15  /* supports SSE2 */
 #define CPU_XCRYPT 16  /* supports VIA xcrypt in userland */
 #define CPU_LIDSUSPEND 17  /* lid close causes a suspend */
-#define CPU_MAXID  18  /* number of valid machdep ids */
+#define CPU_FANCONTROL 18  /* lid close causes a suspend */
+#define CPU_MAXID  19  /* number of valid machdep ids */

 #defineCTL_MACHDEP_NAMES { \
{ 0, 0 }, \
@@ -482,6 +483,7 @@ voidvm86_gpfault(struct proc *, int);
{ sse2, CTLTYPE_INT }, \
{ xcrypt, CTLTYPE_INT }, \
{ lidsuspend, CTLTYPE_INT }, \
+   { fancontrol, CTLTYPE_INT }, \
 }

 /*
Index: dev/acpi/acpithinkpad.c
===
RCS file: /cvs/src/sys/dev/acpi/acpithinkpad.c,v
retrieving revision 1.24
diff -u -p -r1.24 acpithinkpad.c
--- dev/acpi/acpithinkpad.c 7 Aug 2010 16:21:20 -   1.24
+++ dev/acpi/acpithinkpad.c 19 Dec 2010 15:11:17 -
@@ -77,6 +77,16 @@
 #define THINKPAD_ECOFFSET_FANLO0x84
 #define THINKPAD_ECOFFSET_FANHI0x85

+#define THINKPAD_ECOFFSET_FANCTL   0x2f
+#define THINKPAD_FAN_REGVAL0xff
+/* Most thinkpads only support only speed steps 1-7 */
+#define THINKPAD_FAN_MANUAL0x3f
+#define THINKPAD_FAN_DISENGAGE 0x40
+#define THINKPAD_FAN_AUTO  0x80
+/* AntiPulse */
+#define THINKPAD_FAN_AP0x100
+#define THINKPAD_FAN_AP_DISENGAGE  0x200
+
 struct acpithinkpad_softc {
struct devicesc_dev;

@@ -86,8 +96,13 @@ struct acpithinkpad_softc {

struct ksensor   sc_sens[THINKPAD_NSENSORS];
struct ksensordevsc_sensdev;
+
+   /* Fan control */
+   struct timeout  sc_fan_timeout;
 };

+extern int fan_control;
+
 extern void acpiec_read(struct acpiec_softc *, u_int8_t, int, u_int8_t *);

 intthinkpad_match(struct device *, void *, void *);
@@ -103,8 +118,9 @@ int thinkpad_volume_mute(struct acpithin
 intthinkpad_brightness_up(struct acpithinkpad_softc *);
 intthinkpad_brightness_down(struct acpithinkpad_softc *);

-voidthinkpad_sensor_attach(struct acpithinkpad_softc *sc);
+voidthinkpad_sensor_attach(struct acpithinkpad_softc *);
 voidthinkpad_sensor_refresh(void *);
+voidthinkpad_fancontrol(void *);

 struct cfattach acpithinkpad_ca = {
sizeof(struct acpithinkpad_softc), thinkpad_match, thinkpad_attach
@@ -185,6 +201,46 @@ thinkpad_sensor_refresh(void *arg)
 }

 void
+thinkpad_fancontrol(void *arg)
+{
+   u_int8_t fanctl;
+
+   struct acpithinkpad_softc *sc = arg;
+
+   if (fan_control  THINKPAD_FAN_AP)
+   {
+   if 

Re: acpithinkpad(4) fan control - sysctl callback in kernel?

[Christopher Zimmermann]

On 11/22/10 23:01, joshua stein wrote:
 as for code implementing the sysctl, it would probably be something
 created under machdep, like the recently added machdep.lidsuspend.

Thanks for your code. It helped me to do a first test of the fan control
on my T43. It reacts strangely (regulates fan speed on only 3 steps from
3000 to 4000 rpm).

Still I'd like to implement a sysctl interface. The example of
lid_suspend did not help much. I need something like a callback when
userspace changes the sysctl variable. Is this possible? Or should I do
it like it is done for lid_suspend and regularly poll the variable?


Christopher

acpithinkpad(4) fan control

[Christopher Zimmermann]

Hi!

I'd like to implement fan speed control for Thinkpads. It is documented
at http://www.thinkwiki.org/wiki/How_to_control_fan_speed#Hardware_specs
and linux also implements this (but with special case for TP 570,
600e/x, 770e, 770x - anyone here with access to one of these?)
Implementing a driver for this will be a piece of cake, but I need help
with communication to userspace to get started. I guess the way to go is
sysctl (?)
In acpithinkpad only read only sensor variables are created.
How do I create r/w variables - where can I find code examples? And where
in the sysctl tree should I put them?

When I have this working I want to implement a PID controler for it.
Since I'd like to see this get committed, where would be the preferred
place to put the PID-controller? Kernel or userland?


Christopher

Re: wlan scan-after-nwid quirk for ThinkPads

[Christopher Zimmermann]

On 07/06/10 10:10, David Coppa wrote:
 On Tue, Jul 6, 2010 at 4:14 AM, Anders 
Langworthylagrang...@gmail.com  wrote:

 On Mon, Jul 5, 2010 at 4:37 PM, Christopher Zimmermann
 madro...@zakweb.de  wrote:

 The only thing I find funny is that the network link only comes
 up after I have run the 'ifconfig iwi0 scan' a second time. See
 below.

 Yeah, this is a quirk with my iwi device[1] also.  A scan needs to be
 run after you change nwid or the network will not come up as active.
 It does work without a scan the first time you connect to a network
 after the machine has been booted, though.

But not if you do a scan before bringing the inferface up for the
first time - as in my case.

 [1]: iwi0 at pci2 dev 2 function 0 Intel PRO/Wireless 2915ABG rev 0x05



 I think there's a probable regression with:

 ppb1 at pci0 dev 30 function 0 Intel 82801BAM Hub-to-PCI rev 0xd3
 pci2 at ppb1 bus 4

that's what my dmesg says about this:

ppb3 at pci0 dev 30 function 0 Intel 82801BAM Hub-to-PCI rev
0xd3
pci4 at ppb3 bus 11
iwi0 at pci4 dev 2 function 0 Intel PRO/Wireless 2200BG rev 0x05: irq 
11, address 00:12:f0:62:22:ba


 I have a ralink on my ThinkPad X41 that shows the same weird behavior.

 ral0 at pci2 dev 2 function 0 Ralink RT2561 rev 0x00: irq 10,
 address 00:0d:f0:3e:6e:a0
 ral0: MAC/BBP RT2561C, RF RT2527

 And the same ifconfig scan trick is also needed if I swap the card
 with an Atheros minipci, so I think it's not related to a particular
 wireless device.

 No quirks at all with another Ralink card on cardbus (MSI CB54G2)...
 And I'm 99% sure I didn't have this problem before.

 cheers,
 David

my complete dmesg:

OpenBSD 4.7-current (GENERIC) #34: Wed Jun 23 22:16:39 MDT 2010
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Pentium(R) M processor 2.13GHz (GenuineIntel 686-class) 
2.13 GHz
cpu0: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,TM,SBF,EST,TM2

real mem  = 2145808384 (2046MB)
avail mem = 2067968000 (1972MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 08/21/06, BIOS32 rev. 0 @ 0xfd760, 
SMBIOS rev. 2.33 @ 0xe0010 (64 entries)

bios0: vendor IBM version 1YET65WW (1.29 ) date 08/21/2006
bios0: IBM 2668H2G
apm0 at bios0: Power Management spec V1.2
apm0: battery life expectancy 96%
apm0: AC on, battery charge high
acpi at bios0 function 0x0 not configured
pcibios0 at bios0: rev 2.1 @ 0xfd6f0/0x910
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdeb0/256 (14 entries)
pcibios0: PCI Interrupt Router at 000:31:0 (Intel 82371FB ISA rev 0x00)
pcibios0: PCI bus #12 is the last bus
bios0: ROM list: 0xc/0x1 0xd/0x1600 0xd1800/0x1000 
0xdc000/0x4000! 0xe/0x1

cpu0 at mainbus0: (uniprocessor)
cpu0: Enhanced SpeedStep 2129 MHz: speeds: 2133, 1867, 1600, 1333, 1067, 
800 MHz

pci0 at mainbus0 bus 0: configuration mode 1 (bios)
io address conflict 0x5800/0x8
io address conflict 0x5808/0x4
io address conflict 0x5810/0x8
io address conflict 0x580c/0x4
pchb0 at pci0 dev 0 function 0 Intel 82915GM Host rev 0x03
ppb0 at pci0 dev 1 function 0 Intel 82915GM PCIE rev 0x03: irq 11
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 ATI FireGL M24 GL rev 0x80
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
radeondrm0 at vga1: irq 11
drm0 at radeondrm0
ppb1 at pci0 dev 28 function 0 Intel 82801FB PCIE rev 0x03: irq 11
pci2 at ppb1 bus 2
bge0 at pci2 dev 0 function 0 Broadcom BCM5751M rev 0x11, BCM5750 B1 
(0x4101): irq 11, address 00:11:25:ae:0e:0c

brgphy0 at bge0 phy 1: BCM5750 10/100/1000baseT PHY, rev. 0
ppb2 at pci0 dev 28 function 2 Intel 82801FB PCIE rev 0x03: irq 11
pci3 at ppb2 bus 3
uhci0 at pci0 dev 29 function 0 Intel 82801FB USB rev 0x03: irq 11
uhci1 at pci0 dev 29 function 1 Intel 82801FB USB rev 0x03: irq 11
uhci2 at pci0 dev 29 function 2 Intel 82801FB USB rev 0x03: irq 11
uhci3 at pci0 dev 29 function 3 Intel 82801FB USB rev 0x03: irq 11
ehci0 at pci0 dev 29 function 7 Intel 82801FB USB rev 0x03: irq 11
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1
ppb3 at pci0 dev 30 function 0 Intel 82801BAM Hub-to-PCI rev 0xd3
pci4 at ppb3 bus 11
cbb0 at pci4 dev 0 function 0 Ricoh 5C476 CardBus rev 0x8d: irq 11
iwi0 at pci4 dev 2 function 0 Intel PRO/Wireless 2200BG rev 0x05: irq 
11, address 00:12:f0:62:22:ba

cardslot0 at cbb0 slot 0 flags 0
cardbus0 at cardslot0: bus 12 device 0 cacheline 0x0, lattimer 0xb0
pcmcia0 at cardslot0
auich0 at pci0 dev 30 function 2 Intel 82801FB AC97 rev 0x03: irq 11, 
ICH6 AC97

ac97: codec id 0x41445374 (Analog Devices AD1981B)
ac97: codec features headphone, 20 bit DAC, No 3D Stereo
audio0 at auich0
ichpcib0 at pci0 dev 31 function 0 Intel 82801FBM LPC rev 0x03: PM 
disabled
pciide0 at pci0 dev 31 function 2 Intel 82801FBM SATA rev 0x03: DMA, 
channel 0 wired to compatibility, channel 1 wired to compatibility

wd0 at pciide0 channel 0 drive 0: HTS541080G9AT00
wd0

802.1X - WPA enterprise mode not yet supported ?

[Christopher Zimmermann]

Hi,

it just took me 2 hours to figure out that OpenBSD does not yet
support the WPA enterprise mode (aka 802.1X). Is this actually
true? If yes, please include the patch below to make this clear
in the man page.

I like the high quality of the OpenBSD manpages, but today I
really spent 2 hours searching where to put my certificates.


Christopher


Index: sbin/ifconfig/ifconfig.8
===
RCS file: /cvs/src/sbin/ifconfig/ifconfig.8,v
retrieving revision 1.202
diff -u -p -r1.202 ifconfig.8
--- sbin/ifconfig/ifconfig.828 May 2010 13:23:43 -  1.202
+++ sbin/ifconfig/ifconfig.828 Jun 2010 17:58:11 -
@@ -980,8 +980,8 @@ and
 .Ar psk
 authentication (also known as personal mode) uses a 256-bit pre-shared 
key.

 .Ar 802.1x
-authentication (also known as enterprise mode) is meant to be used with
-an external IEEE 802.1X authentication server.
+authentication (also known as enterprise mode; not yet supported) is 
meant to

+be used with an external IEEE 802.1X authentication server.
 The default value is
 .Dq psk .
 .Dq psk

[PATCH] please test if dmesg has ehci0: timed out waiting for BIOS

[Christopher Zimmermann]

Hi, I'm just touching this again because I'd like to see this
fix committed to CVS.

The problem that is fixed by this change is a complete deadlock 
while booting with USB2 HiSpeed enabled in BIOS.


On Mon, 8 Mar 2010 15:39:55 +0100 Christopher Zimmermann wrote:
 Heres my theory of what went wrong:
 
 My (broken) BIOS does not know how to hand over the ehci host
 controller. Therefore the timeout.

That's the ehci0: timed out waiting for BIOS message

 Additionally when 'HiSpeed' is selected it turns on several System 
 Management Interupts in the ehci host controller and won't turn them 
 off when the kernel requests the handover.
 Later, when the kernel sets the CF bit the host controller issues a 
 SMI to notify the BIOS of the ownership change of mouse and keyboard 
 from ohci to the ehci host controller. The BIOS does not seem to like 
 this and the kernel/BIOS freezes.

- complete deadlock

 My fix simply clears all SMI_ENABLE bits in the USBLEGCTLSTS register 
 when the BIOS does not acknowledge the handover. This violates the 
 spec, because the USBLEGCTLSTS register is supposed to be used by the 
 BIOS, not the OS.


Still this will only be done in case we timeout while waiting for 
the bios to acknowledge the handover.
Therefore this should be tested by people seeing the

ehci0: timed out waiting for BIOS

in their dmesg.



Christopher



Index: dev/pci/ehci_pci.c
===
RCS file: /cvs/src/sys/dev/pci/ehci_pci.c,v
retrieving revision 1.18
diff -u -p -r1.18 ehci_pci.c
--- dev/pci/ehci_pci.c  24 Jul 2009 03:18:58 -  1.18
+++ dev/pci/ehci_pci.c  8 Mar 2010 14:17:48 -
@@ -294,9 +294,26 @@ ehci_pci_takecontroller(struct ehci_pci_
break;
DELAY(1000);
}
-   if (legsup  EHCI_LEGSUP_BIOSOWNED)
-   printf(%s: timed out waiting for BIOS\n,
+   if (legsup  EHCI_LEGSUP_BIOSOWNED) {
+   /*
+* The BIOS on the Asus K8S-MV/P does not
+* release the semaphore in time and won't
+* turn of System management interrupts.
+* It will get confused by the SMIs caused
+* by port changes when the CF bit goes up.
+* Therefore we do the job of the BIOS and
+* turn off SMIs. The SMI control bits
+* are bits 0-15 at register eecp+0x04.
+* See the intel EHCI spec.
+*/
+   printf(%s: timed out waiting for BIOS
+- at least disable all SMIs.\n,
sc-sc.sc_bus.bdev.dv_xname);
+   legsup = pci_conf_read(sc-sc_pc, sc-sc_tag,
+   eecp + 0x04);
+   pci_conf_write(sc-sc_pc, sc-sc_tag,
+   eecp + 0x04, legsup  0x);
+   }
}
}
 }



My dmesg after applying the change:

OpenBSD 4.7-current (sys) #55: Mon Mar 29 18:29:56 CEST 2010
madro...@pundit:/var/obj/sys
[...]
ohci0 at pci0 dev 3 function 0 SiS 5597/5598 USB rev 0x0f: apic 
1 int 20 (irq 5), version 1.0, legacy support
ohci1 at pci0 dev 3 function 1 SiS 5597/5598 USB rev 0x0f: apic 1 int 21 (irq 
10), version 1.0, legacy support
ohci2 at pci0 dev 3 function 2 SiS 5597/5598 USB rev 0x0f: apic 1 int 22 (irq 
5), version 1.0, legacy support
ehci0 at pci0 dev 3 function 3 SiS 7002 USB rev 0x00: apic 1 int 23 (irq 10)
ehci0: timed out waiting for BIOS - at least disable all SMIs.
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 SiS EHCI root hub rev 2.00/1.00 addr 1
[...]
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 SiS OHCI root hub rev 1.00/1.00 addr 1
usb2 at ohci1: USB revision 1.0
uhub2 at usb2 SiS OHCI root hub rev 1.00/1.00 addr 1
usb3 at ohci2: USB revision 1.0
uhub3 at usb3 SiS OHCI root hub rev 1.00/1.00 addr 1
mtrr: Pentium Pro MTRR support
uhidev0 at uhub1 port 2 configuration 1 interface 0 Sun Microsystems Type 6 
Keyboard rev 1.10/2.00 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 modifier keys, 6 key codes, country code 33
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub3 port 1 configuration 1 interface 0 Logitech USB Optical 
Mouse rev 2.00/43.01 addr 2
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
[...]

Re: SiS190 driver: finished.

[Christopher Zimmermann]

On Wed, 9 Dec 2009 23:13:57 -0500 Brad wrote:
 Since the driver at the moment only supports the SiS 190 Fast 
 Ethernet
 chipset then it would be pretty strange to name the driver in such a manner
 to explicitly mention Gigabit. I think se(4) would be Ok to use. Having this 
 driver would be a good thing and I think it would be good if you could 
 continue renaming the driver and doing enough clean up to get it into
 a state to be commited to the tree.

ok. I renamed it to se and cleaned it to conform to style(9).
The only deviation I kept is that the registers names are not
all lower case.
Manpage still needs to be written.

I tested this now for quite a while on my computer I use daily, 
but I am not able to do high load tests, because I use it as 
interface for my pppoe0 connection. I don't have a second box to 
test against.

Here's a diff against current:


Index: arch/amd64/conf/GENERIC
===
RCS file: /cvs/src/sys/arch/amd64/conf/GENERIC,v
retrieving revision 1.288
diff -u -p -r1.288 GENERIC
--- arch/amd64/conf/GENERIC 28 Mar 2010 17:04:27
-   1.288 +++ arch/amd64/conf/GENERIC   2 Apr 2010
11:14:42 - @@ -436,6 +436,7 @@ vr*  at
pci?# VIA Rhine ethernet
#wb*at pci? # Winbond
W89C840F ethernet sf*   at
pci?# Adaptec AIC-6915 ethernet
sis*at pci? # SiS
900/7016 ethernet +se*  at
pci?# SiS 190/191 ethernet
#ste*   at pci? # Sundance
ST201 ethernet BORKED pcn*  at
pci?# AMD PCnet-PCI Ethernet
dc* at pci? # 21143,
tulip clone ethernet Index: dev/pci/files.pci
===
RCS file: /cvs/src/sys/dev/pci/files.pci,v retrieving revision
1.270 diff -u -p -r1.270 files.pci --- dev/pci/files.pci
23 Feb 2010 18:43:15 -  1.270 +++
dev/pci/files.pci   2 Apr 2010 11:14:43 - @@ -467,6
+467,11 @@ device   sis: ether, ifnet, mii, ifmedia
attach  sis at pci file
dev/pci/if_sis.csis +# SiS 190/191 ethernet
+device se: ether, ifnet, mii, ifmedia +attach  se
at pci +filedev/pci/if_se.c se
+
 # Sundance ST201 ethernet
 device ste: ether, ifnet, mii, ifmedia
 attach ste at pci
Index: dev/pci/if_se.c
===
RCS file: dev/pci/if_se.c
diff -N dev/pci/if_se.c
--- /dev/null   1 Jan 1970 00:00:00 -
+++ dev/pci/if_se.c 2 Apr 2010 11:14:44 -
@@ -0,0 +1,1425 @@
+/*-
+ * Copyright (c) 2009, 2010 Christopher Zimmermann
madro...@zakweb.de
+ * Copyright (c) 2007, 2008 Alexander Pohoyda
alexander.poho...@gmx.net
+ * Copyright (c) 1997, 1998, 1999
+ * Bill Paul wp...@ctr.columbia.edu.  All rights
reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
without
+ * modification, are permitted provided that the following
conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above
copyright
+ *notice, this list of conditions and the following
disclaimer.
+ * 2. Redistributions in binary form must reproduce the above
copyright
+ *notice, this list of conditions and the following
disclaimer in the
+ *documentation and/or other materials provided with the
distribution.
+ * 3. All advertising materials mentioning features or use of
this software
+ *must display the following acknowledgement:
+ * This product includes software developed by Bill Paul.
+ * 4. Neither the name of the author nor the names of any
co-contributors
+ *may be used to endorse or promote products derived from
this software
+ *without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Bill Paul AND CONTRIBUTORS ``AS
IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL
AUTHORS OR
+ * THE VOICES IN THEIR HEADS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED
+ * OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include sys/cdefs.h
+
+/*
+ * SiS 190 Fast Ethernet PCI NIC driver.
+ *
+ * Adapted to SiS 190 NIC by Alexander Pohoyda based on the
original
+ * SiS 900 driver by Bill Paul, using SiS 190/191 Solaris
driver by
+ * Masayuki Murayama and SiS 190/191 GNU/Linux driver by K.M.
Liu
+ * km...@sis.com.  Thanks to Pyun YongHyeon
pyu...@gmail.com for
+ * review and very

Re: SiS190 driver: finished.

[Christopher Zimmermann]

On Fri, 2 Apr 2010 13:34:27 +0200 Peter Hessler wrote:

 There is some nasty line-wrapping going on.  Can you resend w/o the
 wrapping?  Or possibly post the diff on a web page somewhere.

ups, how did that happen?? So here again:



Index: arch/amd64/conf/GENERIC
===
RCS file: /cvs/src/sys/arch/amd64/conf/GENERIC,v
retrieving revision 1.288
diff -u -p -r1.288 GENERIC
--- arch/amd64/conf/GENERIC 28 Mar 2010 17:04:27 -  1.288
+++ arch/amd64/conf/GENERIC 2 Apr 2010 11:14:42 -
@@ -436,6 +436,7 @@ vr* at pci? # VIA Rhine ethernet
 #wb*   at pci? # Winbond W89C840F ethernet
 sf*at pci? # Adaptec AIC-6915 ethernet
 sis*   at pci? # SiS 900/7016 ethernet
+se*at pci? # SiS 190/191 ethernet
 #ste*  at pci? # Sundance ST201 ethernet BORKED
 pcn*   at pci? # AMD PCnet-PCI Ethernet
 dc*at pci? # 21143, tulip clone ethernet
Index: dev/pci/files.pci
===
RCS file: /cvs/src/sys/dev/pci/files.pci,v
retrieving revision 1.270
diff -u -p -r1.270 files.pci
--- dev/pci/files.pci   23 Feb 2010 18:43:15 -  1.270
+++ dev/pci/files.pci   2 Apr 2010 11:14:43 -
@@ -467,6 +467,11 @@ device sis: ether, ifnet, mii, ifmedia
 attach sis at pci
 file   dev/pci/if_sis.csis
 
+# SiS 190/191 ethernet
+device se: ether, ifnet, mii, ifmedia
+attach se at pci
+file   dev/pci/if_se.c se
+
 # Sundance ST201 ethernet
 device ste: ether, ifnet, mii, ifmedia
 attach ste at pci
Index: dev/pci/if_se.c
===
RCS file: dev/pci/if_se.c
diff -N dev/pci/if_se.c
--- /dev/null   1 Jan 1970 00:00:00 -
+++ dev/pci/if_se.c 2 Apr 2010 11:14:44 -
@@ -0,0 +1,1425 @@
+/*-
+ * Copyright (c) 2009, 2010 Christopher Zimmermann madro...@zakweb.de
+ * Copyright (c) 2007, 2008 Alexander Pohoyda alexander.poho...@gmx.net
+ * Copyright (c) 1997, 1998, 1999
+ * Bill Paul wp...@ctr.columbia.edu.  All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ * 3. All advertising materials mentioning features or use of this software
+ *must display the following acknowledgement:
+ * This product includes software developed by Bill Paul.
+ * 4. Neither the name of the author nor the names of any co-contributors
+ *may be used to endorse or promote products derived from this software
+ *without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY Bill Paul AND CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ * PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL AUTHORS OR
+ * THE VOICES IN THEIR HEADS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ * OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include sys/cdefs.h
+
+/*
+ * SiS 190 Fast Ethernet PCI NIC driver.
+ *
+ * Adapted to SiS 190 NIC by Alexander Pohoyda based on the original
+ * SiS 900 driver by Bill Paul, using SiS 190/191 Solaris driver by
+ * Masayuki Murayama and SiS 190/191 GNU/Linux driver by K.M. Liu
+ * km...@sis.com.  Thanks to Pyun YongHyeon pyu...@gmail.com for
+ * review and very useful comments.
+ *
+ * Ported to OpenBSD by Christopher Zimmermann 2009/10
+ *
+ * It should be easy to adapt this driver to SiS 191 Gigabit Ethernet
+ * PCI NIC.
+ */
+
+#include sys/param.h
+#include sys/systm.h
+#include sys/sockio.h
+#include sys/mbuf.h
+#include sys/socket.h
+
+#include net/if.h
+#include net/if_dl.h
+#include net/if_media.h
+
+#include netinet/in.h
+#include netinet/if_ether.h
+
+#include net/bpf.h
+
+#include dev/mii/miivar.h
+
+#include dev/pci/pcivar.h
+#include dev/pci/pcidevs.h
+
+#include if_sereg.h
+
+/*
+ * Various supported device vendors/types and their names.
+ */
+const struct pci_matchid se_devices

testers needed for auich(4)

[Christopher Zimmermann]

Hi, 

here's a patch for auich(4). It fixes some issues with SIS 7012 
and possibly others. But since it relies on correct behaviour of 
the hardware at some places it needs some testing. Especially try 
to run something like:

aucat -l; play test.wav

right after booting and before anything else is played or 
recorded.

the typical output would be:

auich0: ac97 link rate calibration took 83337 us
sts=7dch,celv,lvbci civ=0 auich0: after calibration and reset
sts=1dch civ=0 auich0: measured ac97 link rate at 47997 Hz,
will use 48000 Hz auich_trigger_output(ed99000, eda7600, 11776,
0x80271e20, 0x8015dc00, 0x8015dd90)
sts=1dch auich_trigger_pipe: qptr=0
auich_trigger_input(eda9000, edb7600, 11776, 0x80272010,
0x8015dc00, 0x8015ddb8) sts=1dch
auich_trigger_pipe: qptr=0 auich0: halt_input auich0: halt_output

Important is the value of sts. dch means DMA halted - one 
should not set AUICH_RR before this bit is set.
In auich_calibrate() I wait for any of the three bits 
dch,celv,lvbci to be set.
If something goes wrong here you will see:
auich0: ac97 link rate timed out %d us sts=%b civ=%d\n

If something goes wrong in halt_pipe you will see:
auich_halt_pipe: halt took %d cycles


please send me your dmesg if anything goes wrong or if you chip 
does set the sts flags in another way.

A short summary of the changes:

- According to specification AUICH_RR may only be set after DMA 
  is halted (AUICH_DCH is 0 in AUICH_STS). To accomplish this I 
revived auich_halt_pipe();
- auich_calibrate() did not clear interrupt and event bits in 
  AUICH_STS. Do that now. Further it did watch the CIV index 
counter to see when all samples are processed. I changed it to 
watch AUICH_STS instead and set LVI=CIV. therefore it won't 
change the CIV counter.
- my last patch introduced a small bug in auich_trigger_pipe() I 
  fixed that.


Cheers,

Christopher Zimmermann


Index: sys/dev/pci/auich.c
===
RCS file: /cvs/src/sys/dev/pci/auich.c,v
retrieving revision 1.81
diff -u -p -r1.81 auich.c
--- sys/dev/pci/auich.c 30 Mar 2010 09:38:07 -  1.81
+++ sys/dev/pci/auich.c 31 Mar 2010 12:30:28 -
@@ -26,7 +26,7 @@
  * THE POSSIBILITY OF SUCH DAMAGE.
  */
 
-/* #define AUICH_DEBUG */
+#defineAUICH_DEBUG
 /*
  * AC'97 audio found on Intel 810/815/820/440MX chipsets.
  * http://developer.intel.com/design/chipsets/datashts/290655.htm
@@ -75,7 +75,7 @@
 #defineAUICH_BCIS  0x08/* r- buf cmplt int sts; wr ack 
*/
 #defineAUICH_LVBCI 0x04/* r- last valid bci, wr ack */
 #defineAUICH_CELV  0x02/* current equals last valid */
-#defineAUICH_DCH   0x01/* dma halted */
+#defineAUICH_DCH   0x01/* dma halted */
 #defineAUICH_ISTS_BITS 
\020\01dch\02celv\03lvbci\04bcis\05fifoe
 #defineAUICH_PICB  0x08/* 16 bits */
 #defineAUICH_PIV   0x0a/* 5 bits prefetched index 
value */
@@ -232,7 +232,8 @@ struct auich_softc {
 
 #ifdef AUICH_DEBUG
 #defineDPRINTF(l,x)do { if (auich_debug  (l)) printf x; } while(0)
-int auich_debug = 0xfffe;
+/*int auich_debug = 0xfffe;*/
+int auich_debug = 0x0002;
 #defineAUICH_DEBUG_CODECIO 0x0001
 #defineAUICH_DEBUG_DMA 0x0002
 #defineAUICH_DEBUG_INTR0x0004
@@ -1174,14 +1175,22 @@ void
 auich_halt_pipe(struct auich_softc *sc, int pipe)
 {
int i;
-   uint32_t status;
+   uint32_t sts;
 
bus_space_write_1(sc-iot, sc-aud_ioh, pipe + AUICH_CTRL, 0);
-   for (i = 0; i  100; i++) {
-   status = bus_space_read_4(sc-iot, sc-aud_ioh, pipe + 
AUICH_STS);
-   if (status  AUICH_DCH)
-   break;
-   DELAY(1);
+
+   /* wait for DMA halted and clear interrupt / event bits if needed */
+   for (i = 0; i  1000; i++) {
+   sts = bus_space_read_2(sc-iot, sc-aud_ioh, pipe + sc-sc_sts_reg);
+
+   if (sts  (AUICH_CELV | AUICH_LVBCI | AUICH_BCIS | AUICH_FIFOE))
+   bus_space_write_2(sc-iot, sc-aud_ioh, pipe + sc-sc_sts_reg,
+   AUICH_CELV | AUICH_LVBCI | AUICH_BCIS | AUICH_FIFOE);
+
+   if (sts  AUICH_DCH)
+   break;
+
+   DELAY(100);
}
bus_space_write_1(sc-iot, sc-aud_ioh, pipe + AUICH_CTRL, AUICH_RR);
 
@@ -1199,7 +1208,8 @@ auich_halt_output(v)
 
DPRINTF(AUICH_DEBUG_DMA, (%s: halt_output\n, sc-sc_dev.dv_xname));
 
-   bus_space_write_1(sc-iot, sc-aud_ioh, AUICH_PCMO + AUICH_CTRL, 
AUICH_RR);
+   auich_halt_pipe(sc, AUICH_PCMO);
+
sc-pcmo.intr = NULL;
 
return 0;
@@ -1216,8 +1226,9 @@ auich_halt_input(v)
 
/* XXX halt both unless known otherwise */
 
-   bus_space_write_1(sc-iot, sc-aud_ioh, AUICH_PCMI + AUICH_CTRL, 
AUICH_RR);
-   bus_space_write_1

[patch] bsd.port.mk(5): document no-autoheader in CONFIGURE_STYLE

[Christopher Zimmermann]

document the no-autoheader flag in CONFIGURE_STYLE


Index: bsd.port.mk.5
===
RCS file: /cvs/src/share/man/man5/bsd.port.mk.5,v
retrieving revision 1.242
diff -u -p -r1.242 bsd.port.mk.5
--- bsd.port.mk.5   22 Mar 2010 22:02:08 -  1.242
+++ bsd.port.mk.5   31 Mar 2010 11:22:19 -
@@ -922,7 +922,10 @@ Add
 if port is an older autoconf port that does not recognize
--sysconfdir. Add
 .Sq autoconf
-if autoconf needs to be rerun first.
+if autoconf needs to be rerun first,
+but set
+.Sq no-autoheader
+to prevent autoheader from running.
 Add
 .Sq automake
 if automake may need to be rerun.

Re: SiS 7002 lockup using USB2.0 'legacy' mode

[Christopher Zimmermann]

Hi J.C.,

wow, thanks for all the work and explanation!


On Sun, 7 Mar 2010 16:56:26 -0800 J.C. Roberts wrote:

 On Sun, 7 Mar 2010 22:31:18 +0100 Christopher Zimmermann
 madro...@zakweb.de wrote:
 
  ok, you misunderstood me. My BIOS needs 'legacy' to be turned on to 
  use the USB-keyboard. 
 
 Yep, you weren't clear on legacy as meaning Legacy USB Support but
 I was able to dig up a possibly similar/correct manual. According to
 Asus, K8S-MV as seen in your dmesg, doesn't exist. The closest I
 found was K8S-MX

Yes, it says K8S-MV on the board, but it looks like the K8S-MX.
The bios menu is not the same, but similar.

 http://dlsvr.asus.com/pub/ASUS/mb/sock754/k8s-mx/e1884_k8s-mx.pdf
 
 Page 56 (2-20) of the above looks kind of like what you've described.
 
 Does your BIOS have the very strange option of:
 
   Stop EHCI HC in OHCI handover [Disabled] [Enabled]
 
 as shown in the (possibly wrong) manual I found?

Yes, I have this option, but it makes no difference to what value I 
set it.

 Additionally, one of the possible reasons why your keyboard and 
 mouse
 will not work without Legacy USB Support and/or OnBoard SiS USB 1.1
 DEVICE enabled is because they are plugged into the wrong port. In
 other words you've shut off the Host Controller (SiS 5597/5598) for
 the port (s) where your mouse and keyboard are plugged in.

As I understand it, the 'legacy' option just means the bios should 
talk to USB devices (like my keyboard). Turning it off means the BIOS 
won't touch my host controler nor my keyboard.

When I turn off the 'legacy' option everything works fine. The code in 
OpenBSD which is waiting for BIOS to give up control will then not 
even be executed (see dev/pci/ehci_pci.c). The relevant register bits 
are even called EHCI_LEGSUP_BIOSOWNED and EHCI_LEGSUP_OSOWNED.
You see, LEG like in LEGacy.

   Is the kernel locked up? --Possibly No. The kernel may be 
   ignoring
   the USB mouse/keyboard that you specifically told the bios to
   ignore by turning on Legacy Mode. When the BIOS tells the kernel
   that it has no mouse or keyboard installed, then the kernel
   thinks, Ah, we're running headless, and adjusts accordingly.
  
  the kernel stops while booting. Read my orignal post again. I found 
  the very line of code causing this problem, but still don't know 
  exactly how to fix it.
 
 The (possibly wrong) manual I found has that very weird BIOS option
 for OHCI to EHCI handover. If your system has something similar, then
 this *might* be the root of your problem. If OHCI refuses to
 handover to EHCI, then it could explain why EHCI is waiting around
 for the BIOS to give up control.

As said above, this option does not change the behavior. Now I tried 
to port the way, linux protects this section where the driver tries to 
take port ownership. Still it did not fix the lockup. The patch is 
attached, if you are interested.


christopher




Index: dev/usb/ehci.c
===
RCS file: /cvs/src/sys/dev/usb/ehci.c,v
retrieving revision 1.106
diff -u -p -r1.106 ehci.c
--- dev/usb/ehci.c  26 Nov 2009 12:27:48 -  1.106
+++ dev/usb/ehci.c  8 Mar 2010 08:44:51 -
@@ -81,7 +81,7 @@ struct cfdriver ehci_cd = {
 #ifdef EHCI_DEBUG
 #define DPRINTF(x) do { if (ehcidebug) printf x; } while(0)
 #define DPRINTFN(n,x)  do { if (ehcidebug(n)) printf x; } while (0)
-int ehcidebug = 0;
+int ehcidebug = 0x19;
 #define bitmask_snprintf(q,f,b,l) snprintf((b), (l), %b, (q), (f))
 #else
 #define DPRINTF(x)
@@ -502,22 +502,49 @@ ehci_init(ehci_softc_t *sc)
EHCI_CMD_PSE |
EHCI_CMD_RS);
 
-   /* Take over port ownership */
+   /* Take over port ownership
+ *  
+ * Start, enabling full USB 2.0 functionality ... usb 1.1 devices
+ * are explicitly handed to companion controller(s), so no TT is
+ * involved with the root hub.  (Except where one is integrated,
+ * and there's no companion controller unless maybe for USB OTG.)
+ *  
+ * Turning on the CF flag will transfer ownership of all ports
+ * from the companions to the EHCI controller.  If any of the
+ * companions are in the middle of a port reset at the time, it
+ * could cause trouble.  Write-locking ehci_cf_port_reset_rwsem
+ * guarantees that no resets are in progress.  After we set CF,
+ * a short delay lets the hardware catch up; new resets shouldn't
+ * be started before the port switching actions could complete.
+* 
+* This explanation is taken from the Linux kernel 2.6.33
+* drivers/usb/host/ehci-hcd.c by David Brownell
+ */   
+   rw_enter_write(ehci_cf_port_reset_rwlock);
+   DPRINTFN(2, (ehci: after lock\n));
+   delay(50);
+   DPRINTFN(2, (ehci: after delay1\n));
EOWRITE4(sc, EHCI_CONFIGFLAG, EHCI_CONF_CF);
 
+   delay(50);
+   DPRINTFN(2, (ehci: after

Re: SiS 7002 lockup using USB2.0 'legacy' mode

[Christopher Zimmermann]

ok, I got it fixed.

Heres my theory of what went wrong:

My (broken) BIOS does not know how to hand over the ehci host
controller. Therefore the timeout.
Additionally when 'HiSpeed' is selected it turns on several System 
Management Interupts in the ehci host controller and won't turn them 
off when the kernel requests the handover.
Later, when the kernel sets the CF bit the host controller issues a 
SMI to notify the BIOS of the ownership change of mouse and keyboard 
from ohci to the ehci host controller. The BIOS does not seem to like 
this and the kernel/BIOS freezes.
My fix simply clears all SMI_ENABLE bits in the USBLEGCTLSTS register 
when the BIOS does not acknowledge the handover. This violates the 
spec, because the USBLEGCTLSTS register is supposed to be used by the 
BIOS, not the OS.
Still I'm wondering why the controller worked with Linux since I could
not find a similar piece of code in the Linux kernel.
The diff is attached. Further down you find my first attempt to fix
this by porting the writelock part from linux. I don't know whether
OpenBSD needs this.


cheers, 

Christopher



Index: dev/pci/ehci_pci.c
===
RCS file: /cvs/src/sys/dev/pci/ehci_pci.c,v
retrieving revision 1.18
diff -u -p -r1.18 ehci_pci.c
--- dev/pci/ehci_pci.c  24 Jul 2009 03:18:58 -  1.18
+++ dev/pci/ehci_pci.c  8 Mar 2010 14:17:48 -
@@ -294,9 +294,26 @@ ehci_pci_takecontroller(struct ehci_pci_
break;
DELAY(1000);
}
-   if (legsup  EHCI_LEGSUP_BIOSOWNED)
-   printf(%s: timed out waiting for BIOS\n,
+   if (legsup  EHCI_LEGSUP_BIOSOWNED) {
+   /*
+* The BIOS on the Asus K8S-MV/P does not
+* release the semaphore in time and won't
+* turn of System management interrupts.
+* It will get confused by the SMIs caused
+* by port changes when the CF bit goes up.
+* Therefore we do the job of the BIOS and
+* turn off SMIs. The SMI control bits
+* are bits 0-15 at register eecp+0x04.
+* See the intel EHCI spec.
+*/
+   printf(%s: timed out waiting for BIOS
+- at least disable all SMIs.\n,
sc-sc.sc_bus.bdev.dv_xname);
+   legsup = pci_conf_read(sc-sc_pc, sc-sc_tag,
+   eecp + 0x04);
+   pci_conf_write(sc-sc_pc, sc-sc_tag,
+   eecp + 0x04, legsup  0x);
+   }
}
}
 }




The folling diff is ported from the linux kernel.
It did not help my chipset, but didn't break anything either.

Index: dev/usb/usb.h
===
RCS file: /cvs/src/sys/dev/usb/usb.h,v
retrieving revision 1.35
diff -u -p -r1.35 usb.h
--- dev/usb/usb.h   1 Mar 2010 23:35:56 -   1.35
+++ dev/usb/usb.h   8 Mar 2010 14:11:01 -
@@ -688,3 +688,6 @@ struct usb_event {
 #define USB_SET_CM_OVER_DATA   _IOW ('U', 131, int)
 
 #endif /* _USB_H_ */
+
+/* see ehci.c, usb_subr.h and usbdi.c */
+extern struct rwlock ehci_cf_port_reset_rwlock;
Index: dev/usb/usbdi.c
===
RCS file: /cvs/src/sys/dev/usb/usbdi.c,v
retrieving revision 1.38
diff -u -p -r1.38 usbdi.c
--- dev/usb/usbdi.c 5 Mar 2010 17:28:54 -   1.38
+++ dev/usb/usbdi.c 8 Mar 2010 14:11:03 -
@@ -67,12 +67,16 @@ usbd_status usbd_open_pipe_ival(usbd_int
 int usbd_nbuses = 0;
 
 struct rwlock usbpalock;
+struct rwlock ehci_cf_port_reset_rwlock;
 
 void
 usbd_init(void)
 {
-   if (usbd_nbuses == 0)
+   if (usbd_nbuses == 0) {
rw_init(usbpalock, usbpalock);
+   rw_init(ehci_cf_port_reset_rwlock,
+   ehci_cf_port_reset_rwlock);
+   }
usbd_nbuses++;
 }
 
Index: dev/usb/ehci.c
===
RCS file: /cvs/src/sys/dev/usb/ehci.c,v
retrieving revision 1.106
diff -u -p -r1.106 ehci.c
--- dev/usb/ehci.c  26 Nov 2009 12:27:48 -  1.106
+++ dev/usb/ehci.c  8 Mar 2010 14:10:58 -
@@ -502,7 +502,25 @@ ehci_init(ehci_softc_t *sc)
EHCI_CMD_PSE |
EHCI_CMD_RS);
 
-   /* Take over port ownership */
+   /* Take over port ownership
+ *  
+ * Start, enabling full USB 2.0 

Re: Bug found in latest -current on amd64 with bigmem = 1

[Christopher Zimmermann]

On Sat, 6 Mar 2010 17:36:37 -0500
Jason Crawford ja...@purebsd.net wrote:

 I was testing out the latest snapshot on my amd64 laptop, and
 compiling latest sources, when I came across a bug. It appears that
 with the latest -current, the kernel floods dmesg with errors if
 bigmem = 1, because of this commit:
 http://marc.info/?l=openbsd-cvsm=126742779701386w=2

Got similar output caused by this commit. As I understand it this 
change is supposed to croak for certain bugs when spl-locks are not 
set appropriately.

 If I roll sys/arch/amd64/amd64/machdep.c back to 1.102, the flood goes
 away. If bigmem = 0, the dmesg flood does not happen. dmesgs for my
 laptop are at the bottom of the email. The message I get flooded with
 is:
 
 splassert: pool_do_put: want 10 have 13

for me it is

splassert: if_up: want 5 have 7

but only one time at startup when setting up pppoe.

I set splassert_ctl=2 in subr_prf.c to get some debugging info. Below 
you find the relevant parts from the dmesg.
Notice that I ported an unofficial FreeBSD driver for my SIS190 
ethernet chip. Still the trace looks like the problem is in the pppoe 
part. If you don't think so and want to have a look at the driver, I 
can post it.

Christopher


===
OpenBSD 4.7 (sys) #0: Sun Mar  7 11:39:42 CET 2010
madro...@pundit:/var/obj/sys
real mem = 1071841280 (1022MB)
avail mem = 103196 (984MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.3 @ 0xf04b0 (57 entries)
bios0: vendor American Megatrends Inc. version 0603 date 03/31/2006
bios0: ASUSTeK Computer INC. K8S-MV-P

[...]

sgb0 at pci0 dev 4 function 0 SiS 190 rev 0x00invalid EEPROM 
signature (0)
: apic 1 int 19 (irq 5), address 00:15:f2:64:0c:83
rlphy0 at sgb0 phy 1: RTL8201L 10/100 PHY, rev. 1

[...]

splassert: if_up: want 5 have 7
Starting stack trace...
if_up() at if_up+0x2a
sppp_ioctl() at sppp_ioctl+0x8f
in_ifinit() at in_ifinit+0x7c
sppp_set_ip_addrs() at sppp_set_ip_addrs+0x108
sppp_ipcp_tlu() at sppp_ipcp_tlu+0x44
sppp_cp_input() at sppp_cp_input+0x3e0
sppp_input() at sppp_input+0x38b
pppoeintr() at pppoeintr+0xd4
Xsoftnet() at Xsoftnet+0x7d
--- interrupt ---
(null)() at 0x8
end of kernel
end trace frame: 0x7, count: 247
End of stack trace.

SiS 7002 lockup using USB2.0 'legacy' mode

[Christopher Zimmermann]

Hi,

I get a complete lock-up while booting during initialisation of my 
SIS7002 EHCI controller.

When enabling USB2.0 support, but disabling 'legacy' mode everything 
is fine. I just can't use my usb keyboard to talk to the boot prompt.

When I enable legacy mode, but set USB2.0 to 'FullSpeed' I just get 
this:

ehci0: timed out waiting for BIOS

I had a look at the source, but don't really get it whether this is 
bad. I could not find corresponding code in the Linux kernel.
It seems to be about handing over the host controler to the OS after 
the bios used it for 'legacy' mode. Why is this only needed for 
USB2.0? I thought my keyboard and mouse were USB1.0 devices!?

When I enable legacy mode and set USB2.0 to 'HighSpeed' I get the 
above message and the kernel locks up completely.
I added some DPRINTF()s to the relevant sections. (See below)
The strange thing is that the last thing the kernel prints out is

ehci: before loop

so the kernel manages to do the printf, but doesn't manage to do the 
funtion call to usb_delay_ms and another printf.
Now I had a look at the linux ehci driver, because with knoppix I have 
no problems using USB2.0. You can find the corresponding section of 
the linux driver below.
The linux kernel protects itself by write-locking the call to 
ehci_writel, but I have no clue what this means and no clue how to 
achieve this in OpenBSD. The call to down_write() seems to lock a 
semaphore. Any help to fix this is appreciated.


Christopher


P.S.

Another interesting observation: When removing my AGB graphics adaptor 
and using the onboard graphics chip I have no problems with USB at 
all.

= LINUX =
drivers/usb/host/ehci-hcd.c:662

/*
 * Start, enabling full USB 2.0 functionality ... usb 1.1 devices
 * are explicitly handed to companion controller(s), so no TT is
 * involved with the root hub.  (Except where one is integrated,
 * and there's no companion controller unless maybe for USB OTG.)
 * 
 * Turning on the CF flag will transfer ownership of all ports
 * from the companions to the EHCI controller.  If any of the
 * companions are in the middle of a port reset at the time, it
 * could cause trouble.  Write-locking ehci_cf_port_reset_rwsem
 * guarantees that no resets are in progress.  After we set CF,
 * a short delay lets the hardware catch up; new resets shouldn't
 * be started before the port switching actions could complete.
 */
down_write(ehci_cf_port_reset_rwsem);
hcd-state = HC_STATE_RUNNING;
ehci_writel(ehci, FLAG_CF, ehci-regs-configured_flag);
ehci_readl(ehci, ehci-regs-command); /* unblock posted writes */
msleep(5);
up_write(ehci_cf_port_reset_rwsem);
ehci-last_periodic_enable = ktime_get_real();


= OpenBSD ===
dev/usb/ehci.c:505

/* Take over port ownership */
EOWRITE4(sc, EHCI_CONFIGFLAG, EHCI_CONF_CF);

DPRINTFN(2, (ehci: before loop\n));

/* LOCKUP HAPPENS HERE */

for (i = 0; i  100; i++) {
usb_delay_ms(sc-sc_bus, 1);
DPRINTFN(2, (ehci: after sleep %d\n, i));


= OpenBSD 
dev/usb/usb_subr.c:336

/* Delay for a certain number of ms */
void
usb_delay_ms(usbd_bus_handle bus, u_int ms)
{
/* Wait at least two clock ticks so we know the time has passed. */
if (bus-use_polling || cold) {
DPRINTF((usb_delay_ms: delaying for %u ms\n, ms));
delay((ms+1) * 1000);
}
else {
DPRINTF((usb_delay_ms: tsleeping for %u ms\n, ms));
tsleep(ms, PRIBIO, usbdly, (ms*hz+999)/1000 + 1);
}
}


==
dmesg with 'legacy' turned on, but only 'FullSpeed':


OpenBSD 4.7 (sys) #6: Sun Mar  7 16:14:48 CET 2010
madro...@pundit:/var/obj/sys
real mem = 1071841280 (1022MB)
avail mem = 1031872512 (984MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.3 @ 0xf04b0 (57 entries)
bios0: vendor American Megatrends Inc. version 0603 date 03/31/2006
bios0: ASUSTeK Computer INC. K8S-MV-P
acpi0 at bios0: rev 2
acpi0: tables DSDT FACP APIC OEMB
acpi0: wakeup devices PS2K(S4) PS2M(S4) EUSB(S4) USB_(S4) USB2(S4) USB3(S4) 
AC97(S4) MC97(S4) PCI1(S4) PCI2(S4) MAC_(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Sempron(tm) Processor 3000+, 1795.72 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 128KB 64b/line 
16-way L2 cache
cpu0: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu0: DTLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu0: AMD erratum 89 present, BIOS upgrade may be required
cpu0: apic clock running at 199MHz
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 14, 24 

Re: SiS 7002 lockup using USB2.0 'legacy' mode

[Christopher Zimmermann]

On Sun, 7 Mar 2010 12:49:00 -0800 J.C. Roberts wrote:

 On Sun, 7 Mar 2010 17:24:07 +0100 Christopher Zimmermann
 madro...@zakweb.de wrote:
 
  Hi,
  
  I get a complete lock-up while booting during initialisation of my 
  SIS7002 EHCI controller.
  
  When enabling USB2.0 support, but disabling 'legacy' mode
  everything is fine. I just can't use my usb keyboard to talk to the
  boot prompt.
  
  When I enable legacy mode, but set USB2.0 to 'FullSpeed' I just get 
  this:
  
  ehci0: timed out waiting for BIOS
  
  I had a look at the source, but don't really get it whether this is 
  bad. I could not find corresponding code in the Linux kernel.
  It seems to be about handing over the host controler to the OS
  after the bios used it for 'legacy' mode. Why is this only needed
  for USB2.0? I thought my keyboard and mouse were USB1.0 devices!?
  
  When I enable legacy mode and set USB2.0 to 'HighSpeed' I get the 
  above message and the kernel locks up completely.
  I added some DPRINTF()s to the relevant sections. (See below)
  The strange thing is that the last thing the kernel prints out is
  
  ehci: before loop
  
  so the kernel manages to do the printf, but doesn't manage to do
  the funtion call to usb_delay_ms and another printf.
  Now I had a look at the linux ehci driver, because with knoppix I
  have no problems using USB2.0. You can find the corresponding
  section of the linux driver below.
  The linux kernel protects itself by write-locking the call to 
  ehci_writel, but I have no clue what this means and no clue how to 
  achieve this in OpenBSD. The call to down_write() seems to lock a 
  semaphore. Any help to fix this is appreciated.
  
  
  Christopher
  
  
  P.S.
  
  Another interesting observation: When removing my AGB graphics
  adaptor and using the onboard graphics chip I have no problems with
  USB at all.
  
 
 The x86 BIOS sucks to begin with, and worse, the use of various terms
 is highly inconsistent... In other words, take this with a grain of
 salt, if not a whole shaker of salt, tequila and a lime.
 
 When you enable Legacy Mode in the BIOS of many systems, you're
 actually telling the system to find and use the keyboard and mouse on
 serial ports (includng PS/2 and similar ports). Of course, this means
 if you're actually using USB based keyboard/mouse and you've turned on
 Legacy Mode, you've just made a real mess. --The system believes it
 does not have a keyboard or mouse attached because it does not find
 them where you told it to find them (serial).

ok, you misunderstood me. My BIOS needs 'legacy' to be turned on to 
use the USB-keyboard. Thats actually consistent with the BSD 
terminology in dev/usb/ehcireg.h. See the EHCI_LEGSUP_* and
dev/pci/ehci_pci.c: ehci_pci_takecontroller()

 Is the kernel locked up? --Possibly No. The kernel may be ignoring the
 USB mouse/keyboard that you specifically told the bios to ignore by
 turning on Legacy Mode. When the BIOS tells the kernel that it has
 no mouse or keyboard installed, then the kernel thinks, Ah, we're
 running headless, and adjusts accordingly.

the kernel stops while booting. Read my orignal post again. I found 
the very line of code causing this problem, but still don't know 
exactly how to fix it.


Christopher

Re: SiS190 driver: How to do pci_get_device and pci_read_config_byte

[Christopher Zimmermann]

Hello again!

In porting this SIS191 driver from freeBSD I'm stuck trying to fix a
kernel panic in pool_do_get:

panic: pool_do_get(mbpl: free list modified

trace:
panic()
pool_do_get()
pool_get()
m_gethdr()

What I don't understand is in my driver m_gethdr(M_DONTWAIT, M_DATA) is
called. Therefore pool_get should only get pool_get(mbpool, 0).
So what is my driver doing wrong to produce this crash???
Could it be a hint that my driver is leaking memory?

What is also very funny is that this panic is very accurately
reproducible. When doing a scp (big packets ?!) to produce some traffic
it almost always panics exactly when handling the 80th interupt for rx
packets.

When doing ping (small packets ?!) it happens in the 256th packet,
therefore when the rx-ring is wrapping. But this may be yet another bug?

Any ideas or hints? I'm pretty lost in all this mbuf handling. The
manpage doesn't help me too much and looking at other drivers confuses
me even more, although they all look pretty similar in the rxeof()
functions.


Christopher

[demime 1.01d removed an attachment of type application/pgp-signature which had 
a name of signature.asc]

Re: SiS190 driver: How to do pci_get_device and pci_read_config_byte

[Christopher Zimmermann]

thanks for the fast replies. I believe you helped me very much to look
in the right direction.

On Sat, 24 Oct 2009 14:19:58 +0200 Claudio Jeker wrote:
 On Sat, Oct 24, 2009 at 01:56:51PM +0200, Christopher Zimmermann wrote:
  panic: pool_do_get(mbpl: free list modified

 Free list modified means that you hit a use after free or even worse
 overflowed the previous mbuf and wrote data into the the mbuf you try to
 allocate now.

 More that it is writing data into memory it should not touch.
 Make sure you map the mbuf correctly into the DMA ring and remove it
 correctly from the ring before passing it on to ether_input.

Puh, all this dma stuff gives me even more headache than all this mbuf
stuff. The manpage sounds reasonable, but when reading the code in
drivers it looks all different. Is there no openBSD kernel hacking HOWTO
or some more documentation than only the manpages?


  What is also very funny is that this panic is very accurately
  reproducible. When doing a scp (big packets ?!) to produce some traffic
  it almost always panics exactly when handling the 80th interupt for rx
  packets.
 
  When doing ping (small packets ?!) it happens in the 256th packet,
  therefore when the rx-ring is wrapping. But this may be yet another bug?

 This sounds like you map the mbuf as cluster without attachin a cluster to
 it and so the dma length is wrong. FreeBSD has a special function to
 return mbufs + clusters that is not implemented in OpenBSD. We on the
 other hand have MCLGETI which does the same and way more.
 I guess you need to check your newbuf function.

Yes, I read something about mbuf cluster is the comments. As I
understand the manpages this means the mbufs are chained together using
the m_hdr.mh_next pointer !? But as far as I understand the code in
sis_newbuf() it uses the MH_dat.m_ext. Isn't this mutually exclusive?

Actually I have no clue about those mbufs. What use is a chained cluster
to my NIC which needs a contiguous memory area, doesn't it?

 It would help to see the code in question. I bet it is just a two or three
 line change to make it work but finding can be painful (been there many
 times).


/*
 * Initialize an RX descriptor and attach an MBUF cluster.
 */
static int
sis_newbuf(sc, i, m)
struct sis_softc*sc;
u_int32_t   i;
struct mbuf *m;
{
struct sis19x_list_data *ld = sc-sis_ldata;
struct sis19x_chain_data*cd = sc-sis_cdata;
int error, alloc;

if (m == NULL) {
m = m_gethdr(M_DONTWAIT, MT_DATA);
if (m == NULL) {
SIS_OUT(unable to get new MBUF);
return (ENOBUFS);
}
cd-sis_rx_mbuf[i] = m;
alloc = 1;
} else {
if(cd-sis_rx_mbuf[i] != m)
SIS_OUT(m != sis_rx_mbuf[i]);
m-m_data = m-m_ext.ext_buf;
alloc = 0;
}

m-m_len = m-m_pkthdr.len = MCLBYTES;

if (alloc) {
/* So this is the call which turns the mbuf to a pointer
 * usable by the NIC ? */
error = bus_dmamap_load_mbuf(sc-sis_tag, cd-sis_rx_map[i],
m, BUS_DMA_NOWAIT);
if (error) {
SIS_OUT(unable to map and load the MBUF);
m_freem(m);
return (ENOBUFS);
}
}

/* This is used both to initialize the newly created RX
 * descriptor as well as for re-initializing it for reuse. */
ld-sis_rx_ring[i].sis_sts_size = 0;
ld-sis_rx_ring[i].sis_cmdsts =
htole32(OWNbit | INTbit | IPbit | TCPbit | UDPbit);
/* And here it is put into the descriptor ? */
ld-sis_rx_ring[i].sis_ptr =
htole32(cd-sis_rx_map[i]-dm_segs[0].ds_addr);
ld-sis_rx_ring[i].sis_flags =
htole32(cd-sis_rx_map[i]-dm_segs[0].ds_len);
KASSERT(cd-sis_rx_map[i]-dm_nsegs == 1,
(wrong number of segments, should be 1));

bus_dmamap_sync(sc-sis_tag, cd-sis_rx_map[i], 0,
cd-sis_rx_map[i]-dm_mapsize,
BUS_DMASYNC_PREREAD);

return (0);
}

/*
 * A frame has been uploaded: pass the resulting mbuf chain up to
 * the higher level protocols.
 */
static void
sis_rxeof(sc)
struct sis_softc*sc;
{
struct mbuf *m, *m0;
struct ifnet*ifp = sc-sis_ifp;
struct sis19x_list_data *ld = sc-sis_ldata;
struct sis19x_chain_data*cd = sc-sis_cdata;
struct sis19x_desc  *cur_rx;
u_int32_t   i, rxstat, total_len = 0;

for (i = cd-sis_rx_prod; !SIS_OWNDESC(ld-sis_rx_ring[i]);
SIS_INC(i, SIS_RX_RING_CNT)) {
bus_dmamap_sync(sc-sis_tag, cd

SiS190 driver: How to do pci_get_device and pci_read_config_byte

[Christopher Zimmermann]

Hello,

I'm trying to port the SiS190 driver from freeBSD. So far I've been 
quite successful. It compiles without warnings and finds the the MAC and 
in my case the Realtek PHY. Sadly I only have one PC with one NIC so I 
can do no real testing. My dsl modem with PPPoE does not yet work. Still 
I can see some activity on the link when I try to set it up. The driver 
is attached. It still needs some cleanup, but first I want to get
working. Please read on if you have only little knowledge about the pci
support in the openBSD kernel. I have none and probably only need a
small hint.

It probably doesn't yet work because the driver is not able to read the 
MAC address. On linux the chip works and on dmesg I read:

 sis190 Gigabit Ethernet driver 1.3 loaded.
   alloc irq_desc for 19 on node -1
   alloc kstat_irqs on node -1
 sis190 :00:04.0: PCI INT A - GSI 19 (level, low) - IRQ 19
 sis190 :00:04.0: setting latency timer to 64
 :00:04.0: Read MAC address from EEPROM
 :00:04.0: Error EEPROM read 0.
Same as above on my ported driver for openBSD.
 :00:04.0: Read MAC address from APC.
 :00:04.0: Realtek PHY RTL8201 transceiver at address 

the relevant parts from the linux driver for Read MAC address from APC 
can be found below for you convenience. Now my problem is I have very 
little BSD experience. I now have used it only for four days and have no 
clue how to port this function. And weather similar functions/macros 
exist in openBSD. Any little hint is appreciated. Thanks.


Cheers,

Christopher Zimmermann


rc = sis190_get_mac_addr_from_eeprom(pdev, dev); /* This seems to fail */
if (rc  0) {
u8 reg;

pci_read_config_byte(pdev, 0x73, reg); /* How to do this in openBSD */

if (reg  0x0001)
rc = sis190_get_mac_addr_from_apc(pdev, dev);
}

/**
 *  sis190_get_mac_addr_from_apc - Get MAC address for SiS96x model
 *  @pdev: PCI device
 *  @dev:  network device to get address for
 *
 *  SiS96x model, use APC CMOS RAM to store MAC address.
 *  APC CMOS RAM is accessed through ISA bridge.
 *  MAC address is read into @net_dev-dev_addr.
 */
static int __devinit sis190_get_mac_addr_from_apc(struct pci_dev *pdev,
  struct net_device *dev)
{
static const u16 __devinitdata ids[] = { 0x0965, 0x0966, 0x0968 };
struct sis190_private *tp = netdev_priv(dev);
struct pci_dev *isa_bridge;
u8 reg, tmp8;
unsigned int i;

net_probe(tp, KERN_INFO %s: Read MAC address from APC.\n,
  pci_name(pdev));

/*
 * Now this is the interesting part. How do I do this on openBSD?
 * Here's the linux source for pc_get_device:
 * 
 **
 * pci_get_device - begin or continue searching for a PCI device by 
vendor/device id
 * @vendor: PCI vendor id to match, or %PCI_ANY_ID to match all vendor ids
 * @device: PCI device id to match, or %PCI_ANY_ID to match all device ids
 * @from: Previous PCI device found in search, or %NULL for new search.
 *
 * Iterates through the list of known PCI devices.  If a PCI device is
 * found with a matching @vendor and @device, the reference count to the
 * device is incremented and a pointer to its device structure is returned.
 * Otherwise, %NULL is returned.  A new search is initiated by passing %NULL
 * as the @from argument.  Otherwise if @from is not %NULL, searches continue
 * from next device on the global list.  The reference count for @from is
 * always decremented if it is not %NULL.
 */
struct pci_dev *
pci_get_device(unsigned int vendor, unsigned int device, struct pci_dev *from)
{
return pci_get_subsys(vendor, device, PCI_ANY_ID, PCI_ANY_ID, from);
}
for (i = 0; i  ARRAY_SIZE(ids); i++) {
isa_bridge = pci_get_device(PCI_VENDOR_ID_SI, ids[i], 
NULL);
if (isa_bridge)
break;
}

if (!isa_bridge) {
net_probe(tp, KERN_INFO %s: Can not find ISA bridge.\n,
  pci_name(pdev));
return -EIO;
}

/* Enable port 78h  79h to access APC Registers. */
pci_read_config_byte(isa_bridge, 0x48, tmp8);
reg = (tmp8  ~0x02);
pci_write_config_byte(isa_bridge, 0x48, reg);
udelay(50);
pci_read_config_byte(isa_bridge, 0x48, reg);

for (i = 0; i  MAC_ADDR_LEN; i++) {
outb(0x9 + i, 0x78);
dev-dev_addr[i] = inb(0x79);
}

outb(0x12, 0x78);
reg = inb(0x79);

sis190_set_rgmii(tp, reg);

/* Restore the value to ISA Bridge */
pci_write_config_byte(isa_bridge, 0x48, tmp8);
pci_dev_put(isa_bridge);

return 0;
}
/*-
 * Copyright (c) 2007, 2008 Alexander Pohoyda alexander.poho...@gmx.net
 * Copyright (c) 1997, 1998, 1999
 *	Bill Paul wp...@ctr.columbia.edu.  All rights reserved.
 *
 * Redistribution and use