Re: add PCI IDs for Thinkpad X1 Extreme Gen 3, enable WLAN
A nit below.
Alexandre Ratchov writes:
> On Thu, Apr 15, 2021 at 03:24:56PM +0200, Ivo Sbalzarini wrote:
> Thanks for looking at this. Few comments:
>
> - the "PCI_VENDOR(this->subid) == PCI_VENDOR_LENOVO" compares the 16
> lower bits of the subid. It's not necessary because below we compare
> the full 32 bits.
>
> - the "name" field is the codec name; it is identified by the "vid"
> field, so it shouldn't depend on the device subid.
>
> - style: indentation is (8 char) TAB, see style(9).
>
> With above tweaks, I ended up this diff. Could you confirm it still
> makes audio work?
>
> Index: azalia.c
> ===
> RCS file: /cvs/src/sys/dev/pci/azalia.c,v
> retrieving revision 1.259
> diff -u -p -u -p -r1.259 azalia.c
> --- azalia.c 25 Oct 2020 07:22:06 - 1.259
> +++ azalia.c 19 Apr 2021 15:37:32 -
> @@ -490,7 +490,8 @@ azalia_configure_pci(azalia_t *az)
>
> const struct pci_matchid azalia_pci_devices[] = {
> { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_200SERIES_U_HDA },
> - { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_300SERIES_U_HDA }
> + { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_300SERIES_U_HDA },
> + { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_400SERIES_CAVS }
We can save the next person a comma-modifying patch next time by adding
a trailing comma before }. There was a discussion of this style some
time last summer.
Thanks
Greg
Re: Change umb(4) devclass from DV_DULL to DV_IFNET
On 4/20/21 7:28 PM, Patrick Wildt wrote:
Am Mon, Apr 19, 2021 at 10:25:39AM +0200 schrieb Tilo Stritzky:
On 10/04/21 22:56 Tilo Stritzky wrote:
umb interfaces advertise themselves as generic devices.
Network makes a lot more sense, I think.
tested on amd64.
Having seen no response on this one, I'ld like to expand a little
further.
This value defines how a device is identified in hotplug events,
eventually showing up in userland as argv to /etc/hotplug/attach.
It doesn't seem to be used by the kernel itself.
Currently umb(4) mobile broadband interfaces identify as ``generic''.
This is not exactly wrong, but there is a ``network interface'' class
which is a much tighter match. All other USB network interfaces set
it to DV_IFNET.
The change lets me group umb related stuff together with other
hotplugged network devices in /etc/hotplug/attach.
Alas, this might break some existing hotplugd setups.
tilo
That does indeed make sense. umb(4) is some kind of a network device,
and especially in hotplug I think it makes a lot more sense there as
well. So, I'm in favour.
Objections? ok?
ok gerhard@
Index: if_umb.c
===
RCS file: /cvs/src/sys/dev/usb/if_umb.c,v
retrieving revision 1.43
diff -u -p -r1.43 if_umb.c
--- if_umb.c1 Apr 2021 08:39:52 - 1.43
+++ if_umb.c10 Apr 2021 20:14:59 -
@@ -212,7 +212,7 @@ uint8_t umb_uuid_qmi_mbim[] = MBIM_UUI
uint32_t umb_session_id = 0;
struct cfdriver umb_cd = {
- NULL, "umb", DV_DULL
+ NULL, "umb", DV_IFNET
};
const struct cfattach umb_ca = {
umb0 at uhub0 port 1 configuration 1 interface 0 "MediaTek Inc Product" rev
2.00/3.00 addr 2
umsm0 at uhub0 port 1 configuration 1 interface 2 "MediaTek Inc Product" rev
2.00/3.00 addr 2
ucom0 at umsm0
umsm1 at uhub0 port 1 configuration 1 interface 3 "MediaTek Inc Product" rev
2.00/3.00 addr 2
ucom1 at umsm1
umsm2 at uhub0 port 1 configuration 1 interface 4 "MediaTek Inc Product" rev
2.00/3.00 addr 2
ucom2 at umsm2
umsm3 at uhub0 port 1 configuration 1 interface 5 "MediaTek Inc Product" rev
2.00/3.00 addr 2
ucom3 at umsm3
umass0 at uhub0 port 1 configuration 1 interface 6 "MediaTek Inc Product" rev
2.00/3.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus2 at umass0: 2 targets, initiator 0
sd1 at scsibus2 targ 1 lun 0: removable
Re: have bpf kq events fire when the interface goes away
On Wed, Apr 21, 2021 at 10:21:32AM +1000, David Gwynne wrote: > if you have a program that uses kq (or libevent) to wait for bytes to > read off an idle network interface via /dev/bpf and that interface > goes away, the program doesnt get woken up. this is because the kq > read filter in bpf only checks if there ares bytes available. because a > detached interface never gets packets (how very zen), this condition > never changes and the program will never know something happened. > > this has the bpf filter check if the interface is detached too. with > this change my test program wakes up, tries to read, and gets EIO. which > is great. > > note that in the middle of this is the vdevgone machinery. when an > interface is detached, bpfdetach gets called, which ends up calling > vdevgone. vdevgone sort of swaps out bpf on the currently open vdev with > some dead operations, part of which involves calling bpfclose() to try > and clean up the existing state associated with the vdev. bpfclose tries > to wake up any waiting listeners, which includes kq handlers. that's how > the kernel goes from an interface being detached to the bpf kq filter > being run. the bpf kq filter just has to check that the interface is > still attached. I thought tun(4) had this same problem, but I wrote a test and couldn't reproduce it. tun works because it addresses the problem in a different way. Instead of having its own kq filter check if the device is dead or not, it calls klist_invalidate, which switches things around like the vdevgone/vop_revoke stuff does with the vdev. So an alternative way to solve this problem in bpf(4) would be the following: Index: bpf.c === RCS file: /cvs/src/sys/net/bpf.c,v retrieving revision 1.203 diff -u -p -r1.203 bpf.c --- bpf.c 21 Jan 2021 12:33:14 - 1.203 +++ bpf.c 21 Apr 2021 00:54:30 - @@ -401,6 +401,7 @@ bpfclose(dev_t dev, int flag, int mode, bpf_wakeup(d); LIST_REMOVE(d, bd_list); mtx_leave(>bd_mtx); + klist_invalidate(>bd_sel.si_note); bpf_put(d); return (0);
have bpf kq events fire when the interface goes away
if you have a program that uses kq (or libevent) to wait for bytes to
read off an idle network interface via /dev/bpf and that interface
goes away, the program doesnt get woken up. this is because the kq
read filter in bpf only checks if there ares bytes available. because a
detached interface never gets packets (how very zen), this condition
never changes and the program will never know something happened.
this has the bpf filter check if the interface is detached too. with
this change my test program wakes up, tries to read, and gets EIO. which
is great.
note that in the middle of this is the vdevgone machinery. when an
interface is detached, bpfdetach gets called, which ends up calling
vdevgone. vdevgone sort of swaps out bpf on the currently open vdev with
some dead operations, part of which involves calling bpfclose() to try
and clean up the existing state associated with the vdev. bpfclose tries
to wake up any waiting listeners, which includes kq handlers. that's how
the kernel goes from an interface being detached to the bpf kq filter
being run. the bpf kq filter just has to check that the interface is
still attached.
ok?
Index: bpf.c
===
RCS file: /cvs/src/sys/net/bpf.c,v
retrieving revision 1.203
diff -u -p -r1.203 bpf.c
--- bpf.c 21 Jan 2021 12:33:14 - 1.203
+++ bpf.c 21 Apr 2021 00:03:15 -
@@ -1222,6 +1222,7 @@ int
filt_bpfread(struct knote *kn, long hint)
{
struct bpf_d *d = kn->kn_hook;
+ struct bpf_if *bp;
KERNEL_ASSERT_LOCKED();
@@ -1229,9 +1230,11 @@ filt_bpfread(struct knote *kn, long hint
kn->kn_data = d->bd_hlen;
if (d->bd_immediate)
kn->kn_data += d->bd_slen;
+
+ bp = d->bd_bif; /* check that the interface is still attached */
mtx_leave(>bd_mtx);
- return (kn->kn_data > 0);
+ return (kn->kn_data > 0 || bp == NULL);
}
/*
mg: replace-regexp
This diff adds a non-interactive version of mg's query-replace-regexp
function called replace-regexp. Unfortunately query-replace-regexp can't
be used in a startup file.
Ok?
Index: def.h
===
RCS file: /cvs/src/usr.bin/mg/def.h,v
retrieving revision 1.172
diff -u -p -u -p -r1.172 def.h
--- def.h 20 Apr 2021 10:02:50 - 1.172
+++ def.h 20 Apr 2021 16:12:50 -
@@ -673,6 +673,7 @@ int re_forwsearch(int, int);
int re_backsearch(int, int);
int re_searchagain(int, int);
int re_queryrepl(int, int);
+int re_repl(int, int);
int replstr(int, int);
int setcasefold(int, int);
int delmatchlines(int, int);
Index: funmap.c
===
RCS file: /cvs/src/usr.bin/mg/funmap.c,v
retrieving revision 1.61
diff -u -p -u -p -r1.61 funmap.c
--- funmap.c20 Apr 2021 10:02:50 - 1.61
+++ funmap.c20 Apr 2021 16:12:50 -
@@ -181,6 +181,7 @@ static struct funmap functnames[] = {
{reposition, "recenter", 0},
{redraw, "redraw-display", 0},
#ifdef REGEX
+ {re_repl, "replace-regexp", 2},
{replstr, "replace-string", 2},
#endif /* REGEX */
{revertbuffer, "revert-buffer", 0},
Index: mg.1
===
RCS file: /cvs/src/usr.bin/mg/mg.1,v
retrieving revision 1.123
diff -u -p -u -p -r1.123 mg.1
--- mg.120 Apr 2021 10:02:50 - 1.123
+++ mg.120 Apr 2021 16:12:50 -
@@ -778,6 +778,8 @@ Display current (global) working directo
.It query-replace
Query Replace.
Search and replace strings selectively, prompting after each match.
+.It replace-regexp
+Replace regular expression globally without individual prompting.
.It replace-string
Replace string globally without individual prompting.
.It query-replace-regexp
Index: re_search.c
===
RCS file: /cvs/src/usr.bin/mg/re_search.c,v
retrieving revision 1.35
diff -u -p -u -p -r1.35 re_search.c
--- re_search.c 22 Jul 2020 13:29:05 - 1.35
+++ re_search.c 20 Apr 2021 16:12:50 -
@@ -211,6 +211,34 @@ stopsearch:
return (TRUE);
}
+int
+re_repl(int f, int n)
+{
+int rcnt = 0; /* replacements made so far */
+int plen, s;/* length of found string */
+charnews[NPAT]; /* replacement string */
+
+if ((s = re_readpattern("RE Replace")) != TRUE)
+return (s);
+if (eread("Replace %s with: ", news, NPAT,
+EFNUL | EFNEW | EFCR, re_pat) == NULL)
+return (ABORT);
+
+ while (re_forwsrch() == TRUE) {
+ plen = regex_match[0].rm_eo - regex_match[0].rm_so;
+ if (re_doreplace((RSIZE)plen, news) == FALSE)
+ return (FALSE);
+ rcnt++;
+ }
+
+ curwp->w_rflag |= WFFULL;
+ update(CMODE);
+ if (!inmacro)
+ ewprintf("(%d replacement(s) done)", rcnt);
+
+ return(TRUE);
+}
+
/*
* Routine re_doreplace calls lreplace to make replacements needed by
* re_query replace. Its reason for existence is to deal with \1, \2. etc.
Re: ntpd offset loop refactoring
I got a positive test report from Paul de Weerd.
Anyone to ok it?
On Thu, Mar 18, 2021 at 01:06:49PM +0100, Alexander Bluhm wrote:
> Hi,
>
> While hunting a bug in ntpd offset handling, I found some things
> that could be improved.
>
> Call the index of the offset loops "shift" consistently.
> Merge the two offset loops in client_update() into one.
> Assign the best value instead of memcpy.
> Use the same mechanism everywhere to avoid an invalid best value.
>
> ok?
>
> bluhm
>
> Index: client.c
> ===
> RCS file: /cvs/src/usr.sbin/ntpd/client.c,v
> retrieving revision 1.115
> diff -u -p -r1.115 client.c
> --- client.c 18 Mar 2021 11:06:41 - 1.115
> +++ client.c 18 Mar 2021 11:20:49 -
> @@ -473,7 +473,7 @@ client_dispatch(struct ntp_peer *p, u_in
> int
> client_update(struct ntp_peer *p)
> {
> - int i, best = 0, good = 0;
> + int shift, best = -1, good = 0;
>
> /*
>* clock filter
> @@ -482,27 +482,22 @@ client_update(struct ntp_peer *p)
>* invalidate it and all older ones
>*/
>
> - for (i = 0; good == 0 && i < OFFSET_ARRAY_SIZE; i++)
> - if (p->reply[i].good) {
> + for (shift = 0; shift < OFFSET_ARRAY_SIZE; shift++)
> + if (p->reply[shift].good) {
> good++;
> - best = i;
> + if (best == -1 ||
> + p->reply[shift].delay < p->reply[best].delay)
> + best = shift;
> }
>
> - for (; i < OFFSET_ARRAY_SIZE; i++)
> - if (p->reply[i].good) {
> - good++;
> - if (p->reply[i].delay < p->reply[best].delay)
> - best = i;
> - }
> -
> - if (good < 8)
> + if (best == -1 || good < 8)
> return (-1);
>
> - memcpy(>update, >reply[best], sizeof(p->update));
> + p->update = p->reply[best];
> if (priv_adjtime() == 0) {
> - for (i = 0; i < OFFSET_ARRAY_SIZE; i++)
> - if (p->reply[i].rcvd <= p->reply[best].rcvd)
> - p->reply[i].good = 0;
> + for (shift = 0; shift < OFFSET_ARRAY_SIZE; shift++)
> + if (p->reply[shift].rcvd <= p->reply[best].rcvd)
> + p->reply[shift].good = 0;
> }
> return (0);
> }
> Index: control.c
> ===
> RCS file: /cvs/src/usr.sbin/ntpd/control.c,v
> retrieving revision 1.18
> diff -u -p -r1.18 control.c
> --- control.c 12 Feb 2020 19:14:56 - 1.18
> +++ control.c 18 Mar 2021 11:20:53 -
> @@ -341,7 +341,7 @@ build_show_peer(struct ctl_show_peer *cp
> const char *a = "not resolved";
> const char *pool = "", *addr_head_name = "";
> const char *auth = "";
> - u_int8_t shift, best, validdelaycnt, jittercnt;
> + int shift, best = -1, validdelaycnt = 0, jittercnt = 0;
> time_t now;
>
> now = getmonotime();
> @@ -360,14 +360,14 @@ build_show_peer(struct ctl_show_peer *cp
> snprintf(cp->peer_desc, sizeof(cp->peer_desc),
> "%s %s%s%s", a, pool, addr_head_name, auth);
>
> - validdelaycnt = best = 0;
> cp->offset = cp->delay = 0.0;
> for (shift = 0; shift < OFFSET_ARRAY_SIZE; shift++) {
> if (p->reply[shift].delay > 0.0) {
> cp->offset += p->reply[shift].offset;
> cp->delay += p->reply[shift].delay;
>
> - if (p->reply[shift].delay < p->reply[best].delay)
> + if (best == -1 ||
> + p->reply[shift].delay < p->reply[best].delay)
> best = shift;
>
> validdelaycnt++;
> @@ -379,18 +379,19 @@ build_show_peer(struct ctl_show_peer *cp
> cp->delay /= validdelaycnt;
> }
>
> - jittercnt = 0;
> cp->jitter = 0.0;
> - for (shift = 0; shift < OFFSET_ARRAY_SIZE; shift++) {
> - if (p->reply[shift].delay > 0.0 && shift != best) {
> - cp->jitter += square(p->reply[shift].delay -
> - p->reply[best].delay);
> - jittercnt++;
> + if (best != -1) {
> + for (shift = 0; shift < OFFSET_ARRAY_SIZE; shift++) {
> + if (p->reply[shift].delay > 0.0 && shift != best) {
> + cp->jitter += square(p->reply[shift].delay -
> + p->reply[best].delay);
> + jittercnt++;
> + }
> }
> + if (jittercnt > 1)
> + cp->jitter /= jittercnt;
> + cp->jitter = sqrt(cp->jitter);
> }
> - if (jittercnt > 1)
> - cp->jitter /= jittercnt;
> -
Re: Change umb(4) devclass from DV_DULL to DV_IFNET
Am Mon, Apr 19, 2021 at 10:25:39AM +0200 schrieb Tilo Stritzky:
> On 10/04/21 22:56 Tilo Stritzky wrote:
> > umb interfaces advertise themselves as generic devices.
> > Network makes a lot more sense, I think.
> > tested on amd64.
>
> Having seen no response on this one, I'ld like to expand a little
> further.
>
> This value defines how a device is identified in hotplug events,
> eventually showing up in userland as argv to /etc/hotplug/attach.
> It doesn't seem to be used by the kernel itself.
>
> Currently umb(4) mobile broadband interfaces identify as ``generic''.
> This is not exactly wrong, but there is a ``network interface'' class
> which is a much tighter match. All other USB network interfaces set
> it to DV_IFNET.
>
> The change lets me group umb related stuff together with other
> hotplugged network devices in /etc/hotplug/attach.
>
> Alas, this might break some existing hotplugd setups.
>
> tilo
That does indeed make sense. umb(4) is some kind of a network device,
and especially in hotplug I think it makes a lot more sense there as
well. So, I'm in favour.
Objections? ok?
> Index: if_umb.c
> ===
> RCS file: /cvs/src/sys/dev/usb/if_umb.c,v
> retrieving revision 1.43
> diff -u -p -r1.43 if_umb.c
> --- if_umb.c 1 Apr 2021 08:39:52 - 1.43
> +++ if_umb.c 10 Apr 2021 20:14:59 -
> @@ -212,7 +212,7 @@ uint8_tumb_uuid_qmi_mbim[] = MBIM_UUI
> uint32_t umb_session_id = 0;
>
> struct cfdriver umb_cd = {
> - NULL, "umb", DV_DULL
> + NULL, "umb", DV_IFNET
> };
>
> const struct cfattach umb_ca = {
>
>
>
>
> umb0 at uhub0 port 1 configuration 1 interface 0 "MediaTek Inc Product" rev
> 2.00/3.00 addr 2
> umsm0 at uhub0 port 1 configuration 1 interface 2 "MediaTek Inc Product" rev
> 2.00/3.00 addr 2
> ucom0 at umsm0
> umsm1 at uhub0 port 1 configuration 1 interface 3 "MediaTek Inc Product" rev
> 2.00/3.00 addr 2
> ucom1 at umsm1
> umsm2 at uhub0 port 1 configuration 1 interface 4 "MediaTek Inc Product" rev
> 2.00/3.00 addr 2
> ucom2 at umsm2
> umsm3 at uhub0 port 1 configuration 1 interface 5 "MediaTek Inc Product" rev
> 2.00/3.00 addr 2
> ucom3 at umsm3
> umass0 at uhub0 port 1 configuration 1 interface 6 "MediaTek Inc Product" rev
> 2.00/3.00 addr 2
> umass0: using SCSI over Bulk-Only
> scsibus2 at umass0: 2 targets, initiator 0
> sd1 at scsibus2 targ 1 lun 0: removable
>
Re: uvm_page_physload: use km_alloc(9)
On Thu, 15 Apr 2021 14:00:18 +0200, Martin Pieuchot wrote:
> On 13/04/21(Tue) 02:05, Alexander Bluhm wrote:
> > On Mon, Mar 22, 2021 at 11:50:00AM +0100, Mark Kettenis wrote:
> > > > Date: Mon, 22 Mar 2021 11:29:52 +0100
> > > > From: Martin Pieuchot
> > > >
> > > > Convert the last MI uvm_km_zalloc(9) to km_alloc(9), ok?
> > >
> > > Also needs some careful testing on multiple architectures.
> >
> > I did run both diffs through a full regress on armv7, arm64, amd64,
> > i386 a while ago. No fallout.
>
> I've been running those on sparc64. So I'd be interested for tests on
> powerpc{,64} and octeon. This is obviously for after release :o)
>
> Index: kern/kern_malloc.c
> ===
> RCS file: /cvs/src/sys/kern/kern_malloc.c,v
> retrieving revision 1.144
> diff -u -p -r1.144 kern_malloc.c
> --- kern/kern_malloc.c23 Feb 2021 13:50:16 - 1.144
> +++ kern/kern_malloc.c13 Apr 2021 10:25:03 -
> @@ -580,8 +580,8 @@ kmeminit(void)
> FALSE, _map_store);
> kmembase = (char *)base;
> kmemlimit = (char *)limit;
> - kmemusage = (struct kmemusage *) uvm_km_zalloc(kernel_map,
> - (vsize_t)(nkmempages * sizeof(struct kmemusage)));
> + kmemusage = km_alloc(round_page(nkmempages * sizeof(struct kmemusage)),
> + _any, _zero, _waitok);
> for (indx = 0; indx < MINBUCKET + 16; indx++) {
> XSIMPLEQ_INIT([indx].kb_freelist);
> }
> Index: uvm/uvm_page.c
> ===
> RCS file: /cvs/src/sys/uvm/uvm_page.c,v
> retrieving revision 1.156
> diff -u -p -r1.156 uvm_page.c
> --- uvm/uvm_page.c26 Mar 2021 13:40:05 - 1.156
> +++ uvm/uvm_page.c13 Apr 2021 10:25:02 -
> @@ -542,8 +542,8 @@ uvm_page_physload(paddr_t start, paddr_t
>
> npages = end - start; /* # of pages */
>
> - pgs = (struct vm_page *)uvm_km_zalloc(kernel_map,
> - npages * sizeof(*pgs));
> + pgs = km_alloc(npages * sizeof(*pgs), _any, _zero,
> + _waitok);
> if (pgs == NULL) {
> printf("uvm_page_physload: can not malloc vm_page "
> "structs for segment\n");
I've been running this over the weekend on a (lightly used) EdgeRouter Lite.
Survived building a few kernels and performing some operations on a small git
repository. Haven't noticed any "bad things" happen on that octeon box with
this.
Cheers,
Scott
Re: uvm_page_physload: use km_alloc(9)
> Date: Thu, 15 Apr 2021 14:00:18 +0200
> From: Martin Pieuchot
>
> On 13/04/21(Tue) 02:05, Alexander Bluhm wrote:
> > On Mon, Mar 22, 2021 at 11:50:00AM +0100, Mark Kettenis wrote:
> > > > Date: Mon, 22 Mar 2021 11:29:52 +0100
> > > > From: Martin Pieuchot
> > > >
> > > > Convert the last MI uvm_km_zalloc(9) to km_alloc(9), ok?
> > >
> > > Also needs some careful testing on multiple architectures.
> >
> > I did run both diffs through a full regress on armv7, arm64, amd64,
> > i386 a while ago. No fallout.
>
> I've been running those on sparc64. So I'd be interested for tests on
> powerpc{,64} and octeon. This is obviously for after release :o)
This works on powerpc as well.
ok kettenis@
> Index: kern/kern_malloc.c
> ===
> RCS file: /cvs/src/sys/kern/kern_malloc.c,v
> retrieving revision 1.144
> diff -u -p -r1.144 kern_malloc.c
> --- kern/kern_malloc.c23 Feb 2021 13:50:16 - 1.144
> +++ kern/kern_malloc.c13 Apr 2021 10:25:03 -
> @@ -580,8 +580,8 @@ kmeminit(void)
> FALSE, _map_store);
> kmembase = (char *)base;
> kmemlimit = (char *)limit;
> - kmemusage = (struct kmemusage *) uvm_km_zalloc(kernel_map,
> - (vsize_t)(nkmempages * sizeof(struct kmemusage)));
> + kmemusage = km_alloc(round_page(nkmempages * sizeof(struct kmemusage)),
> + _any, _zero, _waitok);
> for (indx = 0; indx < MINBUCKET + 16; indx++) {
> XSIMPLEQ_INIT([indx].kb_freelist);
> }
> Index: uvm/uvm_page.c
> ===
> RCS file: /cvs/src/sys/uvm/uvm_page.c,v
> retrieving revision 1.156
> diff -u -p -r1.156 uvm_page.c
> --- uvm/uvm_page.c26 Mar 2021 13:40:05 - 1.156
> +++ uvm/uvm_page.c13 Apr 2021 10:25:02 -
> @@ -542,8 +542,8 @@ uvm_page_physload(paddr_t start, paddr_t
>
> npages = end - start; /* # of pages */
>
> - pgs = (struct vm_page *)uvm_km_zalloc(kernel_map,
> - npages * sizeof(*pgs));
> + pgs = km_alloc(npages * sizeof(*pgs), _any, _zero,
> + _waitok);
> if (pgs == NULL) {
> printf("uvm_page_physload: can not malloc vm_page "
> "structs for segment\n");
>
Re: smtpd: more unused code
On Tue, 20 Apr 2021 18:38:08 +0200, Eric Faurot wrote: > On Sun, Apr 11, 2021 at 01:54:32PM +0200, Eric Faurot wrote: > > Certificate verification is done by libtls. The former code is not used > > anymore and can be unplugged. > > Anyone willing to ok this? OK millert@ - todd
Re: smtpd: more unused code
On Sun, Apr 11, 2021 at 01:54:32PM +0200, Eric Faurot wrote:
> Certificate verification is done by libtls. The former code is not used
> anymore and can be unplugged.
Anyone willing to ok this?
> Eric.
>
> Index: dispatcher.c
> ===
> RCS file: /cvs/src/usr.sbin/smtpd/dispatcher.c,v
> retrieving revision 1.2
> diff -u -p -r1.2 dispatcher.c
> --- dispatcher.c 5 Mar 2021 12:37:32 - 1.2
> +++ dispatcher.c 11 Apr 2021 11:46:17 -
> @@ -64,11 +64,6 @@ dispatcher_imsg(struct mproc *p, struct
> resolver_dispatch_result(p, imsg);
> return;
>
> - case IMSG_CERT_INIT:
> - case IMSG_CERT_VERIFY:
> - cert_dispatch_result(p, imsg);
> - return;
> -
> case IMSG_CONF_START:
> return;
> case IMSG_CONF_END:
> Index: lka.c
> ===
> RCS file: /cvs/src/usr.sbin/smtpd/lka.c,v
> retrieving revision 1.244
> diff -u -p -r1.244 lka.c
> --- lka.c 31 Dec 2020 08:27:15 - 1.244
> +++ lka.c 11 Apr 2021 11:45:24 -
> @@ -111,12 +111,6 @@ lka_imsg(struct mproc *p, struct imsg *i
> resolver_dispatch_request(p, imsg);
> return;
>
> - case IMSG_CERT_INIT:
> - case IMSG_CERT_CERTIFICATE:
> - case IMSG_CERT_VERIFY:
> - cert_dispatch_request(p, imsg);
> - return;
> -
> case IMSG_MTA_DNS_HOST:
> case IMSG_MTA_DNS_MX:
> case IMSG_MTA_DNS_MX_PREFERENCE:
> Index: smtpd.c
> ===
> RCS file: /cvs/src/usr.sbin/smtpd/smtpd.c,v
> retrieving revision 1.337
> diff -u -p -r1.337 smtpd.c
> --- smtpd.c 5 Mar 2021 12:37:32 - 1.337
> +++ smtpd.c 11 Apr 2021 11:46:38 -
> @@ -2003,10 +2003,6 @@ imsg_to_str(int type)
> CASE(IMSG_GETNAMEINFO);
> CASE(IMSG_RES_QUERY);
>
> - CASE(IMSG_CERT_INIT);
> - CASE(IMSG_CERT_CERTIFICATE);
> - CASE(IMSG_CERT_VERIFY);
> -
> CASE(IMSG_SETUP_KEY);
> CASE(IMSG_SETUP_PEER);
> CASE(IMSG_SETUP_DONE);
> Index: smtpd.h
> ===
> RCS file: /cvs/src/usr.sbin/smtpd/smtpd.h,v
> retrieving revision 1.667
> diff -u -p -r1.667 smtpd.h
> --- smtpd.h 11 Apr 2021 07:18:08 - 1.667
> +++ smtpd.h 11 Apr 2021 11:45:58 -
> @@ -102,12 +102,6 @@
> #define P_NEWALIASES 1
> #define P_MAKEMAP2
>
> -#define CERT_ERROR -1
> -#define CERT_OK 0
> -#define CERT_NOCA1
> -#define CERT_NOCERT 2
> -#define CERT_INVALID 3
> -
> struct userinfo {
> char username[SMTPD_VUSERNAME_SIZE];
> char directory[PATH_MAX];
> @@ -211,10 +205,6 @@ enum imsg_type {
> IMSG_GETNAMEINFO,
> IMSG_RES_QUERY,
>
> - IMSG_CERT_INIT,
> - IMSG_CERT_CERTIFICATE,
> - IMSG_CERT_VERIFY,
> -
> IMSG_SETUP_KEY,
> IMSG_SETUP_PEER,
> IMSG_SETUP_DONE,
> @@ -1281,14 +1271,6 @@ int ca_X509_verify(void *, void *, cons
> void ca_imsg(struct mproc *, struct imsg *);
> void ca_init(void);
> void ca_engine_init(void);
> -
> -
> -/* cert.c */
> -int cert_init(const char *, int,
> -void (*)(void *, int, const char *, const void *, size_t), void *);
> -int cert_verify(const void *, const char *, int, void (*)(void *, int), void
> *);
> -void cert_dispatch_request(struct mproc *, struct imsg *);
> -void cert_dispatch_result(struct mproc *, struct imsg *);
>
>
> /* compress_backend.c */
> Index: smtpd/Makefile
> ===
> RCS file: /cvs/src/usr.sbin/smtpd/smtpd/Makefile,v
> retrieving revision 1.112
> diff -u -p -r1.112 Makefile
> --- smtpd/Makefile11 Apr 2021 07:18:08 - 1.112
> +++ smtpd/Makefile11 Apr 2021 11:44:42 -
> @@ -7,7 +7,6 @@ PROG= smtpd
> SRCS=aliases.c
> SRCS+= bounce.c
> SRCS+= ca.c
> -SRCS+= cert.c
> SRCS+= compress_backend.c
> SRCS+= config.c
> SRCS+= control.c
>
>
rpki-client reduce log noise
Switch some warnings into logx calls to reduce noise during runtime.
Should help to locate real errors if run without -v.
--
:wq Claudio
Index: rrdp.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rrdp.c,v
retrieving revision 1.6
diff -u -p -r1.6 rrdp.c
--- rrdp.c 19 Apr 2021 17:04:35 - 1.6
+++ rrdp.c 20 Apr 2021 09:04:25 -
@@ -331,22 +331,19 @@ rrdp_finished(struct rrdp *s)
s->last_mod = NULL;
switch (s->task) {
case NOTIFICATION:
- warnx("%s: repository not modified",
- s->local);
+ logx("%s: repository not modified", s->local);
rrdp_state_send(s);
rrdp_free(s);
rrdp_done(id, 1);
break;
case SNAPSHOT:
- warnx("%s: downloading snapshot",
- s->local);
+ logx("%s: downloading snapshot", s->local);
s->sxml = new_snapshot_xml(p, >current, s);
s->state = RRDP_STATE_REQ;
break;
case DELTA:
- warnx("%s: downloading %lld deltas",
- s->local, s->repository.serial -
- s->current.serial);
+ logx("%s: downloading %lld deltas", s->local,
+ s->repository.serial - s->current.serial);
s->dxml = new_delta_xml(p, >current, s);
s->state = RRDP_STATE_REQ;
break;
@@ -372,7 +369,7 @@ rrdp_finished(struct rrdp *s)
break;
}
} else if (s->res == HTTP_NOT_MOD && s->task == NOTIFICATION) {
- warnx("%s: notification file not modified", s->local);
+ logx("%s: notification file not modified", s->local);
/* no need to update state file */
rrdp_free(s);
rrdp_done(id, 1);
Re: rpki-client http keep-alive support
On Tue, Apr 20, 2021 at 10:27:51AM +0200, Claudio Jeker wrote: > On Fri, Apr 16, 2021 at 12:21:56PM +0200, Claudio Jeker wrote: > > This diff changes the http module to support keep-alive. > > It splits requests (for a resource) from connections (to a server). > > When a request is received the code tries to first use a IDLE connection, > > if none is around a new connection is started (unless there are too many > > connections inflight). > > > > Idle connections are kept for 10sec and closed after that time. For > > rpki-client this should work well since the RRDP exchange will be a burtst > > of requests (one after another). There is only one server that is > > connected twice during the run (one server hosting 2 repos). > > > > The benefit of using keep-alive is less CPU time wasted on constant TLS > > handshakes. I did not notice any speed improvements. > > > > After fixing most issues in the http module the last few days this is less > > urgent to go in. Still sending it out so people can play with it. > > Updated diff to apply after all the KNF commits. OK job@
Re: rpki-client http keep-alive support
On Fri, Apr 16, 2021 at 12:21:56PM +0200, Claudio Jeker wrote:
> This diff changes the http module to support keep-alive.
> It splits requests (for a resource) from connections (to a server).
> When a request is received the code tries to first use a IDLE connection,
> if none is around a new connection is started (unless there are too many
> connections inflight).
>
> Idle connections are kept for 10sec and closed after that time. For
> rpki-client this should work well since the RRDP exchange will be a burtst
> of requests (one after another). There is only one server that is
> connected twice during the run (one server hosting 2 repos).
>
> The benefit of using keep-alive is less CPU time wasted on constant TLS
> handshakes. I did not notice any speed improvements.
>
> After fixing most issues in the http module the last few days this is less
> urgent to go in. Still sending it out so people can play with it.
Updated diff to apply after all the KNF commits.
--
:wq Claudio
Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.31
diff -u -p -r1.31 http.c
--- http.c 19 Apr 2021 17:04:35 - 1.31
+++ http.c 20 Apr 2021 08:06:56 -
@@ -48,6 +48,7 @@
#include
#include
+#include
#include
#include
#include
@@ -66,25 +67,31 @@
#include "extern.h"
-#define HTTP_USER_AGENT"OpenBSD rpki-client"
-#define HTTP_BUF_SIZE (32 * 1024)
-#define MAX_CONNECTIONS12
-
-#define WANT_POLLIN1
-#define WANT_POLLOUT 2
+#define HTTP_USER_AGENT"OpenBSD rpki-client"
+#define HTTP_BUF_SIZE (32 * 1024)
+#define HTTP_IDLE_TIMEOUT 10
+#define MAX_CONNECTIONS64
+#define NPFDS (MAX_CONNECTIONS + 1)
+
+enum res {
+ DONE,
+ WANT_POLLIN,
+ WANT_POLLOUT,
+};
enum http_state {
STATE_FREE,
- STATE_INIT,
STATE_CONNECT,
STATE_TLSCONNECT,
STATE_REQUEST,
STATE_RESPONSE_STATUS,
STATE_RESPONSE_HEADER,
STATE_RESPONSE_DATA,
- STATE_RESPONSE_CHUNKED,
+ STATE_RESPONSE_CHUNKED_HEADER,
+ STATE_RESPONSE_CHUNKED_TRAILER,
STATE_WRITE_DATA,
- STATE_DONE,
+ STATE_IDLE,
+ STATE_CLOSE,
};
struct http_proxy {
@@ -94,50 +101,108 @@ struct http_proxy {
};
struct http_connection {
- char*url;
+ LIST_ENTRY(http_connection) entry;
char*host;
char*port;
- const char *path; /* points into url */
- char*modified_since;
char*last_modified;
+ char*redir_uri;
+ struct http_request *req;
+ struct pollfd *pfd;
struct addrinfo *res0;
struct addrinfo *res;
struct tls *tls;
char*buf;
size_t bufsz;
size_t bufpos;
- size_t id;
off_t iosz;
+ time_t idle_time;
int status;
- int redirect_loop;
int fd;
- int outfd;
+ int chunked;
+ int keep_alive;
short events;
- short chunked;
enum http_state state;
};
-struct msgbuf msgq;
-struct sockaddr_storage http_bindaddr;
-struct tls_config *tls_config;
-uint8_t *tls_ca_mem;
-size_t tls_ca_size;
+LIST_HEAD(http_conn_list, http_connection);
+struct http_request {
+ TAILQ_ENTRY(http_request) entry;
+ char*uri;
+ char*modified_since;
+ char*host;
+ char*port;
+ const char *path; /* points into uri */
+ size_t id;
+ int outfd;
+ int redirect_loop;
+};
+
+TAILQ_HEAD(http_req_queue, http_request);
+
+static struct http_conn_list active = LIST_HEAD_INITIALIZER(active);
+static struct http_conn_list idle = LIST_HEAD_INITIALIZER(idle);
+static struct http_req_queue queue = TAILQ_HEAD_INITIALIZER(queue);
+static size_t http_conn_count;
+
+static struct msgbuf msgq;
+static struct sockaddr_storage http_bindaddr;
+static struct tls_config *tls_config;
+static uint8_t *tls_ca_mem;
+static size_t tls_ca_size;
+
+/* HTTP request API */
+static voidhttp_req_new(size_t, char *, char *, int);
+static voidhttp_req_free(struct http_request *);
+static voidhttp_req_done(size_t, enum http_result, const char *);
+static voidhttp_req_fail(size_t);
+static int http_req_schedule(struct http_request *);
+
+/* HTTP connection API */
