Re: [External] : Re: make 'set skip on ...' dynamic

2021-11-26 Thread Claudio Jeker
On Thu, Nov 25, 2021 at 02:56:02PM +0100, Alexandr Nedvedicky wrote:
> Hello,
> 
> thank you for taking a look at my diff.
> 
> 
> 
> > >   }
> > >  
> > > - if (kif->pfik_ifp != NULL || kif->pfik_group != NULL || kif == pfi_all)
> > > + if (kif->pfik_ifp != NULL || kif->pfik_group != NULL ||kif == pfi_all)
> > 
> > Missing space over^^^ here
> 
> fixed, looks like unintended change
> > > + if (found == 0) {
> > > + if (name == NULL)
> > > + return (0);
> > > +
> > > + n = strlen(name);
> > > + if ((n < 1) || (n >= IFNAMSIZ))
> > I would just use:
> > if (n < 1 || n >= IFNAMSIZ)
> > like in other places.
> 
> sure.
> 
> > 
> > > + return (0);
> > > +
> > > + if (((name[0] < 'a') || (name[0] > 'z')) ||
> > > + ((name[0] < 'A') && (name[0] > 'Z')))
> > > + return (0);
> > 
> > Not sure what you're after here. The logic of this construct is incorrect.
> > I would steal the defines from libsa/stand.h and then us !isalpha:
> > 
> > #define isupper(c)  ((c) >= 'A' && (c) <= 'Z')
> > #define islower(c)  ((c) >= 'a' && (c) <= 'z')
> > #define isalpha(c)  (isupper(c)||islower(c))
> > 
> > if (!isalpha(name[0]))
> > return (0);
> > 
> > You can also expand the defines if you want.
> > I feel the intention here is to check if this is a interface group. So
> > maybe using the check in pfi_skip_if() would be better:
> > 
> > if (name[n-1] >= '0' && name[n-1] <= '9')
> > return (0); /* group names may not end in a digit */
> > 
> 
> it is just sanity check we deal with either interface or group,
> so isalpha(name[0]) is exactly what I want. And thanks for tip
> to steal it from libsa/stand.h. Perhaps one day some day there
> will be ctype.h for for kernel as well.
> 
> the reason to add this extra check is that I want to be sure we eventually
> don't create interface, which starts with space (white char).
> 
> Remember former code did not create interface (kif object), it just walked
> through list of existing interfaces in table. Now the DIOCSETIFFLAG ioctl
> needs to be more cautious.
> 
> 
> updated diff is below.

One more thing to consider, I think the following test in pfi_set_flags():

> + if ((p->pfik_flags_new != p->pfik_flags) &&
> + (p->pfik_flagrefs == 0))
> + pfi_kif_ref(p, PFI_KIF_REF_FLAG);

should actually check for the PFI_IFLAG_SKIP flag and not any flag.

if (ISSET(p->pfik_flags_new, PFI_IFLAG_SKIP) &&
p->pfik_flagrefs == 0)
pfi_kif_ref(p, PFI_KIF_REF_FLAG);

Same goes for pfi_clear_flags() just in reverse:

> + if ((p->pfik_flags_new != p->pfik_flags) &&
> + (p->pfik_flagrefs == 1))
> + pfi_kif_unref(p, PFI_KIF_REF_FLAG);

Should be changed to:
if (!ISSET(p->pfik_flags_new, PFI_IFLAG_SKIP) &&
p->pfik_flagrefs == 1)
pfi_kif_unref(p, PFI_KIF_REF_FLAG);

We only want to track the PFI_IFLAG_SKIP flag but not any other flag like
PFI_IFLAG_ANY. At least I think we want to do that, but then I guess
pfi_set_flags() should only add a kif
if (found == 0 && ISSET(flags, PFI_IFLAG_SKIP))

I don't really like pfi_set_flags() and pfi_clear_flags()
and their ioctls DIOCSETIFFLAG and DIOCCLRIFFLAG. There are no checks for
valid flag combinations. So anything goes for these functions.
Also should the name check not happen in the ioctl handler and return
EINVAL for bad input?

 
> 8<---8<---8<--8<
> index 8de37375ab4..383b8c38f6a 100644
> --- a/sys/net/pf_if.c
> +++ b/sys/net/pf_if.c
> +void
> +pfi_group_delmember(const char *group, struct ifnet *ifp)
> +{
> + struct pfi_kif  *gkif, *ikif;
> +
> + if ((gkif = pfi_kif_get(group, NULL)) == NULL ||
> + (ikif = pfi_kif_get(ifp->if_xname, NULL)) == NULL)
> + panic("%s: pfi_kif_get failed", __func__);
> + ikif->pfik_flags_new = ikif->pfik_flags & ~gkif->pfik_flags;
> +
> + pfi_group_change(group);
> +}
> +
>  void
>  pfi_group_addmember(const char *group, struct ifnet *ifp)
>  {
> @@ -361,7 +395,7 @@ pfi_group_addmember(const char *group, struct ifnet *ifp)
>   if ((gkif = pfi_kif_get(group, NULL)) == NULL ||
>   (ikif = pfi_kif_get(ifp->if_xname, NULL)) == NULL)
>   panic("%s: pfi_kif_get failed", __func__);
> - ikif->pfik_flags |= gkif->pfik_flags;
> + ikif->pfik_flags_new = ikif->pfik_flags | gkif->pfik_flags;
>  
>   pfi_group_change(group);
>  }
> @@ -786,25 +820,64 @@ int
>  pfi_set_flags(const char *name, int flags)
>  {
>   struct pfi_kif  *p;
> + int found = 0;
> + size_t n;
>  

Re: rpki-client: make maximum number of publication points to sync operator configurable

2021-11-25 Thread Claudio Jeker
On Thu, Nov 25, 2021 at 08:18:10PM +0100, Sebastian Benoit wrote:
> Job Snijders(j...@openbsd.org) on 2021.11.25 16:13:51 +:
> > It might be advantageous to permit operators to optionally specify the
> > maximum number of publication points with which rpki-client will
> > synchronize.
> > 
> > For example: "doas rpki-client -m 1 -t /etc/rpki/ripe.tal" has as effect
> > that only RIPE NCC's repository is contacted, but none of the delegated
> > repositories.
> > 
> > This flag perhaps permits us to start shipping with a more conservative
> > default than 1000, like 100 or 200. It is clear that encouraging the
> > ecosystem to embrace 'Publish in Parent' is a saner model than everyone
> > running their own delegation.
> > 
> > Thoughts?
> 
> Yes, if we dont have these configurable, we will have users running into
> them eventually and be sad.

I'm not a big fan of this config option. This is for sure not something
that needs to change between runs and abusing it for the -m 1 case seems
strange.
Will we end up with a option salad for each and every limit? I think this
is a bad direction. rpki-client should just work and the limits need to be
chosen with that in mind.
 
> maxpubpoints sounds a bit funny, but i have no other idea.
> 
> code reads ok.

The way the limit is implemented in this patch is not correct.
 
More inline

> > Index: main.c
> > ===
> > RCS file: /cvs/src/usr.sbin/rpki-client/main.c,v
> > retrieving revision 1.166
> > diff -u -p -r1.166 main.c
> > --- main.c  25 Nov 2021 14:03:40 -  1.166
> > +++ main.c  25 Nov 2021 16:08:10 -
> > @@ -69,6 +69,7 @@ int   verbose;
> >  intnoop;
> >  intrrdpon = 1;
> >  intrepo_timeout = 15*60;
> > +unsigned int   maxpubpoints = 1000;
> >  
> >  struct statsstats;
> >  
> > @@ -714,7 +715,7 @@ main(int argc, char *argv[])
> > "proc exec unveil", NULL) == -1)
> > err(1, "pledge");
> >  
> > -   while ((c = getopt(argc, argv, "b:Bcd:e:f:jnorRs:t:T:vV")) != -1)
> > +   while ((c = getopt(argc, argv, "b:Bcd:e:f:jm:norRs:t:T:vV")) != -1)
> > switch (c) {
> > case 'b':
> > bind_addr = optarg;
> > @@ -738,6 +739,11 @@ main(int argc, char *argv[])
> > case 'j':
> > outformats |= FORMAT_JSON;
> > break;
> > +   case 'm':
> > +   maxpubpoints = strtonum(optarg, 0, 10, );
> > +   if (errs)
> > +   errx(1, "-m: %s", errs);
> > +   break;

Why allow 0 as minimum? Will that even work?

> > case 'n':
> > noop = 1;
> > break;
> > @@ -1220,7 +1226,7 @@ usage:
> > fprintf(stderr,
> > "usage: rpki-client [-BcjnoRrVv] [-b sourceaddr] [-d cachedir]"
> > " [-e rsync_prog]\n"
> > -   "   [-s timeout] [-T table] [-t tal]"
> > -   " [outputdir]\n");
> > +   "   [-m maxpubpoints] [-s timeout] [-T table] "
> > +   "[-t tal] [outputdir]\n");
> > return 1;
> >  }
> > Index: repo.c
> > ===
> > RCS file: /cvs/src/usr.sbin/rpki-client/repo.c,v
> > retrieving revision 1.14
> > diff -u -p -r1.14 repo.c
> > --- repo.c  25 Nov 2021 14:03:40 -  1.14
> > +++ repo.c  25 Nov 2021 16:08:17 -
> > @@ -41,6 +41,7 @@ extern struct stats   stats;
> >  extern int noop;
> >  extern int rrdpon;
> >  extern int repo_timeout;
> > +extern unsigned intmaxpubpoints;
> >  
> >  enum repo_state {
> > REPO_LOADING = 0,
> > @@ -1100,12 +1101,14 @@ ta_lookup(int id, struct tal *tal)
> > if ((rp->repouri = strdup(tal->descr)) == NULL)
> > err(1, NULL);
> >  
> > -   if (++talrepocnt[id] >= MAX_REPO_PER_TAL) {
> > -   if (talrepocnt[id] == MAX_REPO_PER_TAL)
> > -   warnx("too many repositories under %s", tals[id]);
> > +   if (talrepocnt[id] >= maxpubpoints + 1) {
> > +   if (talrepocnt[id] == maxpubpoints)

This can not be right. If the first if is true the 2nd can never be true.

> > +   warnx("too many publication points under %s", tals[id]);
> > nofetch = 1;
> > }
> >  
> > +   talrepocnt[id]++;
> > +
> > rp->ta = ta_get(tal, nofetch);
> >  
> > return rp;
> > @@ -1146,11 +1149,12 @@ repo_lookup(int id, const char *uri, con
> > if ((rp->notifyuri = strdup(notify)) == NULL)
> > err(1, NULL);
> >  
> > -   if (++talrepocnt[id] >= MAX_REPO_PER_TAL) {
> > -   if (talrepocnt[id] == MAX_REPO_PER_TAL)
> > -   warnx("too many repositories under %s", tals[id]);
> > +   if (talrepocnt[id] >= maxpubpoints + 1) {
> > +   if (talrepocnt[id] == maxpubpoints)

Same here.

> > +   warnx("too many publication points 

Re: rpki-client: set repo_timeout to be 1/4th of timeout

2021-11-25 Thread Claudio Jeker
On Thu, Nov 25, 2021 at 12:54:49PM +, Job Snijders wrote:
> Replace MAX_REPO_TIMEOUT with repo_timeout, which is set to 1/4th of
> timeout, or if timeout is disabled set it to 24 hours.
> 
> OK?

OK claudio@
 
> Index: extern.h
> ===
> RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
> retrieving revision 1.97
> diff -u -p -r1.97 extern.h
> --- extern.h  24 Nov 2021 15:24:16 -  1.97
> +++ extern.h  25 Nov 2021 12:52:05 -
> @@ -627,7 +627,4 @@ int   mkpath(const char *);
>  /* Maximum allowd repositories per tal */
>  #define MAX_REPO_PER_TAL 1000
>  
> -/* Timeout for repository synchronisation, in seconds */
> -#define MAX_REPO_TIMEOUT (15 * 60)
> -
>  #endif /* ! EXTERN_H */
> Index: main.c
> ===
> RCS file: /cvs/src/usr.sbin/rpki-client/main.c,v
> retrieving revision 1.165
> diff -u -p -r1.165 main.c
> --- main.c19 Nov 2021 09:47:30 -  1.165
> +++ main.c25 Nov 2021 12:52:05 -
> @@ -68,6 +68,7 @@ const char  *bird_tablename = "ROAS";
>  int  verbose;
>  int  noop;
>  int  rrdpon = 1;
> +int  repo_timeout = 15*60;
>  
>  struct stats  stats;
>  
> @@ -753,6 +754,12 @@ main(int argc, char *argv[])
>   timeout = strtonum(optarg, 0, 24*60*60, );
>   if (errs)
>   errx(1, "-s: %s", errs);
> + if (timeout == 0)
> + repo_timeout = 24*60*60;
> + else if (timeout < 300)
> + errx(1, "-s: %i too small", timeout);
> + else
> + repo_timeout = timeout / 4;
>   break;
>   case 't':
>   if (talsz >= TALSZ_MAX)
> Index: repo.c
> ===
> RCS file: /cvs/src/usr.sbin/rpki-client/repo.c,v
> retrieving revision 1.12
> diff -u -p -r1.12 repo.c
> --- repo.c15 Nov 2021 16:32:15 -  1.12
> +++ repo.c25 Nov 2021 12:52:05 -
> @@ -40,6 +40,7 @@
>  extern struct stats  stats;
>  extern int   noop;
>  extern int   rrdpon;
> +extern int   repo_timeout;
>  
>  enum repo_state {
>   REPO_LOADING = 0,
> @@ -620,7 +621,7 @@ repo_alloc(int talid)
>  
>   rp->id = ++repoid;
>   rp->talid = talid;
> - rp->alarm = getmonotime() + MAX_REPO_TIMEOUT;
> + rp->alarm = getmonotime() + repo_timeout;
>   TAILQ_INIT(>queue);
>   SLIST_INSERT_HEAD(, rp, entry);
>  
> @@ -1225,7 +1228,7 @@ static void
>  repo_fail(struct repo *rp)
>  {
>   /* reset the alarm since code may fallback to rsync */
> - rp->alarm = getmonotime() + MAX_REPO_TIMEOUT;
> + rp->alarm = getmonotime() + repo_timeout;
>  
>   if (rp->ta)
>   http_finish(rp->ta->id, HTTP_FAILED, NULL);
> Index: rpki-client.8
> ===
> RCS file: /cvs/src/usr.sbin/rpki-client/rpki-client.8,v
> retrieving revision 1.49
> diff -u -p -r1.49 rpki-client.8
> --- rpki-client.8 26 Oct 2021 13:26:53 -  1.49
> +++ rpki-client.8 25 Nov 2021 12:52:05 -
> @@ -131,6 +131,8 @@ seconds of runtime, because normal pract
>  .Xr cron 8 .
>  Disable by specifying 0.
>  Defaults to 1 hour.
> +Individual Publication Points are timed out after one fourth of
> +.Em timeout .
>  .It Fl T Ar table
>  For BIRD output generated with the
>  .Fl B
> 

-- 
:wq Claudio



Re: make 'set skip on ...' dynamic

2021-11-25 Thread Claudio Jeker
On Fri, Nov 19, 2021 at 12:59:38AM +0100, Alexandr Nedvedicky wrote:
> Hello,
> 
> it has turned out things are bit more complicated when it comes to interface
> groups. diff below makes following scenario work for me.
> 
> we start with etc/pf.conf as follows:
> 
>   # cat /etc/pf.conf
>   set skip on lo
>   set skip on test1
>   set skip on test2
>   set skip on agroup
>   block return
> 
> Let's see what interfaces are known to pf(4)
> 
>   # pfctl -sI
>   agroup
>   all
>   carp
>   egress
>   enc
>   enc0
>   lo
>   lo0
>   pflog
>   pflog0
>   test1
>   test2
>   vio0
> 
> Using the same rules on current we get different output:
> 
>   # pfctl -sI
>   all
>   carp
>   egress
>   enc
>   enc0
>   lo
>   lo0
>   pflog
>   pflog0
>   vio0
> 
> as you can see test1, test2 and agroup are missing on current.
> it's because the current does not create a pfi_kif entry for
> interfaces/groups referred by 'set skip on' keyword in pf.conf.
> 
> the rules found in /etc/pf.conf block all network traffic:
> 
>   # ping -c 1 192.168.2.1 
>   PING 192.168.2.1 (192.168.2.1): 56 data bytes
>   ping: sendmsg: Permission denied
>   ping: wrote 192.168.2.1 64 chars, ret=-1
> 
>   --- 192.168.2.1 ping statistics ---
>   1 packets transmitted, 0 packets received, 100.0% packet loss
> 
> to demonstrate 'set skip on agroup' works I'll add vio0 interface into
> agroup and repeat test:
> 
>   # ifconfig vio0 group agroup
>   # ping -c 1 192.168.2.1 
>   PING 192.168.2.1 (192.168.2.1): 56 data bytes
>   64 bytes from 192.168.2.1: icmp_seq=0 ttl=254 time=1.058 ms
> 
>   --- 192.168.2.1 ping statistics ---
>   1 packets transmitted, 1 packets received, 0.0% packet loss
>   round-trip min/avg/max/std-dev = 1.058/1.058/1.058/0.000 ms
> 
> removing vio0 from agroup stops network again:
> 
>   # ifconfig vio0 -group agroup   
>   # ping -c 1 192.168.2.1   
>   PING 192.168.2.1 (192.168.2.1): 56 data bytes
>   ping: sendmsg: Permission denied
>   ping: wrote 192.168.2.1 64 chars, ret=-1
> 
>   --- 192.168.2.1 ping statistics ---
>   1 packets transmitted, 0 packets received, 100.0% packet loss
> 
> removing 'set skip on agroup' from /etc/pf.conf also removes
> the matching pfi_kif object from interface table kept by pf(4):
> 
>   # cat /etc/pf.conf
>   set skip on lo
>   set skip on test1
>   set skip on test2
>   block return
>   # pfctl -f /etc/pf.conf
>   # pfctl -sI
>   all
>   carp
>   egress
>   enc
>   enc0
>   lo
>   lo0
>   pflog
>   pflog0
>   test1
>   test2
>   vio0
> 
> 
> The main trick to get things to work is to let 'set skip on ...' create 
> entries
> (pfi_kif objects)) in pf's table for interfaces and groups which are not known
> to IP stack yet. The same thing happens when firewall rule refers interface,
> which does not exist in system yet. Diff below modifies pfi_set_flags()
> function to create pfi_kif object if it does not already. We also obtain
> a reference PFI_KIF_REF_FLAG if and only if the flag is being changed.
> 
> We treat a pfik_reflag reference counter as an indication whether the pfi_kif
> object get created on behalf of 'set skip on ...' keyword, so we can call
> pfi_kif_unref() in pfi_clear_flags() to destroy it. The pfik_reflag also
> prevents other callers to pfi_kif_unref() from destroying pfi_kif, which
> got created by pfi_set_flags().
> 
> Changes described so far are sufficient to get 'set skip on...' working for
> regular interfaces. To make it working for interface group we need to 
> introduce
> pfi_group_delmember() and slightly adjust pfi_group_addmember().
> Both functions update interface flag in the same fashion like pfi_set_flags()
> and pfi_clear_flags(). The caller of 
> pfi_group_delmember()/pfi_group_addmember()
> must call pfi_xcommit() to update interfaces.
> 
> OK?

Some inline comments.


> 8<---8<---8<--8<
> diff --git a/share/man/man5/pf.conf.5 b/share/man/man5/pf.conf.5
> index bff448aa8dc..4afe841651f 100644
> --- a/share/man/man5/pf.conf.5
> +++ b/share/man/man5/pf.conf.5
> @@ -1383,9 +1383,6 @@ Packets passing in or out on such interfaces are passed 
> as if pf was
>  disabled, i.e. pf does not process them in any way.
>  This can be useful on loopback and other virtual interfaces, when
>  packet filtering is not desired and can have unexpected effects.
> -.Ar ifspec
> -is only evaluated when the ruleset is loaded; interfaces created
> -later will not be skipped.
>  PF filters traffic on all interfaces by default.
>  .It Ic set Cm state-defaults Ar state-option , ...
>  The
> diff --git a/sys/net/if.c b/sys/net/if.c
> index 

rpki-client rrdp regress test

2021-11-25 Thread Claudio Jeker
This add an RRDP regress test that checks basic operation.
It checks some valid notification, snapshot and delta XML.
There are also two XML attacks included (billion laughs and XXE).
More bad XML files should be added.

Comments?
-- 
:wq Claudio

Index: Makefile.inc
===
RCS file: /cvs/src/regress/usr.sbin/rpki-client/Makefile.inc,v
retrieving revision 1.15
diff -u -p -r1.15 Makefile.inc
--- Makefile.inc24 Oct 2021 17:54:28 -  1.15
+++ Makefile.inc24 Nov 2021 14:12:39 -
@@ -8,6 +8,7 @@ PROGS += test-gbr
 PROGS += test-mft
 PROGS += test-roa
 PROGS += test-tal
+PROGS += test-rrdp
 
 .for p in ${PROGS}
 REGRESS_TARGETS += run-regress-$p
@@ -50,3 +51,35 @@ SRCS_test-tal+=  test-tal.c tal.c ip.c io
encoding.c print.c dummy.c
 run-regress-test-tal: test-tal
./test-tal -v ${.CURDIR}/../tal/*.tal
+
+SRCS_test-rrdp+=   test-rrdp.c rrdp_delta.c rrdp_notification.c \
+   rrdp_snapshot.c rrdp_util.c \
+   log.c encoding.c ip.c validate.c dummy.c
+LDADD_test-rrdp+=  -lexpat ${LDADD}
+DPADD_test-rrdp+=  ${LIBEXPAT} ${DPADD}
+run-regress-test-rrdp: test-rrdp
+   ./test-rrdp \
+   -n < ${.CURDIR}/../rrdp/notification.xml 2>&1 | tee rrdp-r1.out
+   cmp ${.CURDIR}/../rrdp/rrdp-r1.out rrdp-r1.out
+
+   ./test-rrdp -S 8fe05c2e-047d-49e7-8398-cd4250a572b1 -N 50500 \
+   -n < ${.CURDIR}/../rrdp/notification.xml 2>&1 | tee rrdp-r2.out
+   cmp ${.CURDIR}/../rrdp/rrdp-r2.out rrdp-r2.out
+   
+   ./test-rrdp -S 9b3f7e31-4979-ef8c-d818-73e4dadc3e6b -N 13755 \
+   -H 27571e365a4c87b51a03731415ce2118cc268d686db3209ae752f01ba2d74bc5 \
+   -d < ${.CURDIR}/../rrdp/delta.xml 2>&1 | tee rrdp-r3.out
+   cmp ${.CURDIR}/../rrdp/rrdp-r3.out rrdp-r3.out
+
+   ./test-rrdp -S 7e7d2563-5bbb-40b0-8723-6a2e90c85d9e -N 28483 \
+   -H 2a051bfd199150fe6bcdc777d09e70fe1acdf239fbf98ba378a793685e5adb21 \
+   -s < ${.CURDIR}/../rrdp/snapshot.xml 2>&1 | tee rrdp-r4.out
+   cmp ${.CURDIR}/../rrdp/rrdp-r4.out rrdp-r4.out
+
+   ./test-rrdp \
+   -n < ${.CURDIR}/../rrdp/xxe.xml 2>&1 | tee rrdp-r5.out
+   cmp ${.CURDIR}/../rrdp/rrdp-r5.out rrdp-r5.out
+
+   ./test-rrdp \
+   -n < ${.CURDIR}/../rrdp/billion_lol.xml 2>&1 | tee rrdp-r6.out
+   cmp ${.CURDIR}/../rrdp/rrdp-r6.out rrdp-r6.out
Index: test-rrdp.c
===
RCS file: test-rrdp.c
diff -N test-rrdp.c
--- /dev/null   1 Jan 1970 00:00:00 -
+++ test-rrdp.c 24 Nov 2021 17:34:37 -
@@ -0,0 +1,338 @@
+/* $OpenBSD: rrdp.c,v 1.17 2021/10/29 09:27:36 claudio Exp $ */
+/*
+ * Copyright (c) 2020 Nils Fisher 
+ * Copyright (c) 2021 Claudio Jeker 
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+
+#include "extern.h"
+#include "rrdp.h"
+
+#define REGRESS_NOTIFY_URI "https://rpki.example.com/notify.xml;
+
+#define MAX_SESSIONS   12
+#defineREAD_BUF_SIZE   (32 * 1024)
+
+#define RRDP_STATE_REQ 0x01
+#define RRDP_STATE_WAIT0x02
+#define RRDP_STATE_PARSE   0x04
+#define RRDP_STATE_PARSE_ERROR 0x08
+#define RRDP_STATE_PARSE_DONE  0x10
+#define RRDP_STATE_HTTP_DONE   0x20
+#define RRDP_STATE_DONE(RRDP_STATE_PARSE_DONE | 
RRDP_STATE_HTTP_DONE)
+
+struct rrdp {
+   TAILQ_ENTRY(rrdp)entry;
+   size_t   id;
+   char*notifyuri;
+   char*local;
+   char*last_mod;
+
+   struct pollfd   *pfd;
+   int  infd;
+   int  state;
+   unsigned int file_pending;
+   unsigned int file_failed;
+   enum http_result res;
+   enum rrdp_task   task;
+
+   char hash[SHA256_DIGEST_LENGTH];
+   SHA256_CTX   ctx;
+
+   struct rrdp_session  repository;
+   struct rrdp_sessi

rpki-client code shuffle

2021-11-24 Thread Claudio Jeker
4 Nov 2021 12:52:56 -
@@ -35,14 +35,22 @@ enum rrdp_task {
DELTA,
 };
 
-/* rrdp generic */
-char   *xstrdup(const char *);
-int hex_decode(const char *, char *, size_t);
-
-/* publish or withdraw element */
 struct rrdp;
-struct publish_xml;
 
+struct publish_xml {
+   char*uri;
+   char*data;
+   char hash[SHA256_DIGEST_LENGTH];
+   size_t   data_length;
+   enum publish_typetype;
+};
+
+/* rrdp generic */
+char   *xstrdup(const char *);
+voidrrdp_publish_file(struct rrdp *, struct publish_xml *,
+   unsigned char *, size_t);
+
+/* rrdp util */
 struct publish_xml *new_publish_xml(enum publish_type, char *,
char *, size_t);
 voidfree_publish_xml(struct publish_xml *);
Index: rrdp_notification.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rrdp_notification.c,v
retrieving revision 1.11
diff -u -p -r1.11 rrdp_notification.c
--- rrdp_notification.c 9 Nov 2021 11:01:04 -   1.11
+++ rrdp_notification.c 24 Nov 2021 13:26:02 -
@@ -464,6 +464,19 @@ notification_delta_done(struct notificat
 void
 log_notification_xml(struct notification_xml *nxml)
 {
+   struct delta_item *d;
+   char *hash;
+
logx("session_id: %s, serial: %lld", nxml->session_id, nxml->serial);
logx("snapshot_uri: %s", nxml->snapshot_uri);
+   hash = hex_encode(nxml->snapshot_hash, sizeof(nxml->snapshot_hash));
+   logx("snapshot hash: %s", hash);
+   free(hash);
+
+   TAILQ_FOREACH(d, >delta_q, q) {
+   logx("delta serial %lld uri: %s", d->serial, d->uri);
+   hash = hex_encode(d->hash, sizeof(d->hash));
+   logx("delta hash: %s", hash);
+   free(hash);
+   }
 }
Index: rrdp_util.c
=======
RCS file: rrdp_util.c
diff -N rrdp_util.c
--- /dev/null   1 Jan 1970 00:00:00 -
+++ rrdp_util.c 24 Nov 2021 12:50:03 -
@@ -0,0 +1,120 @@
+/* $OpenBSD$ */
+/*
+ * Copyright (c) 2020 Nils Fisher 
+ * Copyright (c) 2021 Claudio Jeker 
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+
+#include "extern.h"
+#include "rrdp.h"
+
+/*
+ * Both snapshots and deltas use publish_xml to store the publish and
+ * withdraw records. Once all the content is added the request is sent
+ * to the main process where it is processed.
+ */
+struct publish_xml *
+new_publish_xml(enum publish_type type, char *uri, char *hash, size_t hlen)
+{
+   struct publish_xml *pxml;
+
+   if ((pxml = calloc(1, sizeof(*pxml))) == NULL)
+   err(1, "%s", __func__);
+
+   pxml->type = type;
+   pxml->uri = uri;
+   if (hlen > 0) {
+   assert(hlen == sizeof(pxml->hash));
+   memcpy(pxml->hash, hash, hlen);
+   }
+
+   return pxml;
+}
+
+void
+free_publish_xml(struct publish_xml *pxml)
+{
+   if (pxml == NULL)
+   return;
+
+   free(pxml->uri);
+   free(pxml->data);
+   free(pxml);
+}
+
+/*
+ * Add buf to the base64 data string, ensure that this remains a proper
+ * string by NUL-terminating the string.
+ */
+int
+publish_add_content(struct publish_xml *pxml, const char *buf, int length)
+{
+   size_t newlen, outlen;
+
+   /*
+* optmisiation, this often gets called with '\n' as the
+* only data... seems wasteful
+*/
+   if (length == 1 && buf[0] == '\n')
+   return 0;
+
+   /* append content to data */
+   if (SIZE_MAX - length - 1 <= pxml->data_length)
+   return -1;
+   newlen = pxml->data_length + length;
+   if (base64_decode_len(newlen, ) == -1 ||
+   outlen > MAX_FILE_SIZE)
+   return -1;
+
+   pxml->data = realloc(pxml->data, newlen + 1);
+   if (pxml->data == NULL)
+   err(1, "%s", __func__);
+
+   memcpy(p

Re: acme-client: another trivial accessor conversion

2021-11-22 Thread Claudio Jeker
On Mon, Nov 22, 2021 at 12:18:37AM +0100, Theo Buehler wrote:
> bio->num_write aka BIO_number_written(bio). Straightforward. The main
> reason I'm asking is that keeping the two else results in overlong lines
> and awkward line wrapping. So I decided to drop them hoping that's
> acceptable. Otherwise please tell me the preferred way to wrap the
> lines in this part of the tree.
> 
> Index: revokeproc.c
> ===
> RCS file: /cvs/src/usr.sbin/acme-client/revokeproc.c,v
> retrieving revision 1.18
> diff -u -p -r1.18 revokeproc.c
> --- revokeproc.c  13 Oct 2021 18:09:42 -  1.18
> +++ revokeproc.c  21 Nov 2021 23:07:37 -
> @@ -186,15 +186,17 @@ revokeproc(int fd, const char *certfile,
>   if (bio == NULL) {
>   warnx("BIO_new");
>   goto out;
> - } else if (!X509V3_EXT_print(bio, ex, 0, 0)) {
> + }
> + if (!X509V3_EXT_print(bio, ex, 0, 0)) {
>   warnx("X509V3_EXT_print");
>   goto out;
> - } else if ((san = calloc(1, bio->num_write + 1)) == NULL) {
> + }
> + if ((san = calloc(1, BIO_number_written(bio) + 1)) == NULL) {
>   warn("calloc");
>   goto out;
>   }
> - ssz = BIO_read(bio, san, bio->num_write);
> - if (ssz < 0 || (unsigned)ssz != bio->num_write) {
> + ssz = BIO_read(bio, san, BIO_number_written(bio));
> + if (ssz < 0 || (unsigned)ssz != BIO_number_written(bio)) {
>   warnx("BIO_read");
>   goto out;
>   }
> 

OK. These else if chaining is not OpenBSD style and makes the code hard to
read. I like to see less of them.

-- 
:wq Claudio



Re: IPsec tdb ddb print

2021-11-18 Thread Claudio Jeker
On Thu, Nov 18, 2021 at 08:50:37AM +, Stuart Henderson wrote:
> On 2021/11/18 09:15, Claudio Jeker wrote:
> > On Thu, Nov 18, 2021 at 08:10:26AM +, Stuart Henderson wrote:
> > > On 2021/11/15 17:23, Alexander Bluhm wrote:
> > > > +   DUMP(ids, "%p");
> > > > +   DUMP(ids_swapped, "%d");
> > > > +   DUMP(mtu, "%d");
> > > > +   DUMP(mtutimeout, "%lld");
> > > > +   pr("%18s: %08x\n", "udpencap_port",
> > > > +   ntohl(tdb->tdb_udpencap_port));
> > > 
> > > ntohl is incorrect here:
> > > 
> > >  udpencap_port: 1194
> > > 
> > > Index: netinet/ip_ipsp.c
> > > ===
> > > RCS file: /cvs/src/sys/netinet/ip_ipsp.c,v
> > > retrieving revision 1.250
> > > diff -u -p -r1.250 ip_ipsp.c
> > > --- netinet/ip_ipsp.c 16 Nov 2021 13:53:14 -  1.250
> > > +++ netinet/ip_ipsp.c 18 Nov 2021 08:08:47 -
> > > @@ -591,8 +591,7 @@ tdb_printit(void *addr, int full, int (*
> > >   DUMP(ids_swapped, "%d");
> > >   DUMP(mtu, "%d");
> > >   DUMP(mtutimeout, "%lld");
> > > - pr("%18s: %08x\n", "udpencap_port",
> > > - ntohl(tdb->tdb_udpencap_port));
> > > + pr("%18s: %08x\n", "udpencap_port", tdb->tdb_udpencap_port);
> > >   DUMP(tag, "%d");
> > >   DUMP(tap, "%d");
> > >   DUMP(rdomain, "%d");
> > > 
> > 
> > I think this needs to be ntohs(tdb->tdb_udpencap_port) since it is still
> > stored in network byte order.
> 
> With ntohs, it's "udpencap_port: "

I would also use %d as format string. This is just a simple portnumber.

The code using tdb_udpencap_port for sure assumes network byte order:
uh->uh_sport = uh->uh_dport = htons(udpencap_port);
if (tdb->tdb_udpencap_port)
uh->uh_dport = tdb->tdb_udpencap_port;

-- 
:wq Claudio



Re: IPsec tdb ddb print

2021-11-18 Thread Claudio Jeker
On Thu, Nov 18, 2021 at 08:10:26AM +, Stuart Henderson wrote:
> On 2021/11/15 17:23, Alexander Bluhm wrote:
> > +   DUMP(ids, "%p");
> > +   DUMP(ids_swapped, "%d");
> > +   DUMP(mtu, "%d");
> > +   DUMP(mtutimeout, "%lld");
> > +   pr("%18s: %08x\n", "udpencap_port",
> > +   ntohl(tdb->tdb_udpencap_port));
> 
> ntohl is incorrect here:
> 
>  udpencap_port: 1194
> 
> Index: netinet/ip_ipsp.c
> ===
> RCS file: /cvs/src/sys/netinet/ip_ipsp.c,v
> retrieving revision 1.250
> diff -u -p -r1.250 ip_ipsp.c
> --- netinet/ip_ipsp.c 16 Nov 2021 13:53:14 -  1.250
> +++ netinet/ip_ipsp.c 18 Nov 2021 08:08:47 -
> @@ -591,8 +591,7 @@ tdb_printit(void *addr, int full, int (*
>   DUMP(ids_swapped, "%d");
>   DUMP(mtu, "%d");
>   DUMP(mtutimeout, "%lld");
> - pr("%18s: %08x\n", "udpencap_port",
> - ntohl(tdb->tdb_udpencap_port));
> + pr("%18s: %08x\n", "udpencap_port", tdb->tdb_udpencap_port);
>   DUMP(tag, "%d");
>   DUMP(tap, "%d");
>   DUMP(rdomain, "%d");
> 

I think this needs to be ntohs(tdb->tdb_udpencap_port) since it is still
stored in network byte order.

-- 
:wq Claudio



bt.5 document count()

2021-11-16 Thread Claudio Jeker
This documents count(). This function only works when used like this
@map[key] = count();
But it is implemented and works. If used differently you get a syntax
error which is not helpful. This is why I chose to document it like this.
Another option would be to document the language (so it is clear where it
is possible to use what). 

max(), min() and sum() are other functions that behave like this. Their
documentation should also be adjusted IMO.

-- 
:wq Claudio

Index: bt.5
===
RCS file: /cvs/src/usr.sbin/btrace/bt.5,v
retrieving revision 1.13
diff -u -p -r1.13 bt.5
--- bt.512 Nov 2021 16:57:24 -  1.13
+++ bt.516 Nov 2021 09:50:52 -
@@ -120,6 +120,11 @@ Functions:
 .It Fn clear "@map"
 Delete all (key, value) pairs from
 .Va @map .
+.It "@map[key]" = Fn count
+Increment the value of
+.Va key
+from
+.Va @map .
 .It Fn delete "@map[key]"
 Delete the pair indexed by
 .Va key



Re: vport: set UP on ip assign

2021-11-15 Thread Claudio Jeker
On Mon, Nov 15, 2021 at 12:23:02PM +, Klemens Nanni wrote:
> On Mon, Nov 15, 2021 at 12:00:18PM +1000, David Gwynne wrote:
> > On Sat, Nov 13, 2021 at 11:59:59PM +, Klemens Nanni wrote:
> > > Practically all interfaces pull itself up when IPs get assigned, but
> > > vport(4) does not.
> > 
> > Yes, I do (or don't do) this very deliberately when I get the chance.
> > 
> > > This broke IPv4 networking for me on a router I switched from bridge(4)
> > > to veb(4) because hostname.vport0 only contained the equivalent of
> > > 
> > >   descr LAN
> > >   inet 192.0.2.1
> > >   inet6 2001:db8::1
> > > 
> > > e.g. the explicit "up" was missing.
> > > 
> > > dhcpd(8) only considers UP interfaces to listen on during start;
> > > being the only interface it could potentially listen on, dhcpd thus
> > > ignored vport0 and failed to start.
> > > 
> > > `ifconfig vport0 up && rcctl restart dhcpd' fixed this.
> > > Adding "up" to thostname.vport0 also fixed it.
> > > 
> > > Nonetheless, vport should UP itself as the rest does.
> > 
> > My counter argument is that no interface should implicitly bring itself
> > up when an address is configured because it has been a road block to
> > figuring out how to lock the ioctl paths better, and it confuses error
> > handling. If address config works, but the interface fails to come up,
> > the address remains configured but we report an error. But it looks like
> > configuring an address failed? Wat.
> 
> That's a valid point, although I must say I never ran into this issue.
> 
> I fear that users have come to rely on the implicit up semantics, i.e.
> I doubt I'm the only one with hostname.* files lacking "up".

Sure people got used to this mode. I agree with dlg@ that the way that
this auto interface up happens is rather horrible from a network stack
view. Also it makes it impossible to configure an interface all the way
before bringing it up. This may be important for interfaces with more
complex configuration.

The current behaviour is great for casual users (that does not really
care about network) but is annoying for network admins.
 
> > I've suggested previously that netstart should handle bringing an
> > interface up. look for "netstart: implicit up and explicit down for
> > hostname.if conf files" on tech@. I didn't hanve the energy to push
> > it forward though.
> 
> I'll do the digging and try to catch up, thanks.

Another option is to adjust ifconfig but then again one would like to do
the up last after running multiple ifconfig calls. ifconfig has its own
madness when it comes to execute multiple commands in one go.
 
> > dhcpd should cope with an interface being down too. It should be about
> > whetherthe addresses are right more than if the interface is up or not.

I expect an interface that is down to remain down until I bring it up
again. Same goes the other way. ifconfig up/down should only be used to
implement admin shutdown. We fixed a few interfaces in the past that
played with IFF_UP in the driver.

-- 
:wq Claudio



support probe as variable in btrace

2021-11-12 Thread Claudio Jeker
This is something I missed to do easy btrace check like:
syscall:exit:entry,
syscall:fork:entry,
syscall:sigaction:entry,
syscall:execve:entry,
syscall:open:entry { @[probe] = count(); }

This will produce something like this as output:
@[syscall:open:entry]: 844
@[syscall:sigaction:entry]: 480
@[syscall:execve:entry]: 27
@[syscall:exit:entry]: 26
@[syscall:fork:entry]: 24

Works for me. I decided to qsort the dt_dtpis array so that bsearch can be
used for quick lookups by id. If the kernel ensures that the returned
array from the ioctl is always sorted this could be skipped.

-- 
:wq Claudio


Index: bt.5
===
RCS file: /cvs/src/usr.sbin/btrace/bt.5,v
retrieving revision 1.12
diff -u -p -r1.12 bt.5
--- bt.53 Oct 2021 22:01:48 -   1.12
+++ bt.512 Nov 2021 14:05:16 -
@@ -105,6 +105,8 @@ Kernel stack of the current thread.
 Timestamp of the event in nanoseconds.
 .It Va pid
 Process ID of the current thread.
+.It Va probe
+Full name of the probe.
 .It Va retval
 Return value of the traced syscall.
 .It Va tid
Index: bt_parse.y
===
RCS file: /cvs/src/usr.sbin/btrace/bt_parse.y,v
retrieving revision 1.44
diff -u -p -r1.44 bt_parse.y
--- bt_parse.y  3 Oct 2021 22:01:48 -   1.44
+++ bt_parse.y  12 Nov 2021 13:46:22 -
@@ -717,6 +717,7 @@ lookup(char *s)
{ "pid",BUILTIN,B_AT_BI_PID },
{ "print",  F_PRINT,B_AC_PRINT },
{ "printf", FUNCN,  B_AC_PRINTF },
+   { "probe",  BUILTIN,B_AT_BI_PROBE },
{ "retval", BUILTIN,B_AT_BI_RETVAL },
{ "str",STR,B_AT_FN_STR },
{ "sum",MOP1,   B_AT_MF_SUM },
Index: bt_parser.h
===
RCS file: /cvs/src/usr.sbin/btrace/bt_parser.h,v
retrieving revision 1.21
diff -u -p -r1.21 bt_parser.h
--- bt_parser.h 3 Oct 2021 22:01:48 -   1.21
+++ bt_parser.h 12 Nov 2021 13:12:43 -
@@ -147,6 +147,7 @@ struct bt_arg {
B_AT_BI_ARG9,
B_AT_BI_ARGS,
B_AT_BI_RETVAL,
+   B_AT_BI_PROBE,
 
B_AT_FN_STR,/* str($1); str($1, 3); */
 
Index: btrace.c
===
RCS file: /cvs/src/usr.sbin/btrace/btrace.c,v
retrieving revision 1.59
diff -u -p -r1.59 btrace.c
--- btrace.c24 Oct 2021 14:18:58 -  1.59
+++ btrace.c12 Nov 2021 13:55:52 -
@@ -256,6 +256,18 @@ read_btfile(const char *filename, size_t
return fcontent;
 }
 
+static int
+dtpi_cmp(const void *a, const void *b)
+{
+   const struct dtioc_probe_info *ai = a, *bi = b;
+
+   if (ai->dtpi_pbn > bi->dtpi_pbn)
+   return 1;
+   if (ai->dtpi_pbn < bi->dtpi_pbn)
+   return -1;
+   return 0;
+}
+
 void
 dtpi_cache(int fd)
 {
@@ -276,6 +288,8 @@ dtpi_cache(int fd)
dtpr.dtpr_probes = dt_dtpis;
if (ioctl(fd, DTIOCGPLIST, ))
err(1, "DTIOCGPLIST");
+
+   qsort(dt_dtpis, dt_ndtpi, sizeof(*dt_dtpis), dtpi_cmp);
 }
 
 void
@@ -351,6 +365,15 @@ dtpi_get_by_value(const char *prov, cons
return NULL;
 }
 
+static struct dtioc_probe_info *
+dtpi_get_by_id(unsigned int pbn)
+{
+   struct dtioc_probe_info d;
+
+   d.dtpi_pbn = pbn;
+   return bsearch(, dt_dtpis, dt_ndtpi, sizeof(*dt_dtpis), dtpi_cmp);
+}
+
 void
 rules_do(int fd)
 {
@@ -1263,6 +1286,8 @@ ba_name(struct bt_arg *ba)
return "args";
case B_AT_BI_RETVAL:
return "retval";
+   case B_AT_BI_PROBE:
+   return "probe";
case B_AT_FN_STR:
return "str";
case B_AT_OP_PLUS:
@@ -1372,6 +1397,9 @@ ba2long(struct bt_arg *ba, struct dt_evt
case B_AT_BI_RETVAL:
val = dtev->dtev_retval[0];
break;
+   case B_AT_BI_PROBE:
+   val = dtev->dtev_pbn;
+   break;
case B_AT_OP_PLUS ... B_AT_OP_LOR:
val = baexpr2long(ba, dtev);
break;
@@ -1390,6 +1418,7 @@ ba2str(struct bt_arg *ba, struct dt_evt 
 {
static char buf[STRLEN];
struct bt_var *bv;
+   struct dtioc_probe_info *dtpi;
const char *str;
 
buf[0] = '\0';
@@ -1436,6 +1465,15 @@ ba2str(struct bt_arg *ba, struct dt_evt 
snprintf(buf, sizeof(buf), "%ld", (long)dtev->dtev_retval[0]);
str = buf;
break;
+   case B_AT_BI_PROBE:
+   dtpi = dtpi_get_by_id(dtev->dtev_pbn);
+   if (dtpi != NULL)
+   snprintf(buf, sizeof(buf), "%s:%s:%s",
+   dtpi->dtpi_prov, dtpi_func(dtpi), dtpi->dtpi_name);
+   else
+   

Re: sigsuspend(2): sleep on channel?

2021-11-11 Thread Claudio Jeker
On Thu, Nov 11, 2021 at 02:13:26PM -0600, Scott Cheloha wrote:
> On Thu, Nov 11, 2021 at 08:53:20PM +0100, Mark Kettenis wrote:
> > > Date: Thu, 11 Nov 2021 13:30:04 -0600
> > > From: Scott Cheloha 
> > > 
> > > My understanding of sigsuspend(2) is that it only returns if a signal
> > > is delivered to the calling thread.  However, in sys_sigsuspend() we
> > > pass >p_p->ps_sigacts as the wakeup channel to tsleep_nsec(9).
> > > 
> > > Are we actually waiting for a wakeup on that channel?  Or can we sleep
> > > on  here?  Patch attached.  Note that we don't need to loop
> > > here anymore: we can't receieve wakeup so tsleep_nsec(9) will never
> > > return zero.
> > 
> > I don't think we expect a wakeup on that channel.  But the loop is
> > still needed as a spurious wakeup may happen.
> 
> I'm not quite sure how you'd get a spurious wakeup in this context,
> but I'll take your word for it.
> 
> So use  to indicate we're not expecting a wakeup, but keep the
> loop.
> 
> Index: kern_sig.c
> ===
> RCS file: /cvs/src/sys/kern/kern_sig.c,v
> retrieving revision 1.287
> diff -u -p -r1.287 kern_sig.c
> --- kern_sig.c24 Oct 2021 00:02:25 -  1.287
> +++ kern_sig.c11 Nov 2021 20:10:51 -
> @@ -509,7 +509,7 @@ dosigsuspend(struct proc *p, sigset_t ne
>  }
>  
>  /*
> - * Suspend process until signal, providing mask to be set
> + * Suspend thread until signal, providing mask to be set
>   * in the meantime.  Note nonstandard calling convention:
>   * libc stub passes mask, not pointer, to save a copyin.
>   */
> @@ -519,12 +519,10 @@ sys_sigsuspend(struct proc *p, void *v, 
>   struct sys_sigsuspend_args /* {
>   syscallarg(int) mask;
>   } */ *uap = v;
> - struct process *pr = p->p_p;
> - struct sigacts *ps = pr->ps_sigacts;
>  
>   dosigsuspend(p, SCARG(uap, mask) &~ sigcantmask);
> - while (tsleep_nsec(ps, PPAUSE|PCATCH, "sigsusp", INFSLP) == 0)
> - /* void */;
> + while (tsleep_nsec(, PPAUSE|PCATCH, "sigsusp", INFSLP) == 0)
> + continue;
>   /* always return EINTR rather than ERESTART... */
>   return (EINTR);
>  }
> 

OK claudio@
-- 
:wq Claudio



Re: sppp(4)/pppoe(4) - DNS configuration via resolvd(8)

2021-11-10 Thread Claudio Jeker
On Wed, Nov 10, 2021 at 07:35:26AM +0100, Bjorn Ketelaars wrote:
> On Mon 08/11/2021 11:52, Bjorn Ketelaars wrote:
> > Diff below does two things:
> > 1. add PPP IPCP extensions for name server addresses (rfc1877) to
> >sppp(4)
> > 2. propose negotiated name servers from sppp(4) to resolvd(8) using
> >RTM_PROPOSAL_STATIC route messages.
> 
> 
> Updated diff below, based on feedback from kn@ and claudio@:
> 
> - fix forgotten parentheses with `sizeof`
> - instead of using `u_int32_t` use `struct in_addr` for holding dns
>   addresses. Makes it more clear what the data is
> - decouple `IPCP_OPT` definitions from the bitmask values to
>   enable/disable an option. Makes the code look a bit better
> - use `memcpy`
> - fit code within 80 columns
> 
> While here add RFC to sppp(4)'s STANDARDS section.
> 
> @kn, is this still OK for you?
> 
> Other OK's?
> 
> 
> diff --git share/man/man4/sppp.4 share/man/man4/sppp.4
> index 5ca10285953..bccb41eec15 100644
> --- share/man/man4/sppp.4
> +++ share/man/man4/sppp.4
> @@ -230,6 +230,13 @@ take place.
>  .Re
>  .Pp
>  .Rs
> +.%A S. Cobb
> +.%D December 1995
> +.%R RFC 1877
> +.%T PPP Internet Protocol Control Protocol Extensions for Name Server 
> Addresses
> +.Re
> +.Pp
> +.Rs
>  .%A W. Simpson
>  .%D August 1996
>  .%R RFC 1994
> diff --git sys/net/if_sppp.h sys/net/if_sppp.h
> index ff559fcc369..5850a6da963 100644
> --- sys/net/if_sppp.h
> +++ sys/net/if_sppp.h
> @@ -132,6 +132,8 @@ struct sipcp {
> * original one here, in network byte order */
>   u_int32_t req_hisaddr;  /* remote address requested (IPv4) */
>   u_int32_t req_myaddr;   /* local address requested (IPv4) */
> +#define IPCP_MAX_DNSSRV  2
> + struct in_addr dns[IPCP_MAX_DNSSRV]; /* IPv4  DNS servers (RFC 1877) */
>  #ifdef INET6
>   struct in6_aliasreq req_ifid;   /* local ifid requested (IPv6) */
>  #endif
> diff --git sys/net/if_spppsubr.c sys/net/if_spppsubr.c
> index ac1dc9a709d..e460703c089 100644
> --- sys/net/if_spppsubr.c
> +++ sys/net/if_spppsubr.c
> @@ -132,6 +132,14 @@
>  #define IPCP_OPT_ADDRESSES   1   /* both IP addresses; deprecated */
>  #define IPCP_OPT_COMPRESSION 2   /* IP compression protocol (VJ) */
>  #define IPCP_OPT_ADDRESS 3   /* local IP address */
> +#define IPCP_OPT_PRIMDNS 129 /* primary remote dns address */
> +#define IPCP_OPT_SECDNS  131 /* secondary remote dns address 
> */
> +
> +#define SPPP_IPCP_OPT_ADDRESSES  1   /* bitmask value */
> +#define SPPP_IPCP_OPT_COMPRESSION2   /* bitmask value */
> +#define SPPP_IPCP_OPT_ADDRESS3   /* bitmask value */
> +#define SPPP_IPCP_OPT_PRIMDNS4   /* bitmask value */
> +#define SPPP_IPCP_OPT_SECDNS 5   /* bitmask value */

Instead of repeating the /* bitmask value */ just add a comment at the
top. Something like "bitmask value to enable or disable individual IPCP
options"

>  
>  #define IPV6CP_OPT_IFID  1   /* interface identifier */
>  #define IPV6CP_OPT_COMPRESSION   2   /* IPv6 compression protocol */
> @@ -338,6 +346,8 @@ void sppp_update_gw(struct ifnet *ifp);
>  void sppp_set_ip_addrs(void *);
>  void sppp_clear_ip_addrs(void *);
>  void sppp_set_phase(struct sppp *sp);
> +void sppp_update_dns(struct ifnet *ifp);
> +void sppp_rtrequest(struct ifnet *ifp, int req, struct rtentry *rt);
>  
>  /* our control protocol descriptors */
>  static const struct cp lcp = {
> @@ -701,6 +711,7 @@ sppp_attach(struct ifnet *ifp)
>  
>   sp->pp_if.if_type = IFT_PPP;
>   sp->pp_if.if_output = sppp_output;
> + sp->pp_if.if_rtrequest = sppp_rtrequest;
>   ifq_set_maxlen(>pp_if.if_snd, 50);
>   mq_init(>pp_cpq, 50, IPL_NET);
>   sp->pp_loopcnt = 0;
> @@ -2512,13 +2523,19 @@ sppp_ipcp_RCN_rej(struct sppp *sp, struct lcp_header 
> *h, int len)
>* Peer doesn't grok address option.  This is
>* bad.  XXX  Should we better give up here?
>*/
> - sp->ipcp.opts &= ~(1 << IPCP_OPT_ADDRESS);
> + sp->ipcp.opts &= ~(1 << SPPP_IPCP_OPT_ADDRESS);
>   break;
>  #ifdef notyet
>   case IPCP_OPT_COMPRESS:
> - sp->ipcp.opts &= ~(1 << IPCP_OPT_COMPRESS);
> + sp->ipcp.opts &= ~(1 << SPPP_IPCP_OPT_COMPRESS);
>   break;
>  #endif
> + case IPCP_OPT_PRIMDNS:
> + sp->ipcp.opts &= ~(1 << SPPP_IPCP_OPT_PRIMDNS);
> + break;
> + case IPCP_OPT_SECDNS:
> + sp->ipcp.opts &= ~(1 << SPPP_IPCP_OPT_SECDNS);
> + break;
>   }
>   }
>   if (debug)
> @@ -2559,7 +2576,7 @@ sppp_ipcp_RCN_nak(struct sppp *sp, struct lcp_header 
> *h, int len)
>   if (len >= 6 && p[1] == 6) {
>   wantaddr = p[2] << 24 

Re: sppp(4)/pppoe(4) - DNS configuration via resolvd(8)

2021-11-10 Thread Claudio Jeker
On Wed, Nov 10, 2021 at 08:22:52AM +0100, Sebastien Marie wrote:
> On Wed, Nov 10, 2021 at 07:35:26AM +0100, Bjorn Ketelaars wrote:
> > On Mon 08/11/2021 11:52, Bjorn Ketelaars wrote:
> > > Diff below does two things:
> > > 1. add PPP IPCP extensions for name server addresses (rfc1877) to
> > >sppp(4)
> > > 2. propose negotiated name servers from sppp(4) to resolvd(8) using
> > >RTM_PROPOSAL_STATIC route messages.
> > 
> > 
> > Updated diff below, based on feedback from kn@ and claudio@:
> > 
> > - fix forgotten parentheses with `sizeof`
> > - instead of using `u_int32_t` use `struct in_addr` for holding dns
> >   addresses. Makes it more clear what the data is
> > - decouple `IPCP_OPT` definitions from the bitmask values to
> >   enable/disable an option. Makes the code look a bit better
> > - use `memcpy`
> > - fit code within 80 columns
> > 
> > While here add RFC to sppp(4)'s STANDARDS section.
> > 
> > @kn, is this still OK for you?
> > 
> > Other OK's?
> 
> There is one point which bother me a bit: you are using
> RTP_PROPOSAL_STATIC for sending the proposal, whereas all others
> sources (dhcpleased/dhclient, slaacd, umb) are using a specific value.
> 
> By using RTP_PROPOSAL_STATIC, it means also that route(8) nameserver
> subcommand might interfere with it.
> 
> Using a new specific value (like RTP_PROPOSAL_SPPP) would make sense
> to me. But no objection if RTM_PROPOSAL_STATIC is preferred.

Agreed, This should use its own prio. I would suggest to call it
RTP_PROPOSAL_PPP. This is something that can be done independently.

-- 
:wq Claudio



Re: rpki-client ip_addr_print cleanup

2021-11-09 Thread Claudio Jeker
On Tue, Nov 09, 2021 at 07:44:41PM +0100, Claudio Jeker wrote:
> ip_addr_print() can be simplified. ip4_addr2str() and ip6_addr2str() are
> the same apart from the different AF argument to inet_ntop(). Just collaps
> all into ip_addr_print().

This version is using a switch statement and fails hard for unknown AFIs.
Suggested by Theo.

-- 
:wq Claudio

Index: ip.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/ip.c,v
retrieving revision 1.19
diff -u -p -r1.19 ip.c
--- ip.c5 Nov 2021 10:50:41 -   1.19
+++ ip.c9 Nov 2021 18:59:34 -
@@ -234,41 +234,6 @@ ip_addr_parse(const ASN1_BIT_STRING *p,
 }
 
 /*
- * Convert the IPv4 address into CIDR notation conforming to RFC 4632.
- * Buffer should be able to hold xxx.yyy.zzz.www/nn.
- */
-static void
-ip4_addr2str(const struct ip_addr *addr, char *b, size_t bsz)
-{
-   char buf[16];
-   int ret;
-
-   if (inet_ntop(AF_INET, addr->addr, buf, sizeof(buf)) == NULL)
-   err(1, "inet_ntop");
-   ret = snprintf(b, bsz, "%s/%hhu", buf, addr->prefixlen);
-   if (ret < 0 || (size_t)ret >= bsz)
-   err(1, "malformed IPV4 address");
-}
-
-/*
- * Convert the IPv6 address into CIDR notation conforming to RFC 4291.
- * See also RFC 5952.
- * Must hold :::::::/nn.
- */
-static void
-ip6_addr2str(const struct ip_addr *addr, char *b, size_t bsz)
-{
-   char buf[44];
-   int ret;
-
-   if (inet_ntop(AF_INET6, addr->addr, buf, sizeof(buf)) == NULL)
-   err(1, "inet_ntop");
-   ret = snprintf(b, bsz, "%s/%hhu", buf, addr->prefixlen);
-   if (ret < 0 || (size_t)ret >= bsz)
-   err(1, "malformed IPV6 address");
-}
-
-/*
  * Convert a ip_addr into a NUL-terminated CIDR notation string
  * conforming to RFC 4632 or 4291.
  * The size of the buffer must be at least 64 (inclusive).
@@ -277,11 +242,25 @@ void
 ip_addr_print(const struct ip_addr *addr,
 enum afi afi, char *buf, size_t bufsz)
 {
+   char ipbuf[44];
+   int ret, af;
+
+   switch (afi) {
+   case AFI_IPV4:
+   af = AF_INET;
+   break;
+   case AFI_IPV6:
+   af = AF_INET6;
+   break;
+   default:
+   errx(1, "unsupported address family identifier");
+   }
 
-   if (afi == AFI_IPV4)
-   ip4_addr2str(addr, buf, bufsz);
-   else
-   ip6_addr2str(addr, buf, bufsz);
+   if (inet_ntop(af, addr->addr, ipbuf, sizeof(ipbuf)) == NULL)
+   err(1, "inet_ntop");
+   ret = snprintf(buf, bufsz, "%s/%hhu", ipbuf, addr->prefixlen);
+   if (ret < 0 || (size_t)ret >= bufsz)
+   err(1, "malformed IP address");
 }
 
 /*



rpki-client sync http escape handling with ftp(1)

2021-11-09 Thread Claudio Jeker
kn@ removed '~' from unsafe_chars but also changed the code at the same
time. This tries to bring the version in rpki-client back in sync with the
code in ftp(1).

-- 
:wq Claudio

Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.49
diff -u -p -r1.49 http.c
--- http.c  9 Nov 2021 11:00:43 -   1.49
+++ http.c  9 Nov 2021 18:49:27 -
@@ -211,14 +211,17 @@ http_info(const char *uri)
 }
 
 /*
- * Determine whether the character needs encoding, per RFC1738:
- * - No corresponding graphic US-ASCII.
- * - Unsafe characters.
+ * Determine whether the character needs encoding, per RFC2396.
  */
 static int
-unsafe_char(const char *c0)
+to_encode(const char *c0)
 {
-   const char *unsafe_chars = " <>\"#{}|\\^~[]`";
+   /* 2.4.3. Excluded US-ASCII Characters */
+   const char *excluded_chars =
+   " " /* space */
+   "<>#\"" /* delims (modulo "%", see below) */
+   "{}|\\^[]`" /* unwise */
+   ;
const unsigned char *c = (const unsigned char *)c0;
 
/*
@@ -228,16 +231,15 @@ unsafe_char(const char *c0)
return (iscntrl(*c) || !isascii(*c) ||
 
/*
-* Unsafe characters.
-* '%' is also unsafe, if is not followed by two
+* '%' is also reserved, if is not followed by two
 * hexadecimal digits.
 */
-   strchr(unsafe_chars, *c) != NULL ||
+   strchr(excluded_chars, *c) != NULL ||
(*c == '%' && (!isxdigit(c[1]) || !isxdigit(c[2];
 }
 
 /*
- * Encode given URL, per RFC1738.
+ * Encode given URL, per RFC2396.
  * Allocate and return string to the caller.
  */
 static char *
@@ -254,7 +256,7 @@ url_encode(const char *path)
 * final URL.
 */
for (i = 0; i < length; i++)
-   if (unsafe_char(path + i))
+   if (to_encode(path + i))
new_length += 2;
 
epath = epathp = malloc(new_length + 1);/* One more for '\0'. */
@@ -266,7 +268,7 @@ url_encode(const char *path)
 * Encode, and copy final URL.
 */
for (i = 0; i < length; i++)
-   if (unsafe_char(path + i)) {
+   if (to_encode(path + i)) {
snprintf(epathp, 4, "%%" "%02x",
(unsigned char)path[i]);
epathp += 3;



rpki-client ip_addr_print cleanup

2021-11-09 Thread Claudio Jeker
ip_addr_print() can be simplified. ip4_addr2str() and ip6_addr2str() are
the same apart from the different AF argument to inet_ntop(). Just collaps
all into ip_addr_print().

-- 
:wq Claudio

Index: ip.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/ip.c,v
retrieving revision 1.19
diff -u -p -r1.19 ip.c
--- ip.c5 Nov 2021 10:50:41 -   1.19
+++ ip.c9 Nov 2021 15:16:57 -
@@ -234,41 +234,6 @@ ip_addr_parse(const ASN1_BIT_STRING *p,
 }
 
 /*
- * Convert the IPv4 address into CIDR notation conforming to RFC 4632.
- * Buffer should be able to hold xxx.yyy.zzz.www/nn.
- */
-static void
-ip4_addr2str(const struct ip_addr *addr, char *b, size_t bsz)
-{
-   char buf[16];
-   int ret;
-
-   if (inet_ntop(AF_INET, addr->addr, buf, sizeof(buf)) == NULL)
-   err(1, "inet_ntop");
-   ret = snprintf(b, bsz, "%s/%hhu", buf, addr->prefixlen);
-   if (ret < 0 || (size_t)ret >= bsz)
-   err(1, "malformed IPV4 address");
-}
-
-/*
- * Convert the IPv6 address into CIDR notation conforming to RFC 4291.
- * See also RFC 5952.
- * Must hold :::::::/nn.
- */
-static void
-ip6_addr2str(const struct ip_addr *addr, char *b, size_t bsz)
-{
-   char buf[44];
-   int ret;
-
-   if (inet_ntop(AF_INET6, addr->addr, buf, sizeof(buf)) == NULL)
-   err(1, "inet_ntop");
-   ret = snprintf(b, bsz, "%s/%hhu", buf, addr->prefixlen);
-   if (ret < 0 || (size_t)ret >= bsz)
-   err(1, "malformed IPV6 address");
-}
-
-/*
  * Convert a ip_addr into a NUL-terminated CIDR notation string
  * conforming to RFC 4632 or 4291.
  * The size of the buffer must be at least 64 (inclusive).
@@ -277,11 +242,17 @@ void
 ip_addr_print(const struct ip_addr *addr,
 enum afi afi, char *buf, size_t bufsz)
 {
+   char ipbuf[44];
+   int ret, af = AF_INET;
+
+   if (afi == AFI_IPV6)
+   af = AF_INET6;
 
-   if (afi == AFI_IPV4)
-   ip4_addr2str(addr, buf, bufsz);
-   else
-   ip6_addr2str(addr, buf, bufsz);
+   if (inet_ntop(af, addr->addr, ipbuf, sizeof(ipbuf)) == NULL)
+   err(1, "inet_ntop");
+   ret = snprintf(buf, bufsz, "%s/%hhu", ipbuf, addr->prefixlen);
+   if (ret < 0 || (size_t)ret >= bufsz)
+   err(1, "malformed IP address");
 }
 
 /*



Re: sppp(4)/pppoe(4) - DNS configuration via resolvd(8)

2021-11-08 Thread Claudio Jeker
On Mon, Nov 08, 2021 at 11:52:52AM +0100, Bjorn Ketelaars wrote:
> Diff below does two things:
> 1. add PPP IPCP extensions for name server addresses (rfc1877) to
>sppp(4)
> 2. propose negotiated name servers from sppp(4) to resolvd(8) using
>RTM_PROPOSAL_STATIC route messages.
> 
> With this I'm able to use DNS servers as provided by my ISP who uses
> PPPoE. resolv.conf is updated by resolvd(8) as function of status
> changes of pppoe(4).
> 
> rfc1877 implementation is derived from code from NetBSD, and inspired by
> [0] and [1]. Borrowed code from umb(4) for the route messages.
> 
> Opinions/comments/OK?
> 
> 
> [0] https://marc.info/?l=openbsd-tech=134943767022961=2
> [1] https://marc.info/?l=openbsd-tech=159405677416423=2
> 
> 
> diff --git sys/net/if_sppp.h sys/net/if_sppp.h
> index ff559fcc369..f2dab61e46b 100644
> --- sys/net/if_sppp.h
> +++ sys/net/if_sppp.h
> @@ -132,6 +132,8 @@ struct sipcp {
> * original one here, in network byte order */
>   u_int32_t req_hisaddr;  /* remote address requested (IPv4) */
>   u_int32_t req_myaddr;   /* local address requested (IPv4) */
> +#define IPCP_MAX_DNSSRV  2
> + u_int32_t dns[IPCP_MAX_DNSSRV]; /* IPv4  DNS servers (RFC 1877) */

I would very much prefer to use struct in_addr here (or maybe in_addr_t).
It makes it more clear what the data is.

>  #ifdef INET6
>   struct in6_aliasreq req_ifid;   /* local ifid requested (IPv6) */
>  #endif
> diff --git sys/net/if_spppsubr.c sys/net/if_spppsubr.c
> index ac1dc9a709d..225ad8f5a3e 100644
> --- sys/net/if_spppsubr.c
> +++ sys/net/if_spppsubr.c
> @@ -132,6 +132,10 @@
>  #define IPCP_OPT_ADDRESSES   1   /* both IP addresses; deprecated */
>  #define IPCP_OPT_COMPRESSION 2   /* IP compression protocol (VJ) */
>  #define IPCP_OPT_ADDRESS 3   /* local IP address */
> +#define IPCP_OPT_PRIMDNS 129 /* primary remote dns address */
> +#define _IPCP_OPT_PRIMDNS_   4   /* set/check option */
> +#define IPCP_OPT_SECDNS  131 /* secondary remote dns address 
> */
> +#define _IPCP_OPT_SECDNS_5   /* set/check option */

As kn@ noted this underscore version is a bit rough.
I wonder if IPCP_OPT definitions (from the standard) should be decoupled
from the bitmask values to enable/disable a feature.

  
>  #define IPV6CP_OPT_IFID  1   /* interface identifier */
>  #define IPV6CP_OPT_COMPRESSION   2   /* IPv6 compression protocol */
> @@ -338,6 +342,8 @@ void sppp_update_gw(struct ifnet *ifp);
>  void sppp_set_ip_addrs(void *);
>  void sppp_clear_ip_addrs(void *);
>  void sppp_set_phase(struct sppp *sp);
> +void sppp_update_dns(struct ifnet *ifp);
> +void sppp_rtrequest(struct ifnet *ifp, int req, struct rtentry *rt);
>  
>  /* our control protocol descriptors */
>  static const struct cp lcp = {
> @@ -701,6 +707,7 @@ sppp_attach(struct ifnet *ifp)
>  
>   sp->pp_if.if_type = IFT_PPP;
>   sp->pp_if.if_output = sppp_output;
> + sp->pp_if.if_rtrequest = sppp_rtrequest;
>   ifq_set_maxlen(>pp_if.if_snd, 50);
>   mq_init(>pp_cpq, 50, IPL_NET);
>   sp->pp_loopcnt = 0;
> @@ -2519,6 +2526,12 @@ sppp_ipcp_RCN_rej(struct sppp *sp, struct lcp_header 
> *h, int len)
>   sp->ipcp.opts &= ~(1 << IPCP_OPT_COMPRESS);
>   break;
>  #endif
> + case IPCP_OPT_PRIMDNS:
> + sp->ipcp.opts &= ~(1 << _IPCP_OPT_PRIMDNS_);
> + break;
> + case IPCP_OPT_SECDNS:
> + sp->ipcp.opts &= ~(1 << _IPCP_OPT_SECDNS_);
> + break;
>   }
>   }
>   if (debug)
> @@ -2584,6 +2597,16 @@ sppp_ipcp_RCN_nak(struct sppp *sp, struct lcp_header 
> *h, int len)
>*/
>   break;
>  #endif
> + case IPCP_OPT_PRIMDNS:
> + if (len >= 6 && p[1] == 6)
> + sp->ipcp.dns[0] = p[2] << 24 | p[3] << 16 |
> + p[4] << 8 | p[5];

Please use memcpy and here. Since this remains in network byte order it
should just work.

> + break;
> + case IPCP_OPT_SECDNS:
> + if (len >= 6 && p[1] == 6)
> + sp->ipcp.dns[1] = p[2] << 24 | p[3] << 16 |
> + p[4] << 8 | p[5];

See above.

> + break;
>   }
>   }
>   if (debug)
> @@ -2612,6 +2635,7 @@ sppp_ipcp_tls(struct sppp *sp)
>   IPCP_MYADDR_DYN|IPCP_HISADDR_DYN);
>   sp->ipcp.req_myaddr = 0;
>   sp->ipcp.req_hisaddr = 0;
> + memset(>ipcp.dns, 0, sizeof sp->ipcp.dns);
>  
>   sppp_get_ip_addrs(sp, , , 0);
>   /*
> @@ -2644,6 +2668,10 @@ sppp_ipcp_tls(struct sppp *sp)
>   sp->ipcp.flags |= IPCP_HISADDR_DYN;
>   }
>  
> + /* negotiate name server addresses */
> + sp->ipcp.opts |= (1 << _IPCP_OPT_PRIMDNS_);
> + 

Re: rpki-client show attr name in rrdp parse errors

2021-11-05 Thread Claudio Jeker
On Wed, Nov 03, 2021 at 12:58:17PM +0100, Claudio Jeker wrote:
> In one place this is already done but this makes sure we show the bad
> attribute in all cases where a non conforming attribute is found.

Found another bunch of those non conforming attribute errors. Adjust them
as well.

OK?
-- 
:wq Claudio

Index: rrdp_notification.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rrdp_notification.c,v
retrieving revision 1.9
diff -u -p -r1.9 rrdp_notification.c
--- rrdp_notification.c 29 Oct 2021 09:27:36 -  1.9
+++ rrdp_notification.c 5 Nov 2021 14:06:18 -
@@ -141,7 +141,7 @@ start_notification_elem(struct notificat
continue;
}
PARSE_FAIL(p, "parse failed - non conforming "
-   "attribute found in notification elem");
+   "attribute '%s' found in notification elem", attr[i]);
}
if (!(has_xmlns && nxml->version && nxml->session_id && nxml->serial))
PARSE_FAIL(p, "parse failed - incomplete "
@@ -185,7 +185,7 @@ start_snapshot_elem(struct notification_
continue;
}
PARSE_FAIL(p, "parse failed - non conforming "
-   "attribute found in snapshot elem");
+   "attribute '%s' found in snapshot elem", attr[i]);
}
if (hasUri != 1 || hasHash != 1)
PARSE_FAIL(p, "parse failed - incomplete snapshot attributes");
@@ -239,7 +239,7 @@ start_delta_elem(struct notification_xml
continue;
}
PARSE_FAIL(p, "parse failed - non conforming "
-   "attribute found in snapshot elem");
+   "attribute '%s' found in snapshot elem", attr[i]);
}
/* Only add to the list if we are relevant */
if (hasUri != 1 || hasHash != 1 || delta_serial == 0)



speedup io marshal in rpki-client

2021-11-05 Thread Claudio Jeker
Noticed the other day. The ip addr arrays and as number array are
marshalled element by element which is not very efficent.
All the data is in one big blob of memory so just use the basic io
operations for a memory blob and ship the full array at once.

This seems to reduce runtime by 5-10% (in my unscientific testing).
Also it makes the code a fair bit simpler.
-- 
:wq Claudio

Index: cert.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/cert.c,v
retrieving revision 1.46
diff -u -p -r1.46 cert.c
--- cert.c  4 Nov 2021 11:32:55 -   1.46
+++ cert.c  5 Nov 2021 07:53:50 -
@@ -1225,34 +1225,6 @@ cert_free(struct cert *p)
free(p);
 }
 
-static void
-cert_ip_buffer(struct ibuf *b, const struct cert_ip *p)
-{
-   io_simple_buffer(b, >afi, sizeof(enum afi));
-   io_simple_buffer(b, >type, sizeof(enum cert_ip_type));
-
-   if (p->type != CERT_IP_INHERIT) {
-   io_simple_buffer(b, >min, sizeof(p->min));
-   io_simple_buffer(b, >max, sizeof(p->max));
-   }
-
-   if (p->type == CERT_IP_RANGE)
-   ip_addr_range_buffer(b, >range);
-   else if (p->type == CERT_IP_ADDR)
-   ip_addr_buffer(b, >ip);
-}
-
-static void
-cert_as_buffer(struct ibuf *b, const struct cert_as *p)
-{
-   io_simple_buffer(b, >type, sizeof(enum cert_as_type));
-   if (p->type == CERT_AS_RANGE) {
-   io_simple_buffer(b, >range.min, sizeof(uint32_t));
-   io_simple_buffer(b, >range.max, sizeof(uint32_t));
-   } else if (p->type == CERT_AS_ID)
-   io_simple_buffer(b, >id, sizeof(uint32_t));
-}
-
 /*
  * Write certificate parsed content into buffer.
  * See cert_read() for the other side of the pipe.
@@ -1260,18 +1232,15 @@ cert_as_buffer(struct ibuf *b, const str
 void
 cert_buffer(struct ibuf *b, const struct cert *p)
 {
-   size_t   i;
-
io_simple_buffer(b, >expires, sizeof(p->expires));
io_simple_buffer(b, >purpose, sizeof(p->purpose));
io_simple_buffer(b, >talid, sizeof(p->talid));
io_simple_buffer(b, >ipsz, sizeof(p->ipsz));
-   for (i = 0; i < p->ipsz; i++)
-   cert_ip_buffer(b, >ips[i]);
-
io_simple_buffer(b, >asz, sizeof(p->asz));
-   for (i = 0; i < p->asz; i++)
-   cert_as_buffer(b, >as[i]);
+
+   io_simple_buffer(b, p->ips, p->ipsz * sizeof(p->ips[0]));
+   io_simple_buffer(b, p->as, p->asz * sizeof(p->as[0]));
+
io_str_buffer(b, p->mft);
io_str_buffer(b, p->notify);
io_str_buffer(b, p->repo);
@@ -1282,34 +1251,6 @@ cert_buffer(struct ibuf *b, const struct
io_str_buffer(b, p->pubkey);
 }
 
-static void
-cert_ip_read(struct ibuf *b, struct cert_ip *p)
-{
-   io_read_buf(b, >afi, sizeof(enum afi));
-   io_read_buf(b, >type, sizeof(enum cert_ip_type));
-
-   if (p->type != CERT_IP_INHERIT) {
-   io_read_buf(b, >min, sizeof(p->min));
-   io_read_buf(b, >max, sizeof(p->max));
-   }
-
-   if (p->type == CERT_IP_RANGE)
-   ip_addr_range_read(b, >range);
-   else if (p->type == CERT_IP_ADDR)
-   ip_addr_read(b, >ip);
-}
-
-static void
-cert_as_read(struct ibuf *b, struct cert_as *p)
-{
-   io_read_buf(b, >type, sizeof(enum cert_as_type));
-   if (p->type == CERT_AS_RANGE) {
-   io_read_buf(b, >range.min, sizeof(uint32_t));
-   io_read_buf(b, >range.max, sizeof(uint32_t));
-   } else if (p->type == CERT_AS_ID)
-   io_read_buf(b, >id, sizeof(uint32_t));
-}
-
 /*
  * Allocate and read parsed certificate content from descriptor.
  * The pointer must be freed with cert_free().
@@ -1319,7 +1260,6 @@ struct cert *
 cert_read(struct ibuf *b)
 {
struct cert *p;
-   size_t   i;
 
if ((p = calloc(1, sizeof(struct cert))) == NULL)
err(1, NULL);
@@ -1328,19 +1268,17 @@ cert_read(struct ibuf *b)
io_read_buf(b, >purpose, sizeof(p->purpose));
io_read_buf(b, >talid, sizeof(p->talid));
io_read_buf(b, >ipsz, sizeof(p->ipsz));
+   io_read_buf(b, >asz, sizeof(p->asz));
 
p->ips = calloc(p->ipsz, sizeof(struct cert_ip));
if (p->ips == NULL)
err(1, NULL);
-   for (i = 0; i < p->ipsz; i++)
-   cert_ip_read(b, >ips[i]);
+   io_read_buf(b, p->ips, p->ipsz * sizeof(p->ips[0]));
 
-   io_read_buf(b, >asz, sizeof(p->asz));
p->as = calloc(p->asz, sizeof(struct cert_as));
if (p->as == NULL)
err(1, NULL);
-   for (i = 0; i < p->asz; i++)
-   cert_as_read(b, >as[i]);
+   io_read_buf(b, p->as, p->asz * sizeof(p->as[0]));
 
io_read_str(b, >mft);
io_read_str(b, >notify);
Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.93
diff -u -p -r1.93 

Re: rpki-client better exit behaviour when something goes wrong

2021-11-04 Thread Claudio Jeker
On Thu, Nov 04, 2021 at 11:27:46AM -0600, Theo de Raadt wrote:
> Claudio Jeker  wrote:
> 
> > This diff replaces the errx() call in the poll fd check with warnings plus
> > an exit of the main event loop. It also prints an error in case not all
> > files have been processed.
> > 
> > An example after kill -9 of the rsync process is:
> > rpki-client: https://rrdp.lacnic.net/rrdp/notification.xml: loaded from 
> > network
> > rpki-client: poll[1]: hangup
> > rpki-client: rsync terminated signal 9
> 
> I am not thrilled with giving people error messages about a system call 
> (poll).
> 
> In this specific case, the actual issue is on the next like (it is a rsync
> failure, which you caused I suspect)
> 
> Is that 2nd message not enough?
> 
> Could you set hangup = 1, but skip the warnx, so that the code just reacts
> correctly?
> 
> Are there other circumstances (different types of fd), which do not make
> it to a wait (rrdp?).  Can those report a nice termination message?
> 

I agree that the hangup message is not very helpful. It is actually 
the one message I did not alter. I'm happy to remove that message.
For the errors from msgbuf_write() I think those are valid error messages
and I feel the POLLERR|POLLNVAL fall into the same boat.

Updated diff below
-- 
:wq Claudio

Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.48
diff -u -p -r1.48 http.c
--- http.c  4 Nov 2021 14:24:41 -   1.48
+++ http.c  4 Nov 2021 14:25:03 -
@@ -159,7 +159,7 @@ static uint8_t *tls_ca_mem;
 static size_t tls_ca_size;
 
 /* HTTP request API */
-static voidhttp_req_new(size_t, char *, char *, int);
+static voidhttp_req_new(size_t, char *, char *, int, int);
 static voidhttp_req_free(struct http_request *);
 static voidhttp_req_done(size_t, enum http_result, const char *);
 static voidhttp_req_fail(size_t);
@@ -507,7 +507,7 @@ http_resolv(struct addrinfo **res, const
  * Create and queue a new request.
  */
 static void
-http_req_new(size_t id, char *uri, char *modified_since, int outfd)
+http_req_new(size_t id, char *uri, char *modified_since, int count, int outfd)
 {
struct http_request *req;
char *host, *port, *path;
@@ -530,6 +530,7 @@ http_req_new(size_t id, char *uri, char 
req->path = path;
req->uri = uri;
req->modified_since = modified_since;
+   req->redirect_loop = count;
 
TAILQ_INSERT_TAIL(, req, entry);
 }
@@ -1135,7 +1136,8 @@ http_redirect(struct http_connection *co
err(1, NULL);
 
logx("redirect to %s", http_info(uri));
-   http_req_new(conn->req->id, uri, mod_since, outfd); 
+   http_req_new(conn->req->id, uri, mod_since, conn->req->redirect_loop,
+   outfd); 
 
/* clear request before moving connection to idle */
http_req_free(conn->req);
@@ -1867,7 +1869,7 @@ proc_http(char *bind_addr, int fd)
io_read_str(b, );
 
/* queue up new requests */
-   http_req_new(id, uri, mod, b->fd);
+   http_req_new(id, uri, mod, 0, b->fd);
ibuf_free(b);
}
}
Index: main.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/main.c,v
retrieving revision 1.162
diff -u -p -r1.162 main.c
--- main.c  4 Nov 2021 14:24:41 -   1.162
+++ main.c  4 Nov 2021 17:42:42 -
@@ -1020,19 +1020,23 @@ main(int argc, char *argv[])
}
 
for (i = 0; i < NPFD; i++) {
-   if (pfd[i].revents & (POLLERR|POLLNVAL))
-   errx(1, "poll[%zu]: bad fd", i);
-   if (pfd[i].revents & POLLHUP) {
-   warnx("poll[%zu]: hangup", i);
+   if (pfd[i].revents & (POLLERR|POLLNVAL)) {
+   warnx("poll[%zu]: bad fd", i);
hangup = 1;
}
+   if (pfd[i].revents & POLLHUP)
+   hangup = 1;
if (pfd[i].revents & POLLOUT) {
switch (msgbuf_write(queues[i])) {
case 0:
-   errx(1, "write[%zu]: "
+   warnx("write[%zu]: "
"connection closed", i);
+   hangup = 1;
+

rpki-client X509_free XXX fix

2021-11-04 Thread Claudio Jeker
There is this bit in parser.c
X509_free(x509); // needed? XXX

As tb@ properly noted this X509_free() is needed because the cert_parse()
returns an up referenced x509 pointer back.

I moved the X509_free() so the error cases become simpler and we no longer
leak a reference on success. At least this is how I read the code.

OK?
-- 
:wq Claudio

Index: parser.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/parser.c,v
retrieving revision 1.27
diff -u -p -r1.27 parser.c
--- parser.c4 Nov 2021 11:32:55 -   1.27
+++ parser.c4 Nov 2021 17:25:18 -
@@ -232,12 +232,12 @@ proc_parser_cert(const struct entity *en
X509_STORE_CTX_cleanup(ctx);
sk_X509_free(chain);
sk_X509_CRL_free(crls);
+   X509_free(x509);
 
cert->talid = a->cert->talid;
 
/* Validate the cert to get the parent */
if (!valid_cert(entp->file, , cert)) {
-   X509_free(x509); // needed? XXX
cert_free(cert);
return NULL;
}
@@ -247,7 +247,6 @@ proc_parser_cert(const struct entity *en
 */
if (cert->purpose == CERT_PURPOSE_CA) {
if (!auth_insert(, cert, a)) {
-   X509_free(x509); // needed? XXX
cert_free(cert);
return NULL;
}
@@ -318,20 +317,22 @@ proc_parser_root_cert(const struct entit
goto badcert;
}
 
+   X509_free(x509);
+
cert->talid = entp->talid;
 
/*
 * Add valid roots to the RPKI auth tree.
 */
if (!auth_insert(, cert, NULL)) {
-   X509_free(x509); // needed? XXX
cert_free(cert);
return NULL;
}
 
return cert;
+
  badcert:
-   X509_free(x509); // needed? XXX
+   X509_free(x509);
cert_free(cert);
return NULL;
 }



rpki-client better exit behaviour when something goes wrong

2021-11-04 Thread Claudio Jeker
This diff replaces the errx() call in the poll fd check with warnings plus
an exit of the main event loop. It also prints an error in case not all
files have been processed.

An example after kill -9 of the rsync process is:
rpki-client: https://rrdp.lacnic.net/rrdp/notification.xml: loaded from network
rpki-client: poll[1]: hangup
rpki-client: rsync terminated signal 9
rpki-client: not all files processed, giving up

I find this better.
-- 
:wq Claudio

Index: main.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/main.c,v
retrieving revision 1.162
diff -u -p -r1.162 main.c
--- main.c  4 Nov 2021 14:24:41 -   1.162
+++ main.c  4 Nov 2021 17:12:27 -
@@ -1020,8 +1020,10 @@ main(int argc, char *argv[])
}
 
for (i = 0; i < NPFD; i++) {
-   if (pfd[i].revents & (POLLERR|POLLNVAL))
-   errx(1, "poll[%zu]: bad fd", i);
+   if (pfd[i].revents & (POLLERR|POLLNVAL)) {
+   warnx("poll[%zu]: bad fd", i);
+   hangup = 1;
+   }
if (pfd[i].revents & POLLHUP) {
warnx("poll[%zu]: hangup", i);
hangup = 1;
@@ -1029,10 +1031,14 @@ main(int argc, char *argv[])
if (pfd[i].revents & POLLOUT) {
switch (msgbuf_write(queues[i])) {
case 0:
-   errx(1, "write[%zu]: "
+   warnx("write[%zu]: "
"connection closed", i);
+   hangup = 1;
+   break;
case -1:
-   err(1, "write[%zu]", i);
+   warnx("write[%zu]", i);
+   hangup = 1;
+   break;
}
}
}
@@ -1147,7 +1153,7 @@ main(int argc, char *argv[])
 
/* processing did not finish because of error */
if (entity_queue != 0)
-   return 1;
+   errx(1, "not all files processed, giving up");
 
logx("all files parsed: generating output");
 



rpki-client refactor repo code a bit

2021-11-04 Thread Claudio Jeker
When RRDP support was added a repo was added for every caRepository URI
that was different from the others. Now the big RIR repos have many
caRepoistory URIs that are just subdirs and are covered by the same rsync
or RRDP source.

This diff changes this back to not create a new repo for every
caRepository URI. Instead it uses both the rsync base repo URI
(rsync://hostname/module) and RRDP notify URI to decide if a repo is the
same for different certs. With this the reported number of repositories
goes from 26k down to 50.

The lookup function needs to handle the fact that the notifyuri can be
NULL. This is why this is not just two simple strcmp calls.

More cleanup is possible but this is a good first step.
-- 
:wq Claudio

Index: repo.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/repo.c,v
retrieving revision 1.9
diff -u -p -r1.9 repo.c
--- repo.c  12 Aug 2021 15:27:15 -  1.9
+++ repo.c  4 Nov 2021 12:04:33 -
@@ -88,7 +88,8 @@ SLIST_HEAD(, tarepo)  tarepos = SLIST_HEA
 
 struct repo {
SLIST_ENTRY(repo)entry;
-   char*repouri;   /* CA repository base URI */
+   char*repouri;
+   char*notifyuri;
const struct rrdprepo   *rrdp;
const struct rsyncrepo  *rsync;
const struct tarepo *ta;
@@ -1089,18 +1090,33 @@ ta_lookup(struct tal *tal)
 struct repo *
 repo_lookup(const char *uri, const char *notify)
 {
-   struct repo *rp;
+   struct repo *rp;
+   char*repouri;
+
+   if ((repouri = rsync_base_uri(uri)) == NULL)
+   errx(1, "bad caRepository URI: %s", uri);
 
/* Look up in repository table. */
SLIST_FOREACH(rp, , entry) {
-   if (strcmp(rp->repouri, uri) != 0)
+   if (strcmp(rp->repouri, repouri) != 0)
+   continue;
+   if (rp->notifyuri != NULL) {
+   if (notify == NULL)
+   continue;
+   if (strcmp(rp->notifyuri, notify) != 0)
+   continue;
+   } else if (notify != NULL)
continue;
+   /* found matching repo */
+   free(repouri);
return rp;
}
 
rp = repo_alloc();
-   if ((rp->repouri = strdup(uri)) == NULL)
-   err(1, NULL);
+   rp->repouri = repouri;
+   if (notify != NULL)
+   if ((rp->notifyuri = strdup(notify)) == NULL)
+   err(1, NULL);
 
/* try RRDP first if available */
if (notify != NULL)



Re: rpki-client refactor tal handling

2021-11-03 Thread Claudio Jeker
On Wed, Nov 03, 2021 at 12:41:52PM -0600, Theo de Raadt wrote:
> +   size_t   talid; /* covered by which TAL */
> 
> You shouldn't use size_t
> 
> It is 32bit on ILP32 systems, and 64bit on I32LP64 machines, because the
> underlying definition is:
> 
> _types.h:typedefunsigned long   __size_t;
> 
> So suspect you want to use int or u_int.

Other code uses size_t because the ids are used as index in arrays.
It is overkill here the maximum number of TAL is 8 right now. That fits in
any kind of int. 

Here an adjusted diff
-- 
:wq Claudio

Index: cert.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/cert.c,v
retrieving revision 1.45
diff -u -p -r1.45 cert.c
--- cert.c  2 Nov 2021 19:30:30 -   1.45
+++ cert.c  3 Nov 2021 18:52:55 -
@@ -1220,7 +1220,6 @@ cert_free(struct cert *p)
free(p->aia);
free(p->aki);
free(p->ski);
-   free(p->tal);
free(p->pubkey);
X509_free(p->x509);
free(p);
@@ -1263,13 +1262,14 @@ cert_buffer(struct ibuf *b, const struct
 {
size_t   i;
 
-   io_simple_buffer(b, >expires, sizeof(time_t));
-   io_simple_buffer(b, >purpose, sizeof(enum cert_purpose));
-   io_simple_buffer(b, >ipsz, sizeof(size_t));
+   io_simple_buffer(b, >expires, sizeof(p->expires));
+   io_simple_buffer(b, >purpose, sizeof(p->purpose));
+   io_simple_buffer(b, >talid, sizeof(p->talid));
+   io_simple_buffer(b, >ipsz, sizeof(p->ipsz));
for (i = 0; i < p->ipsz; i++)
cert_ip_buffer(b, >ips[i]);
 
-   io_simple_buffer(b, >asz, sizeof(size_t));
+   io_simple_buffer(b, >asz, sizeof(p->asz));
for (i = 0; i < p->asz; i++)
cert_as_buffer(b, >as[i]);
io_str_buffer(b, p->mft);
@@ -1279,7 +1279,6 @@ cert_buffer(struct ibuf *b, const struct
io_str_buffer(b, p->aia);
io_str_buffer(b, p->aki);
io_str_buffer(b, p->ski);
-   io_str_buffer(b, p->tal);
io_str_buffer(b, p->pubkey);
 }
 
@@ -1325,9 +1324,10 @@ cert_read(struct ibuf *b)
if ((p = calloc(1, sizeof(struct cert))) == NULL)
err(1, NULL);
 
-   io_read_buf(b, >expires, sizeof(time_t));
-   io_read_buf(b, >purpose, sizeof(enum cert_purpose));
-   io_read_buf(b, >ipsz, sizeof(size_t));
+   io_read_buf(b, >expires, sizeof(p->expires));
+   io_read_buf(b, >purpose, sizeof(p->purpose));
+   io_read_buf(b, >talid, sizeof(p->talid));
+   io_read_buf(b, >ipsz, sizeof(p->ipsz));
 
p->ips = calloc(p->ipsz, sizeof(struct cert_ip));
if (p->ips == NULL)
@@ -1335,7 +1335,7 @@ cert_read(struct ibuf *b)
for (i = 0; i < p->ipsz; i++)
cert_ip_read(b, >ips[i]);
 
-   io_read_buf(b, >asz, sizeof(size_t));
+   io_read_buf(b, >asz, sizeof(p->asz));
p->as = calloc(p->asz, sizeof(struct cert_as));
if (p->as == NULL)
err(1, NULL);
@@ -1349,7 +1349,6 @@ cert_read(struct ibuf *b)
io_read_str(b, >aia);
io_read_str(b, >aki);
io_read_str(b, >ski);
-   io_read_str(b, >tal);
io_read_str(b, >pubkey);
 
assert(p->mft != NULL || p->purpose == CERT_PURPOSE_BGPSEC_ROUTER);
@@ -1406,8 +1405,7 @@ insert_brk(struct brk_tree *tree, struct
 
b->asid = asid;
b->expires = cert->expires;
-   if ((b->tal = strdup(cert->tal)) == NULL)
-   err(1, NULL);
+   b->talid = cert->talid;
if ((b->ski = strdup(cert->ski)) == NULL)
err(1, NULL);
if ((b->pubkey = strdup(cert->pubkey)) == NULL)
@@ -1420,13 +1418,10 @@ insert_brk(struct brk_tree *tree, struct
if ((found = RB_INSERT(brk_tree, tree, b)) != NULL) {
if (found->expires < b->expires) {
found->expires = b->expires;
-   free(found->tal);
-   found->tal = b->tal;
-   b->tal = NULL;
+   found->talid = b->talid;
}
free(b->ski);
free(b->pubkey);
-   free(b->tal);
free(b);
}
 }
Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.89
diff -u -p -r1.89 extern.h
--- extern.h3 Nov 2021 10:50:18 -   1.89
+++ extern.h3 Nov 2021 18:51:02 -
@@ -118,6 +118,7 @@ struct cert {
size_t   ipsz; /* length of "ips" */
struct cert_as  *as; /* list of AS numbers and ranges */
size_t   asz; /* length of "asz" */
+   int  talid; /* cert is covered by which TAL */
char*repo; /* CA repository (rsync:// uri) */
char*mft; /* manifest (rsync:// uri) */
char*notify; /* RRDP notify (https:// uri) */
@@ -125,8 +126,7 @@ struct cert {

rpki-client refactor tal handling

2021-11-03 Thread Claudio Jeker
This diff changes how the certs and roa track the tal that covers them.
Instead of passing strings around use ids and a simple lookup table
for the description. This will make it possible to add tal ids to more
things.

Usual test run works and the output for openbgpd and json look sane.
-- 
:wq Claudio

Index: cert.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/cert.c,v
retrieving revision 1.45
diff -u -p -r1.45 cert.c
--- cert.c  2 Nov 2021 19:30:30 -   1.45
+++ cert.c  3 Nov 2021 17:45:36 -
@@ -1220,7 +1220,6 @@ cert_free(struct cert *p)
free(p->aia);
free(p->aki);
free(p->ski);
-   free(p->tal);
free(p->pubkey);
X509_free(p->x509);
free(p);
@@ -1265,6 +1264,7 @@ cert_buffer(struct ibuf *b, const struct
 
io_simple_buffer(b, >expires, sizeof(time_t));
io_simple_buffer(b, >purpose, sizeof(enum cert_purpose));
+   io_simple_buffer(b, >talid, sizeof(size_t));
io_simple_buffer(b, >ipsz, sizeof(size_t));
for (i = 0; i < p->ipsz; i++)
cert_ip_buffer(b, >ips[i]);
@@ -1279,7 +1279,6 @@ cert_buffer(struct ibuf *b, const struct
io_str_buffer(b, p->aia);
io_str_buffer(b, p->aki);
io_str_buffer(b, p->ski);
-   io_str_buffer(b, p->tal);
io_str_buffer(b, p->pubkey);
 }
 
@@ -1327,6 +1326,7 @@ cert_read(struct ibuf *b)
 
io_read_buf(b, >expires, sizeof(time_t));
io_read_buf(b, >purpose, sizeof(enum cert_purpose));
+   io_read_buf(b, >talid, sizeof(size_t));
io_read_buf(b, >ipsz, sizeof(size_t));
 
p->ips = calloc(p->ipsz, sizeof(struct cert_ip));
@@ -1349,7 +1349,6 @@ cert_read(struct ibuf *b)
io_read_str(b, >aia);
io_read_str(b, >aki);
io_read_str(b, >ski);
-   io_read_str(b, >tal);
io_read_str(b, >pubkey);
 
assert(p->mft != NULL || p->purpose == CERT_PURPOSE_BGPSEC_ROUTER);
@@ -1406,8 +1405,7 @@ insert_brk(struct brk_tree *tree, struct
 
b->asid = asid;
b->expires = cert->expires;
-   if ((b->tal = strdup(cert->tal)) == NULL)
-   err(1, NULL);
+   b->talid = cert->talid;
if ((b->ski = strdup(cert->ski)) == NULL)
err(1, NULL);
if ((b->pubkey = strdup(cert->pubkey)) == NULL)
@@ -1420,13 +1418,10 @@ insert_brk(struct brk_tree *tree, struct
if ((found = RB_INSERT(brk_tree, tree, b)) != NULL) {
if (found->expires < b->expires) {
found->expires = b->expires;
-   free(found->tal);
-   found->tal = b->tal;
-   b->tal = NULL;
+   found->talid = b->talid;
}
free(b->ski);
free(b->pubkey);
-   free(b->tal);
free(b);
}
 }
Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.89
diff -u -p -r1.89 extern.h
--- extern.h3 Nov 2021 10:50:18 -   1.89
+++ extern.h3 Nov 2021 17:58:11 -
@@ -118,6 +118,7 @@ struct cert {
size_t   ipsz; /* length of "ips" */
struct cert_as  *as; /* list of AS numbers and ranges */
size_t   asz; /* length of "asz" */
+   size_t   talid; /* cert is covered by which TAL */
char*repo; /* CA repository (rsync:// uri) */
char*mft; /* manifest (rsync:// uri) */
char*notify; /* RRDP notify (https:// uri) */
@@ -125,8 +126,7 @@ struct cert {
char*aia; /* AIA (or NULL, for trust anchor) */
char*aki; /* AKI (or NULL, for trust anchor) */
char*ski; /* SKI */
-   char*tal; /* basename of TAL for this cert */
-   enum cert_purposepurpose; /* Certificate Purpose (BGPSec or CA) 
*/
+   enum cert_purposepurpose; /* BGPSec or CA */
char*pubkey; /* Subject Public Key Info */
X509*x509; /* the cert */
time_t   expires; /* do not use after */
@@ -145,6 +145,7 @@ struct tal {
unsigned char   *pkey; /* DER-encoded public key */
size_t   pkeysz; /* length of pkey */
char*descr; /* basename of tal file */
+   size_t   id;
 };
 
 /*
@@ -192,11 +193,11 @@ struct roa {
uint32_t asid; /* asID of ROA (if 0, RFC 6483 sec 4) */
struct roa_ip   *ips; /* IP prefixes */
size_t   ipsz; /* number of IP prefixes */
+   size_t  talid; /* ROAs are covered by which TAL */
int  valid; /* validated resources */
char*aia; /* AIA */
char*aki; /* AKI */
char*ski; /* SKI */
-   char*tal; /* 

Re: rpki-client fix CRLF handling in tal parser

2021-11-03 Thread Claudio Jeker
On Wed, Nov 03, 2021 at 06:48:51PM +0100, Theo Buehler wrote:
> On Wed, Nov 03, 2021 at 06:34:05PM +0100, Claudio Jeker wrote:
> > Fix CRLF handling by properly setting nl to the right NUL byte.
> > In the CRLF case both \r\n are replaced by \0 and so the code
> > needs to adjust the nl pointer else valid_url() and the check for .cer
> > endings fail.
> 
> It feels odd to interrupt CRLF handling by advancing the buffer to the
> next line.
> 
> How about this?

Much better. OK claudio@
 
> Index: tal.c
> ===
> RCS file: /cvs/src/usr.sbin/rpki-client/tal.c,v
> retrieving revision 1.32
> diff -u -p -r1.32 tal.c
> --- tal.c 26 Oct 2021 16:12:54 -  1.32
> +++ tal.c 3 Nov 2021 17:47:40 -
> @@ -58,14 +58,16 @@ tal_parse_buffer(const char *fn, char *b
>   while ((nl = memchr(buf, '\n', len)) != NULL) {
>   line = buf;
>  
> - /* replace LF and optional CR with NUL */
> - *nl = '\0';
> - if (nl > line && nl[-1] == '\r')
> - nl[-1] = '\0';
> -
>   /* advance buffer to next line */
>   len -= nl + 1 - buf;
>   buf = nl + 1;
> +
> + /* replace LF and optional CR with NUL, point nl at first NUL */
> + *nl = '\0';
> + if (nl > line && nl[-1] == '\r') {
> + nl[-1] = '\0';
> + nl--;
> + }
>  
>   if (optcomment) {
>   /* if this is a comment, just eat the line */
> 

-- 
:wq Claudio



rpki-client fix CRLF handling in tal parser

2021-11-03 Thread Claudio Jeker
Fix CRLF handling by properly setting nl to the right NUL byte.
In the CRLF case both \r\n are replaced by \0 and so the code
needs to adjust the nl pointer else valid_url() and the check for .cer
endings fail.

This diff fixes the test.tal I added to regress.
-- 
:wq Claudio

Index: tal.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/tal.c,v
retrieving revision 1.32
diff -u -p -r1.32 tal.c
--- tal.c   26 Oct 2021 16:12:54 -  1.32
+++ tal.c   3 Nov 2021 17:28:41 -
@@ -67,6 +67,9 @@ tal_parse_buffer(const char *fn, char *b
len -= nl + 1 - buf;
buf = nl + 1;
 
+   if (nl > line && nl[-1] == '\0')
+   nl--;
+
if (optcomment) {
/* if this is a comment, just eat the line */
if (line[0] == '#')



rpki-client limit number of concurrent rsyncs

2021-11-03 Thread Claudio Jeker
Don't become a fork bomb. Limit the number of processes by stopping to
poll for new jobs once the limit is reached. I set the limit to 16 which
is larger then the biggest concurrency I have seen during fetches.

OK?
-- 
:wq Claudio

Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.89
diff -u -p -r1.89 extern.h
--- extern.h3 Nov 2021 10:50:18 -   1.89
+++ extern.h3 Nov 2021 14:25:53 -
@@ -618,4 +618,7 @@ int mkpath(const char *);
 /* Maximum depth of the RPKI tree. */
 #defineMAX_CERT_DEPTH  12
 
+/* Maximum number of concurrent rsync processes. */
+#define MAX_RSYNC_PROCESSES 16
+
 #endif /* ! EXTERN_H */
Index: rsync.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rsync.c,v
retrieving revision 1.29
diff -u -p -r1.29 rsync.c
--- rsync.c 28 Oct 2021 13:50:29 -  1.29
+++ rsync.c 3 Nov 2021 14:25:00 -
@@ -119,7 +119,7 @@ proc_child(int signal)
 void
 proc_rsync(char *prog, char *bind_addr, int fd)
 {
-   size_t   i, idsz = 0;
+   size_t   i, idsz = 0, nprocs = 0;
int  rc = 0;
struct pollfdpfd;
struct msgbufmsgq;
@@ -186,7 +186,9 @@ proc_rsync(char *prog, char *bind_addr, 
pid_t pid;
int st;
 
-   pfd.events = POLLIN;
+   pfd.events = 0;
+   if (nprocs < MAX_RSYNC_PROCESSES)
+   pfd.events |= POLLIN;
if (msgq.queued)
pfd.events |= POLLOUT;
 
@@ -228,6 +230,7 @@ proc_rsync(char *prog, char *bind_addr, 
ids[i].uri = NULL;
ids[i].pid = 0;
ids[i].id = 0;
+   nprocs--;
}
if (pid == -1 && errno != ECHILD)
err(1, "waitpid");
@@ -314,6 +317,7 @@ proc_rsync(char *prog, char *bind_addr, 
ids[i].id = id;
ids[i].pid = pid;
ids[i].uri = uri;
+   nprocs++;
 
/* Clean up temporary values. */
 



rpki-client show attr name in rrdp parse errors

2021-11-03 Thread Claudio Jeker
In one place this is already done but this makes sure we show the bad
attribute in all cases where a non conforming attribute is found.

OK?
-- 
:wq Claudio

Index: rrdp_delta.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rrdp_delta.c,v
retrieving revision 1.4
diff -u -p -r1.4 rrdp_delta.c
--- rrdp_delta.c28 Oct 2021 11:57:00 -  1.4
+++ rrdp_delta.c3 Nov 2021 11:52:42 -
@@ -87,7 +87,7 @@ start_delta_elem(struct delta_xml *dxml,
continue;
}
PARSE_FAIL(p, "parse failed - non conforming "
-   "attribute found in delta elem");
+   "attribute '%s' found in delta elem", attr[i]);
}
if (!(has_xmlns && dxml->version && dxml->session_id && dxml->serial))
PARSE_FAIL(p, "parse failed - incomplete delta attributes");
@@ -136,7 +136,7 @@ start_publish_withdraw_elem(struct delta
continue;
}
PARSE_FAIL(p, "parse failed - non conforming "
-   "attribute found in publish/withdraw elem");
+   "attribute '%s' found in publish/withdraw elem", attr[i]);
}
if (hasUri != 1)
PARSE_FAIL(p,
Index: rrdp_snapshot.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rrdp_snapshot.c,v
retrieving revision 1.3
diff -u -p -r1.3 rrdp_snapshot.c
--- rrdp_snapshot.c 28 Oct 2021 11:57:00 -  1.3
+++ rrdp_snapshot.c 3 Nov 2021 11:51:45 -
@@ -80,7 +80,7 @@ start_snapshot_elem(struct snapshot_xml 
}
PARSE_FAIL(p,
"parse failed - non conforming "
-   "attribute found in snapshot elem");
+   "attribute '%s' found in snapshot elem", attr[i]);
}
if (!(has_xmlns && sxml->version && sxml->session_id && sxml->serial))
PARSE_FAIL(p,



rpki-client fix chunked encoding for larger then 32k chunks

2021-11-03 Thread Claudio Jeker
Noticed by accident. The chunked encoding only works for chunks smaller
than 32k (the HTTP internal read buffer). For chunks bigger than 32k the
state machine jumps too early (after the first write of 32k) into the
STATE_RESPONSE_CHUNKED_TRAILER state and so everything gets confused.

Fix is simple. Check in data_write() that for chunked encoding iosz is
null before switching to STATE_RESPONSE_CHUNKED_TRAILER.
-- 
:wq Claudio

Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.46
diff -u -p -r1.46 http.c
--- http.c  29 Oct 2021 08:51:20 -  1.46
+++ http.c  3 Nov 2021 11:39:16 -
@@ -1679,7 +1679,7 @@ data_write(struct http_connection *conn)
 
/* all data written, switch back to read */
if (conn->bufpos == 0 || conn->iosz == 0) {
-   if (conn->chunked)
+   if (conn->chunked && conn->iosz == 0)
conn->state = STATE_RESPONSE_CHUNKED_TRAILER;
else
conn->state = STATE_RESPONSE_DATA;



Re: rpki-client sync output at end of run with output file

2021-10-28 Thread Claudio Jeker
On Thu, Oct 28, 2021 at 08:27:40PM +0200, Theo Buehler wrote:
> On Thu, Oct 28, 2021 at 08:21:12PM +0200, Claudio Jeker wrote:
> > As in $SUBJECT said, sync the output at the end of a rpki-client run with
> > outputheader() -- the function used to dump this info into the openbgpd
> > output file.
> > 
> > OK?
> > -- 
> > :wq Claudio
> > 
> > Index: main.c
> > ===
> > RCS file: /cvs/src/usr.sbin/rpki-client/main.c,v
> > retrieving revision 1.156
> > diff -u -p -r1.156 main.c
> > --- main.c  26 Oct 2021 16:59:19 -  1.156
> > +++ main.c  28 Oct 2021 13:52:27 -
> > @@ -1173,8 +1173,13 @@ main(int argc, char *argv[])
> > rc = 1;
> >  
> >  
> > +   logx("Processing time %lld seconds (%lld seconds user, "
> > +   "%lld seconds system)", stats.elapsed_time.tv_sec,
> > +   stats.user_time.tv_sec, stats.system_time.tv_sec);
> 
> This needs (long long) casts for -portable on systems with 32 bit time_t
> (the other output functions also have them).
> 
> otherwise ok
> 

Indeed, will fix and then commit.

-- 
:wq Claudio



rpki-client sync output at end of run with output file

2021-10-28 Thread Claudio Jeker
As in $SUBJECT said, sync the output at the end of a rpki-client run with
outputheader() -- the function used to dump this info into the openbgpd
output file.

OK?
-- 
:wq Claudio

Index: main.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/main.c,v
retrieving revision 1.156
diff -u -p -r1.156 main.c
--- main.c  26 Oct 2021 16:59:19 -  1.156
+++ main.c  28 Oct 2021 13:52:27 -
@@ -1173,8 +1173,13 @@ main(int argc, char *argv[])
rc = 1;
 
 
+   logx("Processing time %lld seconds (%lld seconds user, "
+   "%lld seconds system)", stats.elapsed_time.tv_sec,
+   stats.user_time.tv_sec, stats.system_time.tv_sec);
logx("Route Origin Authorizations: %zu (%zu failed parse, %zu invalid)",
stats.roas, stats.roas_fail, stats.roas_invalid);
+   logx("BGPsec Router Certificates: %zu (%zu invalid)",
+   stats.brks, stats.brks_invalids);
logx("Certificates: %zu (%zu failed parse, %zu invalid)",
stats.certs, stats.certs_fail, stats.certs_invalid);
logx("Trust Anchor Locators: %zu", stats.tals);
@@ -1182,8 +1187,6 @@ main(int argc, char *argv[])
stats.mfts, stats.mfts_fail, stats.mfts_stale);
logx("Certificate revocation lists: %zu", stats.crls);
logx("Ghostbuster records: %zu", stats.gbrs);
-   logx("BGPsec Router Certificates: %zu (%zu invalid)",
-   stats.brks, stats.brks_invalids);
logx("Repositories: %zu", stats.repos);
logx("Cleanup: removed %zu files, %zu directories",
stats.del_files, stats.del_dirs);



openrsync add --max-size and --min-size support

2021-10-28 Thread Claudio Jeker
This diff should implement --max-size and --min-size almost equivalent to
GNU rsync. I decided to use scan_scaled() instead of building something
new that handles all the extra bits GNU rsync has.
The remote rsync process gets the sizes in bytes so scaling is just a
local issue.

Manpage probably needs more love.
-- 
:wq Claudio

Index: Makefile
===
RCS file: /cvs/src/usr.bin/rsync/Makefile,v
retrieving revision 1.12
diff -u -p -r1.12 Makefile
--- Makefile22 Oct 2021 11:10:34 -  1.12
+++ Makefile28 Oct 2021 14:06:07 -
@@ -4,8 +4,8 @@ PROG=   openrsync
 SRCS=  blocks.c client.c copy.c downloader.c fargs.c flist.c hash.c ids.c \
io.c log.c main.c misc.c mkpath.c mktemp.c receiver.c rmatch.c \
rules.c sender.c server.c session.c socket.c symlinks.c uploader.c
-LDADD+= -lcrypto -lm
-DPADD+= ${LIBCRYPTO} ${LIBM}
+LDADD+= -lcrypto -lm -lutil
+DPADD+= ${LIBCRYPTO} ${LIBM} ${LIBUTIL}
 MAN=   openrsync.1
 
 CFLAGS+= -Wall -Wextra
Index: extern.h
===
RCS file: /cvs/src/usr.bin/rsync/extern.h,v
retrieving revision 1.42
diff -u -p -r1.42 extern.h
--- extern.h22 Oct 2021 11:10:34 -  1.42
+++ extern.h28 Oct 2021 13:59:20 -
@@ -141,6 +141,8 @@ struct  opts {
int  numeric_ids;   /* --numeric-ids */
int  one_file_system;   /* -x */
int  alt_base_mode;
+   off_tmax_size;  /* --max-size */
+   off_tmin_size;  /* --min-size */
char*rsync_path;/* --rsync-path */
char*ssh_prog;  /* --rsh or -e */
char*port;  /* --port */
Index: fargs.c
===
RCS file: /cvs/src/usr.bin/rsync/fargs.c,v
retrieving revision 1.20
diff -u -p -r1.20 fargs.c
--- fargs.c 22 Oct 2021 11:10:34 -  1.20
+++ fargs.c 28 Oct 2021 14:09:23 -
@@ -131,6 +131,10 @@ fargs_cmdline(struct sess *sess, const s
if (!sess->opts->specials && sess->opts->devices)
/* --devices is sent as -D --no-specials */
addargs(, "--no-specials");
+   if (sess->opts->max_size >= 0)
+   addargs(, "--max-size=%lld", sess->opts->max_size);
+   if (sess->opts->min_size >= 0)
+   addargs(, "--min-size=%lld", sess->opts->min_size);
 
/* only add --compare-dest, etc if this is the sender */
if (sess->opts->alt_base_mode != 0 && 
Index: main.c
===
RCS file: /cvs/src/usr.bin/rsync/main.c,v
retrieving revision 1.61
diff -u -p -r1.61 main.c
--- main.c  28 Oct 2021 13:07:43 -  1.61
+++ main.c  28 Oct 2021 15:17:39 -
@@ -26,6 +26,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "extern.h"
 
@@ -341,7 +342,7 @@ main(int argc, char *argv[])
pid_tchild;
int  fds[2], sd = -1, rc, c, st, i, lidx;
size_t   basedir_cnt = 0;
-   struct sess   sess;
+   struct sess  sess;
struct fargs*fargs;
char**args;
const char  *errstr;
@@ -352,6 +353,8 @@ main(int argc, char *argv[])
NULL) == -1)
err(ERR_IPC, "pledge");
 
+   opts.max_size = opts.min_size = -1;
+
while ((c = getopt_long(argc, argv, "Dae:ghlnoprtvxz", lopts, ))
!= -1) {
switch (c) {
@@ -472,8 +475,12 @@ basedir:
opts.basedir[basedir_cnt++] = optarg;
break;
case OP_MAX_SIZE:
+   if (scan_scaled(optarg, _size) == -1)
+   err(1, "bad max-size");
+   break;
case OP_MIN_SIZE:
-   /* for now simply ignore */
+   if (scan_scaled(optarg, _size) == -1)
+   err(1, "bad min-size");
break;
case OP_VERSION:
fprintf(stderr, "openrsync: protocol version %u\n",
Index: rsync.1
===
RCS file: /cvs/src/usr.bin/rsync/rsync.1,v
retrieving revision 1.27
diff -u -p -r1.27 rsync.1
--- rsync.1 22 Oct 2021 16:42:28 -  1.27
+++ rsync.1 28 Oct 2021 15:29:42 -
@@ -31,6 +31,8 @@
 .Op Fl -exclude-from Ns = Ns Ar file
 .Op Fl -include Ar pattern
 .Op Fl -include-from Ns = Ns Ar file
+.Op Fl -max-size Ns = Ns size
+.Op Fl -min-size Ns = Ns size
 .Op Fl -no-motd
 .Op Fl -numeric-ids
 .Op Fl -port Ns = Ns Ar service
@@ -127,6 +129,22 @@ set the numeric group ID to match the so
 Also transfer symbolic links.
 The link is transferred as a standalone file: if the destination does
 not 

rpki-client adjust tal parse to the BIO free world

2021-10-26 Thread Claudio Jeker
This is part 3 of the BIO removal. Switch tal_parse to pass a file buffer
like all other callers. The parent process can now just use load_file()
and pass that buffer to the parser. From there on the magic just happens.

-- 
:wq Claudio

Index: encoding.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/encoding.c,v
retrieving revision 1.4
diff -u -p -r1.4 encoding.c
--- encoding.c  11 Oct 2021 16:06:36 -  1.4
+++ encoding.c  26 Oct 2021 14:40:47 -
@@ -29,11 +29,11 @@
  * Returns 0 on success or -1 for any errors.
  */
 int
-base64_decode(const unsigned char *in, unsigned char **out, size_t *outlen)
+base64_decode(const unsigned char *in, size_t inlen,
+unsigned char **out, size_t *outlen)
 {
static EVP_ENCODE_CTX *ctx;
unsigned char *to;
-   size_t inlen;
int tolen;
 
if (ctx == NULL && (ctx = EVP_ENCODE_CTX_new()) == NULL)
@@ -42,7 +42,6 @@ base64_decode(const unsigned char *in, u
*out = NULL;
*outlen = 0;
 
-   inlen = strlen(in);
if (inlen >= INT_MAX - 3)
return -1;
tolen = ((inlen + 3) / 4) * 3 + 1;
Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.79
diff -u -p -r1.79 extern.h
--- extern.h26 Oct 2021 13:31:05 -  1.79
+++ extern.h26 Oct 2021 14:40:37 -
@@ -337,12 +337,12 @@ enum publish_type {
  * and parsed.
  */
 struct entity {
-   enum rtype   type; /* type of entity (not RTYPE_EOF) */
-   char*file; /* local path to file */
-   int  has_pkey; /* whether pkey/sz is specified */
-   unsigned char   *pkey; /* public key (optional) */
-   size_t   pkeysz; /* public key length (optional) */
-   char*descr; /* tal description */
+   enum rtype   type;  /* type of entity (not RTYPE_EOF) */
+   char*file;  /* local path to file */
+   int  has_data;  /* whether data blob is specified */
+   unsigned char   *data;  /* optional data blob */
+   size_t   datasz;/* length of optional data blob */
+   char*descr; /* tal description */
TAILQ_ENTRY(entity) entries;
 };
 TAILQ_HEAD(entityq, entity);
@@ -397,8 +397,7 @@ extern int verbose;
 
 voidtal_buffer(struct ibuf *, const struct tal *);
 voidtal_free(struct tal *);
-struct tal *tal_parse(const char *, char *);
-char   *tal_read_file(const char *);
+struct tal *tal_parse(const char *, char *, size_t);
 struct tal *tal_read(struct ibuf *);
 
 voidcert_buffer(struct ibuf *, const struct cert *);
@@ -534,8 +533,8 @@ void cryptoerrx(const char *, ...)
 
 /* Encoding functions for hex and base64. */
 
-int base64_decode(const unsigned char *, unsigned char **,
-   size_t *);
+int base64_decode(const unsigned char *, size_t,
+   unsigned char **, size_t *);
 int base64_encode(const unsigned char *, size_t, char **);
 char   *hex_encode(const unsigned char *, size_t);
 
@@ -595,8 +594,9 @@ int  output_csv(FILE *, struct vrp_tree
 int output_json(FILE *, struct vrp_tree *, struct brk_tree *,
struct stats *);
 
-void   logx(const char *fmt, ...)
+void   logx(const char *fmt, ...)
__attribute__((format(printf, 1, 2)));
+unsigned char  *load_file(const char *, size_t *);
 
 intmkpath(const char *);
 
Index: main.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/main.c,v
retrieving revision 1.154
diff -u -p -r1.154 main.c
--- main.c  24 Oct 2021 21:24:19 -  1.154
+++ main.c  26 Oct 2021 14:38:40 -
@@ -20,6 +20,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -82,13 +83,46 @@ logx(const char *fmt, ...)
}
 }
 
+unsigned char *
+load_file(const char *name, size_t *len)
+{
+   unsigned char *buf = NULL;
+   struct stat st;
+   ssize_t n;
+   size_t size;
+   int fd;
+
+   *len = 0;
+
+   if ((fd = open(name, O_RDONLY)) == -1)
+   return NULL;
+   if (fstat(fd, ) != 0)
+   goto err;
+   if (st.st_size < 0)
+   goto err;
+   size = (size_t)st.st_size;
+   if ((buf = malloc(size)) == NULL)
+   goto err;
+   n = read(fd, buf, size);
+   if (n < 0 || (size_t)n != size)
+   goto err;
+   close(fd);
+   *len = size;
+   return buf;
+
+err:
+   close(fd);
+   free(buf);
+   return NULL;
+}
+
 void
 entity_free(struct entity *ent)
 {
if (ent == NULL)
return;
 
-   free(ent->pkey);
+   free(ent->data);
  

remove more BIO from rpki-client part 2

2021-10-26 Thread Claudio Jeker
This switches the cert parser to use d2i_X509 instead of the BIO versions.

-- 
:wq Claudio

Index: cert.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/cert.c,v
retrieving revision 1.40
diff -u -p -r1.40 cert.c
--- cert.c  23 Oct 2021 16:06:04 -  1.40
+++ cert.c  26 Oct 2021 11:32:12 -
@@ -976,7 +976,8 @@ out:
  * is also dereferenced.
  */
 static struct cert *
-cert_parse_inner(X509 **xp, const char *fn, int ta)
+cert_parse_inner(X509 **xp, const char *fn, const unsigned char *der,
+size_t len, int ta)
 {
int  rc = 0, extsz, c;
int  sia_present = 0;
@@ -985,28 +986,19 @@ cert_parse_inner(X509 **xp, const char *
X509_EXTENSION  *ext = NULL;
ASN1_OBJECT *obj;
struct parse p;
-   BIO *bio = NULL;
-   FILE*f;
 
*xp = NULL;
 
-   if ((f = fopen(fn, "rb")) == NULL) {
-   warn("%s", fn);
+   /* just fail for empty buffers, the warning was printed elsewhere */
+   if (der == NULL)
return NULL;
-   }
-
-   if ((bio = BIO_new_fp(f, BIO_CLOSE)) == NULL) {
-   if (verbose > 0)
-   cryptowarnx("%s: BIO_new_file", fn);
-   return NULL;
-   }
 
memset(, 0, sizeof(struct parse));
p.fn = fn;
if ((p.res = calloc(1, sizeof(struct cert))) == NULL)
err(1, NULL);
 
-   if ((x = *xp = d2i_X509_bio(bio, NULL)) == NULL) {
+   if ((x = *xp = d2i_X509(NULL, , len)) == NULL) {
cryptowarnx("%s: d2i_X509_bio", p.fn);
goto out;
}
@@ -1144,7 +1136,6 @@ cert_parse_inner(X509 **xp, const char *
 
rc = 1;
 out:
-   BIO_free_all(bio);
if (rc == 0) {
cert_free(p.res);
X509_free(x);
@@ -1154,19 +1145,20 @@ out:
 }
 
 struct cert *
-cert_parse(X509 **xp, const char *fn)
+cert_parse(X509 **xp, const char *fn, const unsigned char *der, size_t len)
 {
-   return cert_parse_inner(xp, fn, 0);
+   return cert_parse_inner(xp, fn, der, len, 0);
 }
 
 struct cert *
-ta_parse(X509 **xp, const char *fn, const unsigned char *pkey, size_t pkeysz)
+ta_parse(X509 **xp, const char *fn, const unsigned char *der, size_t len,
+const unsigned char *pkey, size_t pkeysz)
 {
EVP_PKEY*pk = NULL, *opk = NULL;
struct cert *p;
int  rc = 0;
 
-   if ((p = cert_parse_inner(xp, fn, 1)) == NULL)
+   if ((p = cert_parse_inner(xp, fn, der, len, 1)) == NULL)
return NULL;
 
if (pkey != NULL) {
Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.78
diff -u -p -r1.78 extern.h
--- extern.h26 Oct 2021 10:52:49 -  1.78
+++ extern.h26 Oct 2021 10:58:44 -
@@ -403,8 +403,10 @@ struct tal *tal_read(struct ibuf *);
 
 voidcert_buffer(struct ibuf *, const struct cert *);
 voidcert_free(struct cert *);
-struct cert*cert_parse(X509 **, const char *);
-struct cert*ta_parse(X509 **, const char *, const unsigned char *, size_t);
+struct cert*cert_parse(X509 **, const char *, const unsigned char *,
+   size_t);
+struct cert*ta_parse(X509 **, const char *, const unsigned char *, size_t,
+   const unsigned char *, size_t);
 struct cert*cert_read(struct ibuf *);
 voidcert_insert_brks(struct brk_tree *, struct cert *);
 
Index: parser.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/parser.c,v
retrieving revision 1.18
diff -u -p -r1.18 parser.c
--- parser.c26 Oct 2021 10:52:50 -  1.18
+++ parser.c26 Oct 2021 10:59:06 -
@@ -191,7 +191,8 @@ proc_parser_mft(struct entity *entp, con
  * parse failure.
  */
 static struct cert *
-proc_parser_cert(const struct entity *entp)
+proc_parser_cert(const struct entity *entp, const unsigned char *der,
+size_t len)
 {
struct cert *cert;
X509*x509;
@@ -204,7 +205,7 @@ proc_parser_cert(const struct entity *en
 
/* Extract certificate data and X509. */
 
-   cert = cert_parse(, entp->file);
+   cert = cert_parse(, entp->file, der, len);
if (cert == NULL)
return NULL;
 
@@ -282,7 +283,8 @@ proc_parser_cert(const struct entity *en
  * parse failure.
  */
 static struct cert *
-proc_parser_root_cert(const struct entity *entp)
+proc_parser_root_cert(const struct entity *entp, const unsigned char *der,
+size_t len)
 {
charsubject[256];
ASN1_TIME   *notBefore, *notAfter;
@@ -296,7 +298,7 @@ proc_parser_root_cert(const struct entit
 
/* Extract certificate data and X509. */
 
-   cert = ta_parse(, entp->file, 

remove more BIO from rpki-client

2021-10-26 Thread Claudio Jeker
I want to be able to pass a buffer to the various parser functions instead
of a filename. This is in preparation for supporting rpki-client -f somefile

This diff switches CMS and CRL to their regular d2i versions. The cert
files will follow in the next diff.
-- 
:wq Claudio

Index: cms.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/cms.c,v
retrieving revision 1.10
diff -u -p -r1.10 cms.c
--- cms.c   9 Sep 2021 14:15:49 -   1.10
+++ cms.c   26 Oct 2021 09:43:10 -
@@ -35,14 +35,12 @@
  * Return the eContent as a string and set "rsz" to be its length.
  */
 unsigned char *
-cms_parse_validate(X509 **xp, const char *fn, const ASN1_OBJECT *oid,
-size_t *rsz)
+cms_parse_validate(X509 **xp, const char *fn, const unsigned char *der,
+size_t derlen, const ASN1_OBJECT *oid, size_t *rsz)
 {
const ASN1_OBJECT   *obj;
ASN1_OCTET_STRING   **os = NULL;
-   BIO *bio = NULL;
CMS_ContentInfo *cms;
-   FILE*f;
int  rc = 0;
STACK_OF(X509)  *certs = NULL;
unsigned char   *res = NULL;
@@ -50,21 +48,11 @@ cms_parse_validate(X509 **xp, const char
*rsz = 0;
*xp = NULL;
 
-   /*
-* This is usually fopen() failure, so let it pass through to
-* the handler, which will in turn ignore the entity.
-*/
-   if ((f = fopen(fn, "rb")) == NULL) {
-   warn("%s", fn);
-   return NULL;
-   }
-
-   if ((bio = BIO_new_fp(f, BIO_CLOSE)) == NULL) {
-   cryptowarnx("%s: BIO_new_fp", fn);
+   /* just fail for empty buffers, the warning was printed elsewhere */
+   if (der == NULL)
return NULL;
-   }
 
-   if ((cms = d2i_CMS_bio(bio, NULL)) == NULL) {
+   if ((cms = d2i_CMS_ContentInfo(NULL, , derlen)) == NULL) {
cryptowarnx("%s: RFC 6488: failed CMS parse", fn);
goto out;
}
@@ -74,8 +62,8 @@ cms_parse_validate(X509 **xp, const char
 * Verify that the self-signage is correct.
 */
 
-   if (!CMS_verify(cms, NULL, NULL,
-   NULL, NULL, CMS_NO_SIGNER_CERT_VERIFY)) {
+   if (!CMS_verify(cms, NULL, NULL, NULL, NULL,
+   CMS_NO_SIGNER_CERT_VERIFY)) {
cryptowarnx("%s: RFC 6488: CMS not self-signed", fn);
goto out;
}
@@ -134,7 +122,6 @@ cms_parse_validate(X509 **xp, const char
 
rc = 1;
 out:
-   BIO_free_all(bio);
sk_X509_free(certs);
CMS_ContentInfo_free(cms);
 
Index: crl.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/crl.c,v
retrieving revision 1.10
diff -u -p -r1.10 crl.c
--- crl.c   29 Jan 2021 10:13:16 -  1.10
+++ crl.c   26 Oct 2021 09:43:18 -
@@ -29,32 +29,22 @@
 #include "extern.h"
 
 X509_CRL *
-crl_parse(const char *fn)
+crl_parse(const char *fn, const unsigned char *der, size_t len)
 {
int  rc = 0;
X509_CRL*x = NULL;
-   BIO *bio = NULL;
-   FILE*f;
 
-   if ((f = fopen(fn, "rb")) == NULL) {
-   warn("%s", fn);
+   /* just fail for empty buffers, the warning was printed elsewhere */
+   if (der == NULL)
return NULL;
-   }
-
-   if ((bio = BIO_new_fp(f, BIO_CLOSE)) == NULL) {
-   if (verbose > 0)
-   cryptowarnx("%s: BIO_new_file", fn);
-   return NULL;
-   }
 
-   if ((x = d2i_X509_CRL_bio(bio, NULL)) == NULL) {
-   cryptowarnx("%s: d2i_X509_CRL_bio", fn);
+   if ((x = d2i_X509_CRL(NULL, , len)) == NULL) {
+   cryptowarnx("%s: d2i_X509_CRL", fn);
goto out;
}
 
rc = 1;
 out:
-   BIO_free_all(bio);
if (rc == 0) {
X509_CRL_free(x);
x = NULL;
Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.77
diff -u -p -r1.77 extern.h
--- extern.h24 Oct 2021 17:53:07 -  1.77
+++ extern.h25 Oct 2021 18:42:49 -
@@ -410,22 +410,25 @@ void   cert_insert_brks(struct brk_tree 
 
 voidmft_buffer(struct ibuf *, const struct mft *);
 voidmft_free(struct mft *);
-struct mft *mft_parse(X509 **, const char *);
+struct mft *mft_parse(X509 **, const char *, const unsigned char *,
+   size_t);
 int mft_check(const char *, struct mft *);
 struct mft *mft_read(struct ibuf *);
 
 voidroa_buffer(struct ibuf *, const struct roa *);
 voidroa_free(struct roa *);
-struct roa *roa_parse(X509 **, const char *);
+struct roa *roa_parse(X509 **, const char *, const unsigned char *,
+   

rpki-client proc_parser cleanup

2021-10-25 Thread Claudio Jeker
Refactor this code and instead of passing various things around just use
globals.

-- 
:wq Claudio

Index: parser.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/parser.c,v
retrieving revision 1.16
diff -u -p -r1.16 parser.c
--- parser.c23 Oct 2021 20:01:16 -  1.16
+++ parser.c25 Oct 2021 16:55:11 -
@@ -38,20 +38,23 @@
 #include "extern.h"
 
 static void build_chain(const struct auth *, STACK_OF(X509) **);
-static struct crl  *get_crl(const struct auth *, struct crl_tree *);
+static struct crl  *get_crl(const struct auth *);
 static void build_crls(const struct crl *, STACK_OF(X509_CRL) **);
 
 /* Limit how deep the RPKI tree can be. */
 #defineMAX_CERT_DEPTH  12
 
+static X509_STORE_CTX  *ctx;
+static struct auth_tree  auths = RB_INITIALIZER();
+static struct crl_tree  crlt = RB_INITIALIZER();
+
 /*
  * Parse and validate a ROA.
  * This is standard stuff.
  * Returns the roa on success, NULL on failure.
  */
 static struct roa *
-proc_parser_roa(struct entity *entp, X509_STORE_CTX *ctx,
-struct auth_tree *auths, struct crl_tree *crlt)
+proc_parser_roa(struct entity *entp)
 {
struct roa  *roa;
X509*x509;
@@ -64,10 +67,10 @@ proc_parser_roa(struct entity *entp, X50
if ((roa = roa_parse(, entp->file)) == NULL)
return NULL;
 
-   a = valid_ski_aki(entp->file, auths, roa->ski, roa->aki);
+   a = valid_ski_aki(entp->file, , roa->ski, roa->aki);
 
build_chain(a, );
-   crl = get_crl(a, crlt);
+   crl = get_crl(a);
build_crls(crl, );
 
assert(x509 != NULL);
@@ -113,7 +116,7 @@ proc_parser_roa(struct entity *entp, X50
 * the code around roa_read() to check the "valid" field itself.
 */
 
-   if (valid_roa(entp->file, auths, roa))
+   if (valid_roa(entp->file, , roa))
roa->valid = 1;
 
sk_X509_free(chain);
@@ -134,8 +137,7 @@ proc_parser_roa(struct entity *entp, X50
  * Return the mft on success or NULL on failure.
  */
 static struct mft *
-proc_parser_mft(struct entity *entp, X509_STORE_CTX *ctx,
-   struct auth_tree *auths, struct crl_tree *crlt)
+proc_parser_mft(struct entity *entp)
 {
struct mft  *mft;
X509*x509;
@@ -146,7 +148,7 @@ proc_parser_mft(struct entity *entp, X50
if ((mft = mft_parse(, entp->file)) == NULL)
return NULL;
 
-   a = valid_ski_aki(entp->file, auths, mft->ski, mft->aki);
+   a = valid_ski_aki(entp->file, , mft->ski, mft->aki);
build_chain(a, );
 
if (!X509_STORE_CTX_init(ctx, NULL, x509, NULL))
@@ -187,8 +189,7 @@ proc_parser_mft(struct entity *entp, X50
  * parse failure.
  */
 static struct cert *
-proc_parser_cert(const struct entity *entp, X509_STORE_CTX *ctx,
-struct auth_tree *auths, struct crl_tree *crlt)
+proc_parser_cert(const struct entity *entp)
 {
struct cert *cert;
X509*x509;
@@ -205,9 +206,9 @@ proc_parser_cert(const struct entity *en
if (cert == NULL)
return NULL;
 
-   a = valid_ski_aki(entp->file, auths, cert->ski, cert->aki);
+   a = valid_ski_aki(entp->file, , cert->ski, cert->aki);
build_chain(a, );
-   build_crls(get_crl(a, crlt), );
+   build_crls(get_crl(a), );
 
assert(x509 != NULL);
if (!X509_STORE_CTX_init(ctx, NULL, x509, NULL))
@@ -236,7 +237,7 @@ proc_parser_cert(const struct entity *en
sk_X509_CRL_free(crls);
 
/* Validate the cert to get the parent */
-   if (!valid_cert(entp->file, auths, cert)) {
+   if (!valid_cert(entp->file, , cert)) {
X509_free(x509); // needed? XXX
return cert;
}
@@ -262,7 +263,7 @@ proc_parser_cert(const struct entity *en
if (na->fn == NULL)
err(1, NULL);
 
-   if (RB_INSERT(auth_tree, auths, na) != NULL)
+   if (RB_INSERT(auth_tree, , na) != NULL)
err(1, "auth tree corrupted");
 
return cert;
@@ -279,8 +280,7 @@ proc_parser_cert(const struct entity *en
  * parse failure.
  */
 static struct cert *
-proc_parser_root_cert(const struct entity *entp, X509_STORE_CTX *ctx,
-struct auth_tree *auths, struct crl_tree *crlt)
+proc_parser_root_cert(const struct entity *entp)
 {
charsubject[256];
ASN1_TIME   *notBefore, *notAfter;
@@ -327,7 +327,7 @@ proc_parser_root_cert(const struct entit
subject);
goto badcert;
}
-   if (!valid_ta(entp->file, auths, cert)) {
+   if (!valid_ta(entp->file, , cert)) {
warnx("%s: certificate not a valid ta, subject='%s'",
entp->file, subject);
goto badcert;
@@ -353,7 +353,7 @@ proc_parser_root_cert(const struct entit
if (na->fn == NULL)
   

Re: libagentx: Don't allow OIDs < 2

2021-10-24 Thread Claudio Jeker
On Sun, Oct 24, 2021 at 06:39:42PM +0100, Martijn van Duren wrote:
> libagentx currently allows OIDs with a length of 0.
> This isn't wrong from an agentx protocol point of view, but ber encoding
> can't handle OIDs with less then 2 elements, which makes it unable to
> map the values back to SNMP. netsnmp maps a null-oid to 0.0, but I don't
> think we should do that.
> 
> This diff also adds the check to a couple of other places where there
> was no active length checking prior. This is not an issue with current
> code using it (relayd(8)), because all OIDs are static, so are not
> susceptible for manipulation.
> 
> regress still passes.
> 
> OK?

Agreed that this is the right approach. agentx protocol spec was sloppy
in this regard. NULL OIDs only make sense for some messages but not for
others.

Diff looks OK.
 
> Index: agentx.h
> ===
> RCS file: /cvs/src/lib/libagentx/agentx.h,v
> retrieving revision 1.5
> diff -u -p -r1.5 agentx.h
> --- agentx.h  27 Oct 2020 18:24:01 -  1.5
> +++ agentx.h  24 Oct 2021 17:39:16 -
> @@ -36,6 +36,7 @@ enum agentx_request_type {
>  };
>  
>  #define AGENTX_MASTER_PATH "/var/agentx/master"
> +#define AGENTX_OID_MIN_LEN 2
>  #define AGENTX_OID_MAX_LEN 128
>  #define AGENTX_OID_INDEX_MAX_LEN 10
>  #define AGENTX_MIB2 1, 3, 6, 1, 2, 1
> Index: agentx.c
> ===
> RCS file: /cvs/src/lib/libagentx/agentx.c,v
> retrieving revision 1.13
> diff -u -p -r1.13 agentx.c
> --- agentx.c  23 Oct 2021 17:13:50 -  1.13
> +++ agentx.c  24 Oct 2021 17:39:17 -
> @@ -189,6 +189,8 @@ static int agentx_request(struct agentx 
>  static int agentx_request_cmp(struct agentx_request *,
>  struct agentx_request *);
>  static int agentx_strcat(char **, const char *);
> +static int agentx_oidfill(struct ax_oid *, const uint32_t[], size_t,
> +const char **);
>  
>  RB_PROTOTYPE_STATIC(ax_requests, agentx_request, axr_ax_requests,
>  agentx_request_cmp)
> @@ -362,25 +364,26 @@ agentx_session(struct agentx *ax, uint32
>  size_t oidlen, const char *descr, uint8_t timeout)
>  {
>   struct agentx_session *axs;
> - size_t i;
> + const char *errstr;
>  
> - if (oidlen > AGENTX_OID_MAX_LEN) {
> -#ifdef AX_DEBUG
> - agentx_log_ax_fatalx(ax, "%s: oidlen > %d", __func__,
> - AGENTX_OID_MAX_LEN);
> -#else
> - errno = EINVAL;
> - return NULL;
> -#endif
> - }
>   if ((axs = calloc(1, sizeof(*axs))) == NULL)
>   return NULL;
>  
>   axs->axs_ax = ax;
>   axs->axs_timeout = timeout;
> - for (i = 0; i < oidlen; i++)
> - axs->axs_oid.aoi_id[i] = oid[i];
> - axs->axs_oid.aoi_idlen = oidlen;
> + /* RFC 2741 section 6.2.1: may send a null Object Identifier */
> + if (oidlen == 0)
> + axs->axs_oid.aoi_idlen = oidlen;
> + else {
> + if (agentx_oidfill((>axs_oid), oid, oidlen,
> + ) == -1) {
> +#ifdef AX_DEBUG
> + agentx_log_ax_fatalx(ax, "%s: %s", __func__, errstr);
> +#else
> + return NULL;
> +#endif
> + }
> + }
>   axs->axs_descr.aos_string = (unsigned char *)strdup(descr);
>   if (axs->axs_descr.aos_string == NULL) {
>   free(axs);
> @@ -670,11 +673,21 @@ agentx_context_object_find(struct agentx
>  const uint32_t oid[], size_t oidlen, int active, int instance)
>  {
>   struct agentx_object *axo, axo_search;
> - size_t i;
> + const char *errstr;
>  
> - for (i = 0; i < oidlen; i++)
> - axo_search.axo_oid.aoi_id[i] = oid[i];
> - axo_search.axo_oid.aoi_idlen = oidlen;
> + if (agentx_oidfill(&(axo_search.axo_oid), oid, oidlen, ) == -1) {
> + if (oidlen > AGENTX_OID_MIN_LEN) {
> +#ifdef AX_DEBUG
> + agentx_log_axc_fatalx(axc, "%s: %s", __func__, errstr);
> +#else
> + agentx_log_axc_warnx(axc, "%s: %s", __func__, errstr);
> + return NULL;
> + }
> +#endif
> + if (oidlen == 1)
> + axo_search.axo_oid.aoi_id[0] = oid[0];
> + axo_search.axo_oid.aoi_idlen = oidlen;
> + }
>  
>   axo = RB_FIND(axc_objects, &(axc->axc_objects), _search);
>   while (axo == NULL && !instance && axo_search.axo_oid.aoi_idlen > 0) {
> @@ -691,11 +704,21 @@ agentx_context_object_nfind(struct agent
>  const uint32_t oid[], size_t oidlen, int active, int inclusive)
>  {
>   struct agentx_object *axo, axo_search;
> - size_t i;
> + const char *errstr;
>  
> - for (i = 0; i < oidlen; i++)
> - axo_search.axo_oid.aoi_id[i] = oid[i];
> - axo_search.axo_oid.aoi_idlen = oidlen;
> + if (agentx_oidfill(&(axo_search.axo_oid), oid, oidlen, ) == -1) {
> + if (oidlen > AGENTX_OID_MIN_LEN) {
> +#ifdef AX_DEBUG
> + 

Re: libagentx: always initialize buf in ax_oidrange2string

2021-10-24 Thread Claudio Jeker
On Sun, Oct 24, 2021 at 06:31:29PM +0100, Martijn van Duren wrote:
> This diff should be superfluous with the next diff, but I don't think
> this should be left as is anyway.
> 
> It's not a big problem, since it's a static buffer and it gets
> initialized by previous calls, so it's always NUL-terminated, but
> it's not accurate.
> 
> OK?

OK claudio@
 
> martijn@
> 
> Index: ax.c
> ===
> RCS file: /cvs/src/lib/libagentx/ax.c,v
> retrieving revision 1.7
> diff -u -p -r1.7 ax.c
> --- ax.c  2 Jan 2021 01:06:31 -   1.7
> +++ ax.c  24 Oct 2021 17:31:03 -
> @@ -764,6 +764,8 @@ ax_oidrange2string(struct ax_oid *oid, u
>  
>   rest = sizeof(buf);
>   p = buf;
> + if (oid->aoi_idlen == 0)
> + (void)strlcpy(buf, "null", sizeof(buf));
>   for (i = 0; i < oid->aoi_idlen; i++) {
>   if (range_subid != 0 && range_subid - 1 == (uint8_t)i)
>   ret = snprintf(p, rest, ".[%u-%u]", oid->aoi_id[i],
> 
> 

-- 
:wq Claudio



rpki-client add regress print functions to main repo

2021-10-24 Thread Claudio Jeker
This diff adds the print functions from the regress test to rpki-client.
Currently not hooked up in rpki-client itself but that will follow soon.

-- 
:wq Claudio

Index: usr.sbin/rpki-client/extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.76
diff -u -p -r1.76 extern.h
--- usr.sbin/rpki-client/extern.h   24 Oct 2021 12:06:16 -  1.76
+++ usr.sbin/rpki-client/extern.h   24 Oct 2021 17:37:20 -
@@ -558,6 +558,13 @@ char   *x509_crl_get_aki(X509_CRL *, cons
 char   *x509_get_pubkey(X509 *, const char *);
 enum cert_purpose   x509_get_purpose(X509 *, const char *);
 
+/* printers */
+void   tal_print(const struct tal *);
+void   cert_print(const struct cert *);
+void   mft_print(const struct mft *);
+void   roa_print(const struct roa *);
+void   gbr_print(const struct gbr *);
+
 /* Output! */
 
 extern int  outformats;
Index: usr.sbin/rpki-client/print.c
===
RCS file: usr.sbin/rpki-client/print.c
diff -N usr.sbin/rpki-client/print.c
--- /dev/null   1 Jan 1970 00:00:00 -
+++ usr.sbin/rpki-client/print.c24 Oct 2021 17:30:47 -
@@ -0,0 +1,169 @@
+/* $OpenBSD$ */
+/*
+ * Copyright (c) 2021 Claudio Jeker 
+ * Copyright (c) 2019 Kristaps Dzonsons 
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+
+#include "extern.h"
+
+static const char *
+pretty_key_id(char *hex)
+{
+   static char buf[128];   /* bigger than SHA_DIGEST_LENGTH * 3 */
+   size_t i;
+
+   for (i = 0; i < sizeof(buf) && *hex != '\0'; i++) {
+   if  (i % 3 == 2 && *hex != '\0')
+   buf[i] = ':';
+   else
+   buf[i] = *hex++;
+   }
+   if (i == sizeof(buf))
+   memcpy(buf + sizeof(buf) - 4, "...", 4);
+   return buf;
+}
+
+void
+tal_print(const struct tal *p)
+{
+   size_t   i;
+
+   for (i = 0; i < p->urisz; i++)
+   printf("%5zu: URI: %s\n", i + 1, p->uri[i]);
+}
+
+void
+cert_print(const struct cert *p)
+{
+   size_t   i;
+   char buf1[64], buf2[64];
+   int  sockt;
+   char tbuf[21];
+
+   printf("Subject key identifier: %s\n", pretty_key_id(p->ski));
+   if (p->aki != NULL)
+   printf("Authority key identifier: %s\n", pretty_key_id(p->aki));
+   if (p->aia != NULL)
+   printf("Authority info access: %s\n", p->aia);
+   if (p->mft != NULL)
+   printf("Manifest: %s\n", p->mft);
+   if (p->repo != NULL)
+   printf("caRepository: %s\n", p->repo);
+   if (p->notify != NULL)
+   printf("Notify URL: %s\n", p->notify);
+   if (p->pubkey != NULL)
+   printf("BGPsec P-256 ECDSA public key: %s\n", p->pubkey);
+   strftime(tbuf, sizeof(tbuf), "%FT%TZ", gmtime(>expires));
+   printf("Valid until: %s\n", tbuf);
+
+   printf("Subordinate Resources:\n");
+
+   for (i = 0; i < p->asz; i++)
+   switch (p->as[i].type) {
+   case CERT_AS_ID:
+   printf("%5zu: AS: %u\n", i + 1, p->as[i].id);
+   break;
+   case CERT_AS_INHERIT:
+   printf("%5zu: AS: inherit\n", i + 1);
+   break;
+   case CERT_AS_RANGE:
+   printf("%5zu: AS: %u -- %u\n", i + 1,
+   p->as[i].range.min, p->as[i].range.max);
+   break;
+   }
+
+   for (i = 0; i < p->ipsz; i++)
+   switch (p->ips[i].type) {
+   case CERT_IP_INHERIT:
+   printf("%5zu: IP: inherit\n", i + 1);
+   break;
+   case CERT_IP_ADDR:
+   ip_ad

Re: simplify sys___thrsigdivert a bit

2021-10-23 Thread Claudio Jeker
On Sat, Oct 23, 2021 at 05:47:58PM +0200, Mark Kettenis wrote:
> > Date: Sat, 23 Oct 2021 17:29:36 +0200
> > From: Claudio Jeker 
> > 
> > The sys___thrsigdivert code can be simplified a bit. It is possible to
> > set the error before the loop and then have the loop exit after polling
> > for pending signals. IMO the results looks nicer than what we have now.
> > 
> > OK?
> 
> That does not look right:
> 
>   The function sigtimedwait() behaves the same as sigwaitinfo() except
>   that if none of the signals specified by set are pending,
>   sigtimedwait() waits for the time interval specified in the timespec
>   structure referenced by timeout. If the timespec structure pointed
>   to by timeout is zero-valued and if none of the signals specified by
>   set are pending, then sigtimedwait() returns immediately with an
>   error. If timeout is the NULL pointer, the behaviour is unspecified.
> 
> So you need to check for signals first.

The code does the check first. It just sets the error before doing the
pending signal check. If a signal is pending the error is reset to 0 and
the function returns. Else it will break out of the loop with the error
and not hit the tsleep.
 
> The current code looks fine to me.
> 
> 
> > Index: kern/kern_sig.c
> > ===
> > RCS file: /cvs/src/sys/kern/kern_sig.c,v
> > retrieving revision 1.286
> > diff -u -p -r1.286 kern_sig.c
> > --- kern/kern_sig.c 23 Oct 2021 14:56:55 -  1.286
> > +++ kern/kern_sig.c 23 Oct 2021 15:15:21 -
> > @@ -1761,7 +1761,6 @@ sys___thrsigdivert(struct proc *p, void 
> > sigset_t mask = SCARG(uap, sigmask) &~ sigcantmask;
> > siginfo_t si;
> > uint64_t nsecs = INFSLP;
> > -   int timeinvalid = 0;
> > int error = 0;
> >  
> > memset(, 0, sizeof(si));
> > @@ -1774,12 +1773,21 @@ sys___thrsigdivert(struct proc *p, void 
> > if (KTRPOINT(p, KTR_STRUCT))
> > ktrreltimespec(p, );
> >  #endif
> > -   if (!timespecisvalid())
> > -   timeinvalid = 1;
> > -   else
> > +   if (timespecisvalid()) {
> > nsecs = TIMESPEC_TO_NSEC();
> > +   } else {
> > +   /*
> > +* per-POSIX, delay this error until
> > +* after checking for signals
> > +*/
> > +   error = EINVAL;
> > +   }
> > }
> >  
> > +   /* per-POSIX, return immediatly if timeout is zero-valued */
> > +   if (nsecs == 0)
> > +   error = EAGAIN;
> > +
> > dosigsuspend(p, p->p_sigmask &~ mask);
> > for (;;) {
> > si.si_signo = cursig(p);
> > @@ -1791,13 +1799,6 @@ sys___thrsigdivert(struct proc *p, void 
> > break;
> > }
> > }
> > -
> > -   /* per-POSIX, delay this error until after the above */
> > -   if (timeinvalid)
> > -   error = EINVAL;
> > -   /* per-POSIX, return immediatly if timeout is zero-valued */
> > -   if (nsecs == 0)
> > -   error = EAGAIN;
> >  
> > if (error != 0)
> > break;
> > 
> > 

-- 
:wq Claudio



simplify sys___thrsigdivert a bit

2021-10-23 Thread Claudio Jeker
The sys___thrsigdivert code can be simplified a bit. It is possible to
set the error before the loop and then have the loop exit after polling
for pending signals. IMO the results looks nicer than what we have now.

OK?
-- 
:wq Claudio

Index: kern/kern_sig.c
===
RCS file: /cvs/src/sys/kern/kern_sig.c,v
retrieving revision 1.286
diff -u -p -r1.286 kern_sig.c
--- kern/kern_sig.c 23 Oct 2021 14:56:55 -  1.286
+++ kern/kern_sig.c 23 Oct 2021 15:15:21 -
@@ -1761,7 +1761,6 @@ sys___thrsigdivert(struct proc *p, void 
sigset_t mask = SCARG(uap, sigmask) &~ sigcantmask;
siginfo_t si;
uint64_t nsecs = INFSLP;
-   int timeinvalid = 0;
int error = 0;
 
memset(, 0, sizeof(si));
@@ -1774,12 +1773,21 @@ sys___thrsigdivert(struct proc *p, void 
if (KTRPOINT(p, KTR_STRUCT))
ktrreltimespec(p, );
 #endif
-   if (!timespecisvalid())
-   timeinvalid = 1;
-   else
+   if (timespecisvalid()) {
nsecs = TIMESPEC_TO_NSEC();
+   } else {
+   /*
+* per-POSIX, delay this error until
+* after checking for signals
+*/
+   error = EINVAL;
+   }
}
 
+   /* per-POSIX, return immediatly if timeout is zero-valued */
+   if (nsecs == 0)
+   error = EAGAIN;
+
dosigsuspend(p, p->p_sigmask &~ mask);
for (;;) {
si.si_signo = cursig(p);
@@ -1791,13 +1799,6 @@ sys___thrsigdivert(struct proc *p, void 
break;
}
}
-
-   /* per-POSIX, delay this error until after the above */
-   if (timeinvalid)
-   error = EINVAL;
-   /* per-POSIX, return immediatly if timeout is zero-valued */
-   if (nsecs == 0)
-   error = EAGAIN;
 
if (error != 0)
break;



fix IO handling in rpki-client

2021-10-23 Thread Claudio Jeker
This diff changes the io read functions to work on ibufs.
With this the poll loops will consume data with io_buf_read() until a full
message is received and then that message is processed. Thanks to this
the processes no longer block while waiting for more data in the io read
functions.

With this also the blocking/nonblocking dance done in a few places is no
longer needed. Everything is now nonblocking.

Further cleanup of the various marshaling functions can follow in a later
step.
-- 
:wq Claudio

Index: cert.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/cert.c,v
retrieving revision 1.39
diff -u -p -r1.39 cert.c
--- cert.c  15 Oct 2021 22:30:33 -  1.39
+++ cert.c  23 Oct 2021 11:58:43 -
@@ -1281,33 +1281,31 @@ cert_buffer(struct ibuf *b, const struct
 }
 
 static void
-cert_ip_read(int fd, struct cert_ip *p)
+cert_ip_read(struct ibuf *b, struct cert_ip *p)
 {
-
-   io_simple_read(fd, >afi, sizeof(enum afi));
-   io_simple_read(fd, >type, sizeof(enum cert_ip_type));
+   io_read_buf(b, >afi, sizeof(enum afi));
+   io_read_buf(b, >type, sizeof(enum cert_ip_type));
 
if (p->type != CERT_IP_INHERIT) {
-   io_simple_read(fd, >min, sizeof(p->min));
-   io_simple_read(fd, >max, sizeof(p->max));
+   io_read_buf(b, >min, sizeof(p->min));
+   io_read_buf(b, >max, sizeof(p->max));
}
 
if (p->type == CERT_IP_RANGE)
-   ip_addr_range_read(fd, >range);
+   ip_addr_range_read(b, >range);
else if (p->type == CERT_IP_ADDR)
-   ip_addr_read(fd, >ip);
+   ip_addr_read(b, >ip);
 }
 
 static void
-cert_as_read(int fd, struct cert_as *p)
+cert_as_read(struct ibuf *b, struct cert_as *p)
 {
-
-   io_simple_read(fd, >type, sizeof(enum cert_as_type));
+   io_read_buf(b, >type, sizeof(enum cert_as_type));
if (p->type == CERT_AS_RANGE) {
-   io_simple_read(fd, >range.min, sizeof(uint32_t));
-   io_simple_read(fd, >range.max, sizeof(uint32_t));
+   io_read_buf(b, >range.min, sizeof(uint32_t));
+   io_read_buf(b, >range.max, sizeof(uint32_t));
} else if (p->type == CERT_AS_ID)
-   io_simple_read(fd, >id, sizeof(uint32_t));
+   io_read_buf(b, >id, sizeof(uint32_t));
 }
 
 /*
@@ -1316,7 +1314,7 @@ cert_as_read(int fd, struct cert_as *p)
  * Always returns a valid pointer.
  */
 struct cert *
-cert_read(int fd)
+cert_read(struct ibuf *b)
 {
struct cert *p;
size_t   i;
@@ -1324,35 +1322,36 @@ cert_read(int fd)
if ((p = calloc(1, sizeof(struct cert))) == NULL)
err(1, NULL);
 
-   io_simple_read(fd, >valid, sizeof(int));
-   io_simple_read(fd, >expires, sizeof(time_t));
-   io_simple_read(fd, >purpose, sizeof(enum cert_purpose));
-   io_simple_read(fd, >ipsz, sizeof(size_t));
+   io_read_buf(b, >valid, sizeof(int));
+   io_read_buf(b, >expires, sizeof(time_t));
+   io_read_buf(b, >purpose, sizeof(enum cert_purpose));
+   io_read_buf(b, >ipsz, sizeof(size_t));
+
p->ips = calloc(p->ipsz, sizeof(struct cert_ip));
if (p->ips == NULL)
err(1, NULL);
for (i = 0; i < p->ipsz; i++)
-   cert_ip_read(fd, >ips[i]);
+   cert_ip_read(b, >ips[i]);
 
-   io_simple_read(fd, >asz, sizeof(size_t));
+   io_read_buf(b, >asz, sizeof(size_t));
p->as = calloc(p->asz, sizeof(struct cert_as));
if (p->as == NULL)
err(1, NULL);
for (i = 0; i < p->asz; i++)
-   cert_as_read(fd, >as[i]);
+   cert_as_read(b, >as[i]);
+
+   io_read_str(b, >mft);
+   io_read_str(b, >notify);
+   io_read_str(b, >repo);
+   io_read_str(b, >crl);
+   io_read_str(b, >aia);
+   io_read_str(b, >aki);
+   io_read_str(b, >ski);
+   io_read_str(b, >tal);
+   io_read_str(b, >pubkey);
 
-   io_str_read(fd, >mft);
assert(p->mft != NULL || p->purpose == CERT_PURPOSE_BGPSEC_ROUTER);
-   io_str_read(fd, >notify);
-   io_str_read(fd, >repo);
-   io_str_read(fd, >crl);
-   io_str_read(fd, >aia);
-   io_str_read(fd, >aki);
-   io_str_read(fd, >ski);
assert(p->ski);
-   io_str_read(fd, >tal);
-   io_str_read(fd, >pubkey);
-
return p;
 }
 
Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.73
diff -u -p -r1.73 extern.h
--- extern.h22 Oct 2021 11:13:06 -  1.73
+++ extern.h23 Oct 2021 12:03:19 -
@@ -399,25 +399,25 @@ void   tal_buffer(struct ibuf *, const s
 voidtal_free(struct tal *);
 struct tal *tal_parse(const char *, char *);
 char   *tal_read_file(const char *);
-struct tal *tal_read(int);
+struct tal 

Re: ixl(4): add checksum receive offloading

2021-10-22 Thread Claudio Jeker
On Fri, Oct 22, 2021 at 04:45:09PM +0200, Hrvoje Popovski wrote:
> On 22.10.2021. 16:09, Florian Obser wrote:
> > 
> > 
> > On 22 October 2021 13:55:20 CEST, Stuart Henderson  
> > wrote:
> >> On 2021/10/22 11:25, Jan Klemkow wrote:
> >>> this diff add hardware checksum offloading for the receive path of
> >>> ixl(4) interfaces.
> >>
> >> Would be good to have this tested with NFS if anyone has a way to do so.
> >> nics are probably better now but I'm pretty sure we have had problems
> >> with NFS and offloading in the past.
> > 
> > ospf as well.
> > 
> 
> ospf seems to work
> 
> 
> smc24# ospfctl show nei
> ID  Pri StateDeadTime Address Iface Uptime
> 10.1.1.11   FULL/BCKUP   00:00:30 192.168.15.123  ixl0  00:01:20
> 
> 
> 
> 10.11.0.0/16 192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 10.12.0.0/16 192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 10.13.0.0/16 192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 10.14.0.0/16 192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 10.15.0.0/16 192.168.15.11 Type 1 ext   Network   110
> 00:00:13
> 10.16.0.0/16 192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 10.17.0.0/16 192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 10.18.0.0/16 192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 
> 192.168.11.0/24  192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 192.168.12.0/24  192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 192.168.13.0/24  192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 192.168.14.0/24  192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 192.168.16.0/24  192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 192.168.17.0/24  192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 192.168.18.0/24  192.168.15.1  Type 1 ext   Network   110
> 00:00:13
> 

For ospfd tests you want to make sure that some of the ospf packets need
fragmenting. So this needs a sizeable network to hit this.

-- 
:wq Claudio



fix symlink read in openrsync

2021-10-22 Thread Claudio Jeker
flist_gen_dirent() does a fts_read and inside that tries to read the
symlink information. Now since fts_open did not specifiy FTS_NOCHDIR
the symlink_read call needs to use ent->fts_accpath instead of f->path
which was based on ent->fts_path.

OK?
-- 
:wq Claudio

Index: flist.c
===
RCS file: /cvs/src/usr.bin/rsync/flist.c,v
retrieving revision 1.34
diff -u -p -r1.34 flist.c
--- flist.c 2 Sep 2021 21:06:06 -   1.34
+++ flist.c 3 Sep 2021 13:09:13 -
@@ -992,7 +992,7 @@ flist_gen_dirent(struct sess *sess, char
/* Optionally copy link information. */
 
if (S_ISLNK(ent->fts_statp->st_mode)) {
-   f->link = symlink_read(f->path);
+   f->link = symlink_read(ent->fts_accpath);
if (f->link == NULL) {
ERRX1("symlink_read");
goto out;



Re: retire hifn safe ubsec

2021-10-21 Thread Claudio Jeker
On Thu, Oct 21, 2021 at 04:30:02PM +0200, Alexander Bluhm wrote:
> Hi,
> 
> Goal is to retire the async crypto API.  It is slow and adds
> complexity which hinders MP progress in IPsec.  It is used by the
> old PCI devices hifn(4), safe(4), and ubsec(4).
> 
> These devices are not common anymore.  Using the CPU for crypto is
> faster than offloading via the PCI bus.  By having special requirements
> for the crypto API, those devices slow down modern machines.  They
> only support crypto algorithms that are insecure nowadays.
> 
> ok to remove hifn(4) safe(4) ubsec(4) ?
> 

Agreed, I see little benefit in having them around now.

-- 
:wq Claudio



normalize ldap DN in the config

2021-10-06 Thread Claudio Jeker
Run into this while setting up a new DN.
The DN in namespace only matches if it is normalized.
So it may be best to do this by default when adding a namespace.
With this using a capitalized namespace like "o=OpenBSD,c=CA" will
work. Also as a side note the rootdn is already normalized so no need
to fix that.

Totally unrelated but I could not resist to change the error check for
host from the somewhat funky <= 0 to a != 1 (host returns 1 on success and
0 and -1 on failure).

I will split this into two commits in the end.
-- 
:wq Claudio

Index: parse.y
===
RCS file: /cvs/src/usr.sbin/ldapd/parse.y,v
retrieving revision 1.40
diff -u -p -r1.40 parse.y
--- parse.y 2 May 2021 14:39:05 -   1.40
+++ parse.y 6 Oct 2021 16:56:50 -
@@ -207,7 +207,7 @@ conf_main   : LISTEN ON STRING port ssl ce
if (! interface($3, cert, >listeners,
$4, $5)) {
if (host($3, cert, >listeners,
-   $4, $5) <= 0) {
+   $4, $5) != 1) {
yyerror("invalid virtual ip or 
interface: %s", $3);
free($6);
free($3);
@@ -1206,15 +1206,16 @@ namespace_new(const char *suffix)
 
if ((ns = calloc(1, sizeof(*ns))) == NULL)
return NULL;
-   ns->suffix = strdup(suffix);
ns->sync = 1;
ns->cache_size = 1024;
ns->index_cache_size = 512;
+   ns->suffix = strdup(suffix);
if (ns->suffix == NULL) {
free(ns->suffix);
free(ns);
return NULL;
}
+   normalize_dn(ns->suffix);
TAILQ_INIT(>indices);
TAILQ_INIT(>request_queue);
SIMPLEQ_INIT(>acl);



Re: sigwaitinfo(2) and sigtimedwait(2)

2021-09-28 Thread Claudio Jeker
On Sun, Sep 26, 2021 at 02:36:02PM +0200, Mark Kettenis wrote:
> > Date: Fri, 24 Sep 2021 19:36:21 +0200
> > From: Rafael Sadowski 
> > 
> > I'm trying to port the more KDE stuff so my question is from porter
> > perspective.
> > 
> > I need sigwaitinfo(2)/sigtimedwait(2) and I found both functions in
> > lib/libc/gen/sigwait.c with the comment "need kernel to fill in more
> > siginfo_t bits first". Is the comment still up to date? If no, is it
> > possible to unlock the functions?
> 
> Still true.  These functions are somewhat underspecified by POSIX so
> it isn't really obvious whatadditional bits need to be filled in.
> Having examples of code that use these interfaces from ports could
> help with that.

Especially if the siginfo SIGCHLD bits are required. This will require
major work in the kernel since we don't have a method in place to
transport the exit information from the exiting process to the parent
signal handler.

-- 
:wq Claudio



OpenBGPD 7.2 released

2021-09-23 Thread Claudio Jeker
We have released OpenBGPD 7.2, which will be arriving in the
OpenBGPD directory of your local OpenBSD mirror soon.

This release includes the following changes to the previous release:

* Support for RFC 9072 - Extended Optional Parameters Lenght for
  BGP OPEN Message

* Support for RFC 8050 - MRT Format with BGP Additional Path Extensions

* Implement receive side of RFC 7911 - Advertisement of Multiple Paths
  in BGP. OpenBGPD is currently not able to send multiple paths out.

* Improve checks of VRPs loaded via RTR or from the roa-set table.

* Allow to optionally specify an expiry time for roa-set entries to
  mitigate BGP route decision making based on outdated RPKI data.
  OpenBGPD's companion rpki-client(8) produces roa-sets with the
  new 'expires' property

OpenBGPD-portable is known to compile and run on FreeBSD, and
the Linux distributions Alpine, Debian, Fedora, RHEL/CentOS and Ubuntu.
It is our hope that packagers take interest and help adapt OpenBGPD-portable
to more distributions.

We welcome feedback and improvements from the broader community.
Thanks to all of the contributors who helped make this release
possible.



bgpctl mrtdump adjustment

2021-09-22 Thread Claudio Jeker
In bgpd we do not follow the RFC8050 encoding for RIB_GENERIC_ADDPATH.
Mainly because it does not fit the way the code works and also because the
only other BGP implementation that seems to care about RIB_GENERIC_ADDPATH
does it the same way.

Because of this it makes no sense to parse RIB_GENERIC_ADDPATH strictly
per RFC8050 but instead parse it the way it is implemented.

While there cleanup some unused variables.
-- 
:wq Claudio

Index: mrtparser.c
===
RCS file: /cvs/src/usr.sbin/bgpctl/mrtparser.c,v
retrieving revision 1.15
diff -u -p -r1.15 mrtparser.c
--- mrtparser.c 27 Jul 2021 07:42:37 -  1.15
+++ mrtparser.c 22 Sep 2021 10:22:54 -
@@ -103,10 +103,8 @@ mrt_parse(int fd, struct mrt_parser *p, 
struct mrt_bgp_state*s;
struct mrt_bgp_msg  *m;
void*msg;
-   int  addpath;
 
while ((msg = mrt_read_msg(fd, ))) {
-   addpath = 0;
switch (ntohs(h.type)) {
case MSG_NULL:
case MSG_START:
@@ -376,7 +374,7 @@ mrt_parse_v2_rib(struct mrt_hdr *hdr, vo
u_int32_t   snum, path_id = 0;
u_int16_t   cnt, i, afi;
u_int8_tsafi, aid;
-   int ret, addpath = 0;
+   int ret;
 
if (len < sizeof(snum) + 1)
return NULL;
@@ -417,6 +415,11 @@ mrt_parse_v2_rib(struct mrt_hdr *hdr, vo
goto fail;
break;
case MRT_DUMP_V2_RIB_GENERIC_ADDPATH:
+   /*
+* RFC8050 handling for add-path has special handling for
+* RIB_GENERIC_ADDPATH but nobody implements it that way.
+* So just use the same way as for the other _ADDPATH types.
+*/
r->add_path = 1;
/* FALLTHROUGH */
case MRT_DUMP_V2_RIB_GENERIC:
@@ -434,16 +437,6 @@ mrt_parse_v2_rib(struct mrt_hdr *hdr, vo
if ((aid = mrt_afi2aid(afi, safi, verbose)) == AID_UNSPEC)
goto fail;

-   /* RFC8050 handling for add-path */
-   if (r->add_path) {
-   if (len < sizeof(path_id))
-   goto fail;
-   memcpy(_id, b, sizeof(path_id));
-   b += sizeof(path_id);
-   len -= sizeof(path_id);
-   path_id = ntohl(path_id);
-   }
-
/* prefix */
ret = mrt_extract_prefix(b, len, aid, >prefix,
>prefixlen, verbose);
@@ -487,12 +480,9 @@ mrt_parse_v2_rib(struct mrt_hdr *hdr, vo
len -= sizeof(otm);
entries[i].originated = ntohl(otm);
 
-   /* RFC8050 handling for add-path */
-   if (r->add_path &&
-   ntohs(hdr->subtype) != MRT_DUMP_V2_RIB_GENERIC_ADDPATH) {
+   if (r->add_path) {
if (len < sizeof(path_id) + sizeof(alen))
goto fail;
-   addpath = 0;
memcpy(_id, b, sizeof(path_id));
b += sizeof(path_id);
len -= sizeof(path_id);



Re: tls_pending(3)

2021-09-16 Thread Claudio Jeker
On Thu, Sep 16, 2021 at 11:00:31PM +0200, Kristaps Dzonsons wrote:
> Hi,
> 
> I'm porting a nonblocking, polling OpenSSL system to libtls.  However, I'm
> not sure how this is non-hackily possible without SSL_pending(3) to detect
> if less data is read with tls_read() than is buffered.
> 
>  writer:
>   tls_write(40)
> 
>  reader:
>   poll(POLLIN, INFTIM) -> POLLIN /* descriptor has a read */
>   tls_read(20) /* 20 bytes read, 20 bytes remain */
>   poll(POLLIN, INFTIM) -> 0 /* data was buffered */
> 
> This introduces tls_pending(3), which calls through to SSL_pending(3), and
> includes a simple example in tls_read(3).
> 
> Beck's tutorial says that "libtls is designed to do the common things you do
> for making TLS connections easy", and I'm not sure my use case meets that
> standard.  If it looks reasonable, however, I can also add the regression
> tests.

In general you should call tls_read() until it retruns TLS_WANT_POLL*.
Then you poll again. So you need to adapt the poll loop a bit but in most
cases this is not hard to do.
 
> Thank you,
> 
> Kristaps

> Index: tls.c
> ===
> RCS file: /cvs/src/lib/libtls/tls.c,v
> retrieving revision 1.89
> diff -u -p -r1.89 tls.c
> --- tls.c 1 Feb 2021 15:35:41 -   1.89
> +++ tls.c 16 Sep 2021 20:57:32 -
> @@ -788,6 +788,16 @@ tls_handshake(struct tls *ctx)
>   return (rv);
>  }
>  
> +size_t
> +tls_pending(const struct tls *ctx)
> +{
> + 
> + if ((ctx->state & TLS_HANDSHAKE_COMPLETE) != 0)
> + return 0;
> +
> + return (size_t)SSL_pending(ctx->ssl_conn);
> +}
> +
>  ssize_t
>  tls_read(struct tls *ctx, void *buf, size_t buflen)
>  {
> Index: tls.h
> ===
> RCS file: /cvs/src/lib/libtls/tls.h,v
> retrieving revision 1.58
> diff -u -p -r1.58 tls.h
> --- tls.h 22 Jan 2020 06:44:02 -  1.58
> +++ tls.h 16 Sep 2021 20:57:32 -
> @@ -177,6 +177,7 @@ int tls_connect_socket(struct tls *_ctx,
>  int tls_connect_cbs(struct tls *_ctx, tls_read_cb _read_cb,
>  tls_write_cb _write_cb, void *_cb_arg, const char *_servername);
>  int tls_handshake(struct tls *_ctx);
> +size_t tls_pending(const struct tls *_ctx);
>  ssize_t tls_read(struct tls *_ctx, void *_buf, size_t _buflen);
>  ssize_t tls_write(struct tls *_ctx, const void *_buf, size_t _buflen);
>  int tls_close(struct tls *_ctx);
> Index: man/tls_read.3
> ===
> RCS file: /cvs/src/lib/libtls/man/tls_read.3,v
> retrieving revision 1.7
> diff -u -p -r1.7 tls_read.3
> --- man/tls_read.39 Jul 2019 17:58:33 -   1.7
> +++ man/tls_read.316 Sep 2021 20:57:32 -
> @@ -27,6 +27,7 @@
>  .Nm tls_handshake ,
>  .Nm tls_error ,
>  .Nm tls_close ,
> +.Nm tls_pending ,
>  .Nm tls_reset
>  .Nd use a TLS connection
>  .Sh SYNOPSIS
> @@ -51,6 +52,8 @@
>  .Fn tls_close "struct tls *ctx"
>  .Ft void
>  .Fn tls_reset "struct tls *ctx"
> +.Ft size_t
> +.Fn tls_pending "const struct tls *ctx"
>  .Sh DESCRIPTION
>  .Fn tls_read
>  reads
> @@ -58,6 +61,8 @@ reads
>  bytes of data from the socket into
>  .Fa buf .
>  It returns the amount of data read.
> +.Fn tls_pending
> +returns the number of bytes that may be read without blocking.
>  .Pp
>  .Fn tls_write
>  writes
> @@ -99,6 +104,9 @@ and
>  .Fn tls_write
>  return a size on success or -1 on error.
>  .Pp
> +.Fn tls_pending
> +returns a size or zero if no data is buffered for immediate reading.
> +.Pp
>  .Fn tls_handshake
>  and
>  .Fn tls_close
> @@ -198,6 +206,28 @@ while (len > 0) {
>   buf += ret;
>   len -= ret;
>   }
> + }
> +}
> +\&...
> +.Ed
> +.Pp
> +For non-blocking input, the following example demonstrates how to handle
> +buffered data when the descriptor may have none to read:
> +.Bd -literal -offset indent
> +\&...
> +pfd[0].fd = fd;
> +pfd[0].events = POLLIN;
> +for (;;) {
> + int timeo;
> +
> + timeo = tls_pending(tls) ? 0 : INFTIM;
> + if (poll(pfd, 1, timeo) == -1)
> + err(1, "poll");
> + if ((pfd[0].revents & POLLIN) || tls_pending(tls)) {
> + ssize_t ret;
> +
> + ret = tls_read(ctx, buf, len);
> + \&...
>   }
>  }
>  \&...


-- 
:wq Claudio



rpki-client RRDP delta vs snapshot sync

2021-09-14 Thread Claudio Jeker
I'm tired waiting 30min and more for apnic and idnic to finish their delta
syncs with 1000+ deltas to fetch from a server on the other side of the
planet. If a repo is more than 300 deltas behind just grab the snapshot,
it is way faster in the end.

The number 300 was selected to be not too low to allow repos to mostly use
the delta downloads and save bandwidth.

-- 
:wq Claudio

Index: rrdp_notification.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rrdp_notification.c,v
retrieving revision 1.6
diff -u -p -r1.6 rrdp_notification.c
--- rrdp_notification.c 12 Aug 2021 15:23:16 -  1.6
+++ rrdp_notification.c 10 Sep 2021 13:16:23 -
@@ -371,6 +371,10 @@ notification_done(struct notification_xm
return NOTIFICATION;
}
 
+   /* it makes no sense to process too many deltas */
+   if (nxml->serial - nxml->repository->serial > 300)
+   goto snapshot;
+
/* check that all needed deltas are available */
s = nxml->repository->serial + 1;
TAILQ_FOREACH(d, >delta_q, q) {



Re: rpki-client add back keep-alive to http requests

2021-09-10 Thread Claudio Jeker
On Thu, Sep 09, 2021 at 09:18:04AM -0600, Bob Beck wrote:
> 
> ok beck@
> 
> On Thu, Sep 09, 2021 at 09:35:51AM +0200, Claudio Jeker wrote:
> > While Connection: keep-alive should be the default it seems that at least
> > some of the CA repositories fail to behave like that. Adding back the
> > Connection header seems to fix this and delta downloads go faster again.
> > 

After further inspection of the code and the HTTP/1.1 RFC I came to the
conclusion that the code is wrong. For HTTP/1.1 the default should be
keep-alive being on. For non-HTTP/1.1 (aka HTTP/1.0 there is no other 1.X
spec) keep-alive should be off by default.

So look at the HTTP protocol header and set keep-alive then. This seems to
work. Also check for Connection: close. At least one server sends a
Connection: close header after a bunch of requests.

So this is a better fix for the keep-alive issue.
-- 
:wq Claudio

Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.38
diff -u -p -r1.38 http.c
--- http.c  1 Sep 2021 09:39:14 -   1.38
+++ http.c  10 Sep 2021 10:02:12 -
@@ -1053,11 +1053,16 @@ http_request(struct http_connection *con
 static int
 http_parse_status(struct http_connection *conn, char *buf)
 {
+#define HTTP_11"HTTP/1.1 "
const char *errstr;
char *cp, ststr[4];
char gerror[200];
int status;
 
+   /* Check if the protocol is 1.1 and enable keep-alive in that case */
+   if (strncmp(buf, HTTP_11, strlen(HTTP_11)) == 0)
+   conn->keep_alive = 1;
+
cp = strchr(buf, ' ');
if (cp == NULL) {
warnx("Improper response from %s", http_info(conn->host));
@@ -1226,7 +1231,9 @@ http_parse_header(struct http_connection
} else if (strncasecmp(cp, CONNECTION, sizeof(CONNECTION) - 1) == 0) {
cp += sizeof(CONNECTION) - 1;
cp[strcspn(cp, " \t")] = '\0';
-   if (strcasecmp(cp, "keep-alive") == 0)
+   if (strcasecmp(cp, "close") == 0)
+   conn->keep_alive = 0;
+   else if (strcasecmp(cp, "keep-alive") == 0)
conn->keep_alive = 1;
} else if (strncasecmp(cp, LAST_MODIFIED,
sizeof(LAST_MODIFIED) - 1) == 0) {



rpki-client compare oid with OBJ_cmp

2021-09-09 Thread Claudio Jeker
Trying to remove work that is done over and over again.
One of those checks are the various OID compares.
Instead of converting the ASN1_OBJECT into a string and comparing the
strings, convert the string into an ASN1_OBJECT once and then compare
these objects with OBJ_cmp().

Any comments about this?
-- 
:wq Claudio

Index: cert.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/cert.c,v
retrieving revision 1.31
diff -u -p -r1.31 cert.c
--- cert.c  13 Jul 2021 18:39:39 -  1.31
+++ cert.c  9 Sep 2021 12:25:47 -
@@ -46,6 +46,21 @@ struct   parse {
const char  *fn; /* currently-parsed file */
 };
 
+static ASN1_OBJECT *carepo_oid;/* 1.3.6.1.5.5.7.48.5 (caRepository) */
+static ASN1_OBJECT *mft_oid;   /* 1.3.6.1.5.5.7.48.10 (rpkiManifest) */
+static ASN1_OBJECT *notify_oid;/* 1.3.6.1.5.5.7.48.13 (rpkiNotify) */
+
+static void
+cert_init_oid(void)
+{
+   if ((carepo_oid = OBJ_txt2obj("1.3.6.1.5.5.7.48.5", 1)) == NULL)
+   errx(1, "OBJ_txt2obj for %s failed", "1.3.6.1.5.5.7.48.5");
+   if ((mft_oid = OBJ_txt2obj("1.3.6.1.5.5.7.48.10", 1)) == NULL)
+   errx(1, "OBJ_txt2obj for %s failed", "1.3.6.1.5.5.7.48.10");
+   if ((notify_oid = OBJ_txt2obj("1.3.6.1.5.5.7.48.13", 1)) == NULL)
+   errx(1, "OBJ_txt2obj for %s failed", "1.3.6.1.5.5.7.48.13");
+}
+
 /*
  * Append an IP address structure to our list of results.
  * This will also constrain us to having at most one inheritence
@@ -207,9 +222,9 @@ sbgp_sia_resource_entry(struct parse *p,
const unsigned char *d, size_t dsz)
 {
ASN1_SEQUENCE_ANY   *seq;
+   ASN1_OBJECT *oid;
const ASN1_TYPE *t;
int  rc = 0, ptag;
-   char buf[128];
long plen;
 
if ((seq = d2i_ASN1_SEQUENCE_ANY(NULL, , dsz)) == NULL) {
@@ -233,7 +248,7 @@ sbgp_sia_resource_entry(struct parse *p,
p->fn, ASN1_tag2str(t->type), t->type);
goto out;
}
-   OBJ_obj2txt(buf, sizeof(buf), t->value.object, 1);
+   oid = t->value.object;
 
t = sk_ASN1_TYPE_value(seq, 1);
if (t->type != V_ASN1_OTHER) {
@@ -257,11 +272,14 @@ sbgp_sia_resource_entry(struct parse *p,
 *  - 1.3.6.1.5.5.7.48.10 (rpkiManifest)
 *  - 1.3.6.1.5.5.7.48.13 (rpkiNotify)
 */
-   if (strcmp(buf, "1.3.6.1.5.5.7.48.5") == 0)
+   if (carepo_oid == NULL)
+   cert_init_oid();
+
+   if (OBJ_cmp(oid, carepo_oid) == 0)
rc = sbgp_sia_resource_carepo(p, d, plen);
-   else if (strcmp(buf, "1.3.6.1.5.5.7.48.10") == 0)
+   else if (OBJ_cmp(oid, mft_oid) == 0)
rc = sbgp_sia_resource_mft(p, d, plen);
-   else if (strcmp(buf, "1.3.6.1.5.5.7.48.13") == 0)
+   else if (OBJ_cmp(oid, notify_oid) == 0)
rc = sbgp_sia_resource_notify(p, d, plen);
else
rc = 1; /* silently ignore */
Index: cms.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/cms.c,v
retrieving revision 1.9
diff -u -p -r1.9 cms.c
--- cms.c   13 Jul 2021 18:39:39 -  1.9
+++ cms.c   9 Sep 2021 12:25:47 -
@@ -35,16 +35,15 @@
  * Return the eContent as a string and set "rsz" to be its length.
  */
 unsigned char *
-cms_parse_validate(X509 **xp, const char *fn,
-const char *oid, size_t *rsz)
+cms_parse_validate(X509 **xp, const char *fn, const ASN1_OBJECT *oid,
+size_t *rsz)
 {
const ASN1_OBJECT   *obj;
ASN1_OCTET_STRING   **os = NULL;
BIO *bio = NULL;
CMS_ContentInfo *cms;
FILE*f;
-   char buf[128];
-   int  rc = 0, sz;
+   int  rc = 0;
STACK_OF(X509)  *certs = NULL;
unsigned char   *res = NULL;
 
@@ -84,16 +83,12 @@ cms_parse_validate(X509 **xp, const char
/* RFC 6488 section 2.1.3.1: check the object's eContentType. */
 
obj = CMS_get0_eContentType(cms);
-   if ((sz = OBJ_obj2txt(buf, sizeof(buf), obj, 1)) < 0)
-   cryptoerrx("OBJ_obj2txt");
-
-   if ((size_t)sz >= sizeof(buf)) {
-   warnx("%s: RFC 6488 section 2.1.3.1: "
-   "eContentType: OID too long", fn);
-   goto out;
-   } else if (strcmp(buf, oid)) {
+   if (OBJ_cmp(obj, oid) != 0) {
+   char buf[128], obuf[128];
+   OBJ_obj2txt(buf, sizeof(buf), obj, 1);
+   OBJ_obj2txt(obuf, sizeof(obuf), oid, 1);
warnx("%s: RFC 6488 section 2.1.3.1: eContentType: "
-   "unknown OID: %s, want %s", fn, buf, oid);
+   "unknown OID: %s, want %s", fn, buf, obuf);
goto out;
}
 
Index: 

rpki-client add back keep-alive to http requests

2021-09-09 Thread Claudio Jeker
While Connection: keep-alive should be the default it seems that at least
some of the CA repositories fail to behave like that. Adding back the
Connection header seems to fix this and delta downloads go faster again.

-- 
:wq Claudio

Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.38
diff -u -p -r1.38 http.c
--- http.c  1 Sep 2021 09:39:14 -   1.38
+++ http.c  9 Sep 2021 07:26:35 -
@@ -1026,6 +1026,7 @@ http_request(struct http_connection *con
conn->bufpos = 0;
if ((r = asprintf(>buf,
"GET /%s HTTP/1.1\r\n"
+   "Connection: keep-alive\r\n"
"Host: %s\r\n"
"Accept-Encoding: identity\r\n"
"User-Agent: " HTTP_USER_AGENT "\r\n"



Re: do less recallocarray calls in rpki-client

2021-09-08 Thread Claudio Jeker
On Wed, Sep 08, 2021 at 05:40:31PM +0200, Theo Buehler wrote:
> On Wed, Sep 08, 2021 at 03:05:41PM +0200, Claudio Jeker wrote:
> > Looking at profiling information and the code made me realize that these
> > recallocarray calls growing the array by one every time are unnecessary.
> > The size of the array is known in advance so use that information and
> > build it up ahead of time.
> > 
> > In the roa case the IP list is double nested and so one still needs to
> > resize the array but for mft the file-and-hash list is only defined once
> > per MFT and so calloc can be used there.
> > 
> > Works for me, the speed-up is not really noticable but I think this change
> > is still worth it.
> 
> I'm ok with this.
> 
> Should we assert after the for loops that we added as many filehashes or
> addresses as allocated? This is currently more obvious than after this
> change.

Hmm. The code is correct even if it loads less elements. It will
overallocate but with NULL. Also currently the code fails hard if an
element does not parse. So not sure about that.
 
> > -- 
> > :wq Claudio
> > 
> > Index: roa.c
> > ===
> > RCS file: /cvs/src/usr.sbin/rpki-client/roa.c,v
> > retrieving revision 1.23
> > diff -u -p -r1.23 roa.c
> > --- roa.c   1 Aug 2021 22:29:49 -   1.23
> > +++ roa.c   7 Sep 2021 13:19:19 -
> > @@ -109,10 +109,6 @@ roa_parse_addr(const ASN1_OCTET_STRING *
> > }
> > }
> >  
> > -   p->res->ips = recallocarray(p->res->ips, p->res->ipsz, p->res->ipsz + 1,
> > -   sizeof(struct roa_ip));
> > -   if (p->res->ips == NULL)
> > -   err(1, NULL);
> > res = >res->ips[p->res->ipsz++];
> >  
> > res->addr = addr;
> > @@ -180,6 +176,12 @@ roa_parse_ipfam(const ASN1_OCTET_STRING 
> > "failed ASN.1 sequence parse", p->fn);
> > goto out;
> > }
> > +
> > +   /* will be called multiple times so use recallocarray */
> > +   p->res->ips = recallocarray(p->res->ips, p->res->ipsz,
> > +   p->res->ipsz + sk_ASN1_TYPE_num(sseq), sizeof(struct roa_ip));
> > +   if (p->res->ips == NULL)
> > +   err(1, NULL);
> >  
> > for (i = 0; i < sk_ASN1_TYPE_num(sseq); i++) {
> > t = sk_ASN1_TYPE_value(sseq, i);
> > Index: mft.c
> > ===
> > RCS file: /cvs/src/usr.sbin/rpki-client/mft.c,v
> > retrieving revision 1.36
> > diff -u -p -r1.36 mft.c
> > --- mft.c   13 Jul 2021 18:39:39 -  1.36
> > +++ mft.c   7 Sep 2021 12:59:57 -
> > @@ -194,12 +194,6 @@ mft_parse_filehash(struct parse *p, cons
> > }
> >  
> > /* Insert the filename and hash value. */
> > -
> > -   p->res->files = recallocarray(p->res->files, p->res->filesz,
> > -   p->res->filesz + 1, sizeof(struct mftfile));
> > -   if (p->res->files == NULL)
> > -   err(1, NULL);
> > -
> > fent = >res->files[p->res->filesz++];
> >  
> > fent->file = fn;
> > @@ -231,6 +225,10 @@ mft_parse_flist(struct parse *p, const A
> > "failed ASN.1 sequence parse", p->fn);
> > goto out;
> > }
> > +
> > +   p->res->files = calloc(sk_ASN1_TYPE_num(seq), sizeof(struct mftfile));
> > +   if (p->res->files == NULL)
> > +   err(1, NULL);
> >  
> > for (i = 0; i < sk_ASN1_TYPE_num(seq); i++) {
> > t = sk_ASN1_TYPE_value(seq, i);
> > 
> 
> 

-- 
:wq Claudio



do less recallocarray calls in rpki-client

2021-09-08 Thread Claudio Jeker
Looking at profiling information and the code made me realize that these
recallocarray calls growing the array by one every time are unnecessary.
The size of the array is known in advance so use that information and
build it up ahead of time.

In the roa case the IP list is double nested and so one still needs to
resize the array but for mft the file-and-hash list is only defined once
per MFT and so calloc can be used there.

Works for me, the speed-up is not really noticable but I think this change
is still worth it.
-- 
:wq Claudio

Index: roa.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/roa.c,v
retrieving revision 1.23
diff -u -p -r1.23 roa.c
--- roa.c   1 Aug 2021 22:29:49 -   1.23
+++ roa.c   7 Sep 2021 13:19:19 -
@@ -109,10 +109,6 @@ roa_parse_addr(const ASN1_OCTET_STRING *
}
}
 
-   p->res->ips = recallocarray(p->res->ips, p->res->ipsz, p->res->ipsz + 1,
-   sizeof(struct roa_ip));
-   if (p->res->ips == NULL)
-   err(1, NULL);
res = >res->ips[p->res->ipsz++];
 
res->addr = addr;
@@ -180,6 +176,12 @@ roa_parse_ipfam(const ASN1_OCTET_STRING 
"failed ASN.1 sequence parse", p->fn);
goto out;
}
+
+   /* will be called multiple times so use recallocarray */
+   p->res->ips = recallocarray(p->res->ips, p->res->ipsz,
+   p->res->ipsz + sk_ASN1_TYPE_num(sseq), sizeof(struct roa_ip));
+   if (p->res->ips == NULL)
+   err(1, NULL);
 
for (i = 0; i < sk_ASN1_TYPE_num(sseq); i++) {
t = sk_ASN1_TYPE_value(sseq, i);
Index: mft.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/mft.c,v
retrieving revision 1.36
diff -u -p -r1.36 mft.c
--- mft.c   13 Jul 2021 18:39:39 -  1.36
+++ mft.c   7 Sep 2021 12:59:57 -
@@ -194,12 +194,6 @@ mft_parse_filehash(struct parse *p, cons
}
 
/* Insert the filename and hash value. */
-
-   p->res->files = recallocarray(p->res->files, p->res->filesz,
-   p->res->filesz + 1, sizeof(struct mftfile));
-   if (p->res->files == NULL)
-   err(1, NULL);
-
fent = >res->files[p->res->filesz++];
 
fent->file = fn;
@@ -231,6 +225,10 @@ mft_parse_flist(struct parse *p, const A
"failed ASN.1 sequence parse", p->fn);
goto out;
}
+
+   p->res->files = calloc(sk_ASN1_TYPE_num(seq), sizeof(struct mftfile));
+   if (p->res->files == NULL)
+   err(1, NULL);
 
for (i = 0; i < sk_ASN1_TYPE_num(seq); i++) {
t = sk_ASN1_TYPE_value(seq, i);



Re: Change vm_dsize to vsize_t

2021-09-07 Thread Claudio Jeker
On Mon, Sep 06, 2021 at 12:39:56PM -0700, Greg Steuck wrote:
> In the course of making ASan work on OpenBSD I ran into an accounting
> limitation. struct vmspace declares vm_dsize as segsz_t (aka int32_t).
> This effectively limits it to 2^31 pages (2^43 bytes on amd64). This
> would be enough if didn't also count sparse allocation.
> 
> ASan allocates 1/8th of the process address space as shadow memory. It
> is very sparsely populated, still given VM_MAXUSER_ADDRESS value of
> 0x7f7fc000, it goes up to 2^47 bytes which then requires 2^44
> bytes of shadow. So, it won't fit.
> 
> Hence the following unfinished patch which allows simple ASan'd programs
> to detect memory errors. If people don't see an alternative solution,
> I'll fix up the users of kinfo_proc.p_vm_dsize and we can decide
> when/how this should land.
> 
> From 42c776531620e9baa6735da71349c3c045fb64d8 Mon Sep 17 00:00:00 2001
> From: Greg Steuck 
> Date: Sun, 5 Sep 2021 13:28:43 -0700
> Subject: [PATCH] Change struct vmspace to use vsize_t vm_dused
> 
> This was overflowing given high MAXDSIZ. This is very appropriate given
> that the field is usually incremented by a value returned by
> uvmspace_dused which returns vsize_t.
> 
> The change is not finished, only kernel is fixed so far. Userspace
> tools consuming p_vm_dsize from kinfo_proc are likely not correct.
> ---
>  sys/sys/sysctl.h | 2 +-
>  sys/uvm/uvm_extern.h | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/sys/sys/sysctl.h b/sys/sys/sysctl.h
> index afdc0689dee..868ef82c696 100644
> --- a/sys/sys/sysctl.h
> +++ b/sys/sys/sysctl.h
> @@ -443,7 +443,7 @@ struct kinfo_proc {
>  
>   int32_t p_vm_rssize;/* SEGSZ_T: current resident set size 
> in pages */
>   int32_t p_vm_tsize; /* SEGSZ_T: text size (pages) */
> - int32_t p_vm_dsize; /* SEGSZ_T: data size (pages) */
> + u_int64_t   p_vm_dsize; /* VSIZE_T: data size (pages) */
>   int32_t p_vm_ssize; /* SEGSZ_T: stack size (pages) */
>  
>   int64_t p_uvalid;   /* CHAR: following p_u* members from 
> struct user are valid */

>From my understanding this is not how struct kinfo_proc should be modified.
Instead the code should add the u_int64_t version at the end and leave the
old in place. This way old userland still works with new kernel.

> diff --git a/sys/uvm/uvm_extern.h b/sys/uvm/uvm_extern.h
> index faa4a2e5449..ebc74d97917 100644
> --- a/sys/uvm/uvm_extern.h
> +++ b/sys/uvm/uvm_extern.h
> @@ -207,7 +207,7 @@ struct vmspace {
>   segsz_t vm_swrss;   /* resident set size before last swap */
>   segsz_t vm_tsize;   /* text size (pages) XXX */
>   segsz_t vm_dsize;   /* data size (pages) XXX */
> - segsz_t vm_dused;   /* data segment length (pages) XXX */
> + vsize_t vm_dused;   /* data segment length (pages) XXX */
>   segsz_t vm_ssize;   /* [v] stack size (pages) */
>   caddr_t vm_taddr;   /* [I] user virtual address of text */
>   caddr_t vm_daddr;   /* [I] user virtual address of data */
> -- 
> 2.33.0
> 

-- 
:wq Claudio



Re: iked(8): make proto option accept lists

2021-09-03 Thread Claudio Jeker
On Fri, Sep 03, 2021 at 10:12:57AM +0200, Sebastian Benoit wrote:
> Tobias Heider(tobias.hei...@stusta.de) on 2021.09.02 15:39:46 +0200:
> > The diff below makes iked accept a list of protocols for the "proto" config
> > option in iked.conf(5).
> > This would allow us to have a single policy with "proto { ipencap, ipv6 }"
> > to secure a gif(4) tunnel, instead of requiring one policy for each 
> > protocol.
> > 
> > ok?
> 
> I only gave the parser bits a quick read.
> 
> The manpage change would be nice to compare the parser with what you
> document.
> 
> A bit more below.
>  
> > Index: iked.h
> > ===
> > RCS file: /cvs/src/sbin/iked/iked.h,v
> > retrieving revision 1.193
> > diff -u -p -r1.193 iked.h
> > --- iked.h  1 Sep 2021 15:30:06 -   1.193
> > +++ iked.h  2 Sep 2021 13:37:21 -
> > @@ -242,10 +242,9 @@ struct iked_policy {
> >  
> >  #define IKED_SKIP_FLAGS 0
> >  #define IKED_SKIP_AF1
> > -#define IKED_SKIP_PROTO 2
> > -#define IKED_SKIP_SRC_ADDR  3
> > -#define IKED_SKIP_DST_ADDR  4
> > -#define IKED_SKIP_COUNT 5
> > +#define IKED_SKIP_SRC_ADDR  2
> > +#define IKED_SKIP_DST_ADDR  3
> > +#define IKED_SKIP_COUNT 4
> > struct iked_policy  *pol_skip[IKED_SKIP_COUNT];
> >  
> > uint8_t  pol_flags;
> > @@ -265,7 +264,8 @@ struct iked_policy {
> > int  pol_af;
> > int  pol_rdomain;
> > uint8_t  pol_saproto;
> > -   unsigned int pol_ipproto;
> > +   unsigned int pol_ipproto[IKED_IPPROTO_MAX];
> > +   unsigned int pol_nipproto;
> >  
> > struct iked_addr pol_peer;
> > struct iked_static_idpol_peerid;
> > Index: parse.y
> > ===
> > RCS file: /cvs/src/sbin/iked/parse.y,v
> > retrieving revision 1.131
> > diff -u -p -r1.131 parse.y
> > --- parse.y 28 May 2021 18:01:39 -  1.131
> > +++ parse.y 2 Sep 2021 13:37:21 -
> > @@ -374,7 +374,7 @@ void copy_transforms(unsigned int,
> > const struct ipsec_xf **, unsigned int,
> > struct iked_transform **, unsigned int *,
> > struct iked_transform *, size_t);
> > -int create_ike(char *, int, uint8_t,
> > +int create_ike(char *, int, struct ipsec_addr_wrap 
> > *,
> > int, struct ipsec_hosts *,
> > struct ipsec_hosts *, struct ipsec_mode *,
> > struct ipsec_mode *, uint8_t,
> > @@ -388,9 +388,9 @@ uint8_t  x2i(unsigned char *);
> >  int parsekey(unsigned char *, size_t, struct 
> > iked_auth *);
> >  int parsekeyfile(char *, struct iked_auth *);
> >  voidiaw_free(struct ipsec_addr_wrap *);
> > -static int  create_flow(struct iked_policy *pol, struct 
> > ipsec_addr_wrap *ipa,
> > +static int  create_flow(struct iked_policy *pol, int, struct 
> > ipsec_addr_wrap *ipa,
> > struct ipsec_addr_wrap *ipb);
> > -static int  expand_flows(struct iked_policy *, struct 
> > ipsec_addr_wrap *,
> > +static int  expand_flows(struct iked_policy *, int, struct 
> > ipsec_addr_wrap *,
> > struct ipsec_addr_wrap *);
> >  static struct ipsec_addr_wrap *
> >  expand_keyword(struct ipsec_addr_wrap *);
> > @@ -407,7 +407,6 @@ typedef struct {
> > uint8_t  ikemode;
> > uint8_t  dir;
> > uint8_t  satype;
> > -   uint8_t  proto;
> > char*string;
> > uint16_t port;
> > struct ipsec_hosts  *hosts;
> > @@ -415,6 +414,7 @@ typedef struct {
> > struct ipsec_addr_wrap  *anyhost;
> > struct ipsec_addr_wrap  *host;
> > struct ipsec_addr_wrap  *cfg;
> > +   struct ipsec_addr_wrap  *proto;
> > struct {
> > char*srcid;
> > char*dstid;
> > @@ -449,8 +449,7 @@ typedef struct {
> >  %token   NUMBER
> >  %typestring
> >  %typesatype
> > -%type proto
> > -%typeprotoval
> > +%type proto proto_list protoval
> >  %type hosts hosts_list
> >  %type  port
> >  %typeportval af rdomain
> > @@ -632,8 +631,21 @@ af : /* empty */ 

Re: rpki-client add http_proxy support

2021-09-01 Thread Claudio Jeker
On Wed, Sep 01, 2021 at 09:38:55AM +, Job Snijders wrote:
> On Tue, Aug 31, 2021 at 09:58:54AM +0200, Claudio Jeker wrote:
> > This diff improves the http code by a) adding an IO timeout and b)
> > implementing http_proxy support.
> > 
> > Works for me using tinyproxy as proxy server.
> 
> OK?

Already fixed :)
 
> Index: http.c
> ===
> RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
> retrieving revision 1.37
> diff -u -p -r1.37 http.c
> --- http.c1 Sep 2021 08:09:41 -   1.37
> +++ http.c1 Sep 2021 09:38:29 -
> @@ -658,7 +658,7 @@ http_new(struct http_request *req)
>   LIST_INSERT_HEAD(, conn, entry);
>   http_conn_count++;
>  
> - if (proxy.proxyhost == NULL) {
> + if (proxy.proxyhost != NULL) {
>   if (http_resolv(>res0, proxy.proxyhost,
>   proxy.proxyport) == -1) {
>   http_req_fail(req->id);
> @@ -796,7 +796,7 @@ http_connect_done(struct http_connection
>   conn->res0 = NULL;
>   conn->res = NULL;
>  
> - if (proxy.proxyhost == NULL)
> + if (proxy.proxyhost != NULL)
>   return proxy_connect(conn);
>   return http_tls_connect(conn);
>  }
> 

-- 
:wq Claudio



Re: rpki-client exclude files from rsync fetch

2021-09-01 Thread Claudio Jeker
On Tue, Aug 31, 2021 at 02:23:57PM +0200, Claudio Jeker wrote:
> RPKI repository can only include a few specific files, everything else is
> just ignored and deleted after every fetch.  Since openrsync supports
> --exclude-file now we can use this to limit what is actually accepted by
> the client.
> 
> I used a config file in /etc/rpki instead of using multiple --exclude /
> --include arguments. Mostly to keep the execvp argv short.
> 
> What you think?

It seems using a config file to keep the argv list short is too
controversial and all alternate suggestions are worse.
So just add the include/exclude list as arguments.

-- 
:wq Claudio

Index: rsync.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rsync.c,v
retrieving revision 1.24
diff -u -p -r1.24 rsync.c
--- rsync.c 19 Apr 2021 17:04:35 -  1.24
+++ rsync.c 1 Sep 2021 09:08:06 -
@@ -277,8 +277,14 @@ proc_rsync(char *prog, char *bind_addr, 
args[i++] = (char *)prog;
args[i++] = "-rt";
args[i++] = "--no-motd";
-   args[i++] = "--timeout";
-   args[i++] = "180";
+   args[i++] = "--timeout=180";
+   args[i++] = "--include=*/";
+   args[i++] = "--include=*.cer";
+   args[i++] = "--include=*.crl";
+   args[i++] = "--include=*.gbr";
+   args[i++] = "--include=*.mft";
+   args[i++] = "--include=*.roa";
+   args[i++] = "--exclude=*";
if (bind_addr != NULL) {
args[i++] = "--address";
args[i++] = (char *)bind_addr;



rpki-client exclude files from rsync fetch

2021-08-31 Thread Claudio Jeker
RPKI repository can only include a few specific files, everything else is
just ignored and deleted after every fetch.  Since openrsync supports
--exclude-file now we can use this to limit what is actually accepted by
the client.

I used a config file in /etc/rpki instead of using multiple --exclude /
--include arguments. Mostly to keep the execvp argv short.

What you think?
-- 
:wq Claudio

Index: etc/Makefile
===
RCS file: /cvs/src/etc/Makefile,v
retrieving revision 1.484
diff -u -p -r1.484 Makefile
--- etc/Makefile1 May 2021 16:11:07 -   1.484
+++ etc/Makefile31 Aug 2021 12:17:40 -
@@ -156,7 +156,7 @@ distribution-etc-root-var: distrib-dirs
${DESTDIR}/etc/ppp
cd rpki; \
${INSTALL} -c -o root -g wheel -m 644 \
-   afrinic.tal apnic.tal lacnic.tal ripe.tal \
+   afrinic.tal apnic.tal lacnic.tal ripe.tal rsync.filter \
${DESTDIR}/etc/rpki
cd examples; \
${INSTALL} -c -o root -g wheel -m 644 ${EXAMPLES} \
Index: etc/rpki/rsync.filter
===
RCS file: etc/rpki/rsync.filter
diff -N etc/rpki/rsync.filter
--- /dev/null   1 Jan 1970 00:00:00 -
+++ etc/rpki/rsync.filter   31 Aug 2021 12:09:24 -
@@ -0,0 +1,7 @@
++ */
++ *.cer
++ *.crl
++ *.gbr
++ *.mft
++ *.roa
+- *
Index: usr.sbin/rpki-client/rsync.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/rsync.c,v
retrieving revision 1.24
diff -u -p -r1.24 rsync.c
--- usr.sbin/rpki-client/rsync.c19 Apr 2021 17:04:35 -  1.24
+++ usr.sbin/rpki-client/rsync.c31 Aug 2021 12:17:11 -
@@ -279,6 +279,8 @@ proc_rsync(char *prog, char *bind_addr, 
args[i++] = "--no-motd";
args[i++] = "--timeout";
args[i++] = "180";
+   args[i++] = "--exclude-from";
+   args[i++] = "/etc/rpki/rsync.filter";
if (bind_addr != NULL) {
args[i++] = "--address";
args[i++] = (char *)bind_addr;



rpki-client add http_proxy support

2021-08-31 Thread Claudio Jeker
This diff improves the http code by a) adding an IO timeout and b)
implementing http_proxy support.

Works for me using tinyproxy as proxy server.
-- 
:wq Claudio

Index: encoding.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/encoding.c,v
retrieving revision 1.2
diff -u -p -r1.2 encoding.c
--- encoding.c  19 Apr 2021 17:04:35 -  1.2
+++ encoding.c  30 Aug 2021 19:53:47 -
@@ -64,6 +64,36 @@ fail:
return -1;
 }
 
+int
+base64_encode(const unsigned char *in, size_t inlen, char **out)
+{
+   static EVP_ENCODE_CTX *ctx;
+   unsigned char *to;
+   int tolen;
+
+   if (ctx == NULL && (ctx = EVP_ENCODE_CTX_new()) == NULL)
+   err(1, "EVP_ENCODE_CTX_new");
+
+   *out = NULL;
+
+   if (inlen >= INT_MAX / 2)
+   return -1;
+   tolen = ((inlen + 2) / 3) * 4 + 1;
+   if ((to = malloc(tolen)) == NULL)
+   return -1;
+
+   EVP_EncodeInit(ctx);
+   if (EVP_EncodeUpdate(ctx, to, , in, inlen) != 1)
+   goto fail;
+   EVP_EncodeFinal(ctx, to + tolen, );
+   *out = to;
+   return 0;
+
+fail:
+   free(to);
+   return -1;
+}
+
 /*
  * Convert binary buffer of size dsz into an upper-case hex-string.
  * Returns pointer to the newly allocated string. Function can't fail.
Index: extern.h
===
RCS file: /cvs/src/usr.sbin/rpki-client/extern.h,v
retrieving revision 1.65
diff -u -p -r1.65 extern.h
--- extern.h13 Jul 2021 18:39:39 -  1.65
+++ extern.h30 Aug 2021 19:52:47 -
@@ -364,8 +364,6 @@ extern int verbose;
 
 /* Routines for RPKI entities. */
 
-int base64_decode(const unsigned char *, unsigned char **,
-   size_t *);
 voidtal_buffer(struct ibuf *, const struct tal *);
 voidtal_free(struct tal *);
 struct tal *tal_parse(const char *, char *);
@@ -499,6 +497,7 @@ void cryptoerrx(const char *, ...)
 
 int base64_decode(const unsigned char *, unsigned char **,
size_t *);
+int base64_encode(const unsigned char *, size_t, char **);
 char   *hex_encode(const unsigned char *, size_t);
 
 
Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.36
diff -u -p -r1.36 http.c
--- http.c  9 Aug 2021 10:30:23 -   1.36
+++ http.c  31 Aug 2021 07:52:16 -
@@ -70,6 +70,7 @@
 #define HTTP_USER_AGENT"OpenBSD rpki-client"
 #define HTTP_BUF_SIZE  (32 * 1024)
 #define HTTP_IDLE_TIMEOUT  10
+#define HTTP_IO_TIMEOUT(3 * 60)
 #define MAX_CONNECTIONS64
 #define NPFDS  (MAX_CONNECTIONS + 1)
 
@@ -83,6 +84,9 @@ enum http_state {
STATE_FREE,
STATE_CONNECT,
STATE_TLSCONNECT,
+   STATE_PROXY_REQUEST,
+   STATE_PROXY_STATUS,
+   STATE_PROXY_RESPONSE,
STATE_REQUEST,
STATE_RESPONSE_STATUS,
STATE_RESPONSE_HEADER,
@@ -96,9 +100,9 @@ enum http_state {
 
 struct http_proxy {
char*proxyhost;
-   char*proxyuser;
-   char*proxypw;
-};
+   char*proxyport;
+   char*proxyauth;
+} proxy;
 
 struct http_connection {
LIST_ENTRY(http_connection) entry;
@@ -116,6 +120,7 @@ struct http_connection {
size_t  bufpos;
off_t   iosz;
time_t  idle_time;
+   time_t  io_time;
int status;
int fd;
int chunked;
@@ -177,10 +182,13 @@ static enum res http_handle(struct http_
 
 /* Internal state functions used by the above functions */
 static enum reshttp_finish_connect(struct http_connection *);
+static enum resproxy_connect(struct http_connection *);
 static enum reshttp_tls_connect(struct http_connection *);
 static enum reshttp_tls_handshake(struct http_connection *);
 static enum reshttp_read(struct http_connection *);
 static enum reshttp_write(struct http_connection *);
+static enum resproxy_read(struct http_connection *);
+static enum resproxy_write(struct http_connection *);
 static enum resdata_write(struct http_connection *);
 
 static time_t
@@ -277,6 +285,138 @@ url_encode(const char *path)
return (epath);
 }
 
+static char
+hextochar(const char *str)
+{
+   unsigned char c, ret;
+
+   c = str[0];
+   ret = c;
+   if (isalpha(c))
+   ret -= isupper(c) ? 'A' - 10 : 'a' - 10;
+   else
+   ret -= '0';
+   ret *= 16;
+
+   c = str[1];
+   ret += c;
+   if (isalpha(c))
+   ret -= isupper(c) ? 'A' - 10 : 'a' - 10;
+   else
+   ret -= '0';
+   return 

Re: route(1): add an address family validation

2021-08-27 Thread Claudio Jeker
On Fri, Aug 27, 2021 at 03:58:23PM +0900, morimoto wrote:
> hi,
> I found an interesting issue while toying routing.
> route(1) accepts IPv4 destination and IPv6 gateway entry.
> command is as below:
> route add 192.0.2.1 2001:db8::1
> 
> Curiously it has no error.
> The entry is pointless, I think it should teach a mistake.
> If destination and gateway address family are not the same, it should return 
> error.
> 
> Comments?

I don't think the entry is pointless. There are setups where nexthop of
different address family do make sense. Things like rfc5549 can do
IPv4 over IPv6 Core. In some cases this is used for network autodiscovery
(using IPv6 link-local addresses as nexthops). 
Because of this I think it would make more sense to make this actually
work.
 
> Index: sbin/route/route.c
> ===
> RCS file: /cvs/src/sbin/route/route.c,v
> retrieving revision 1.254
> diff -u -p -r1.254 route.c
> --- sbin/route/route.c12 Mar 2021 19:35:43 -  1.254
> +++ sbin/route/route.c26 Aug 2021 09:02:17 -
> @@ -782,6 +782,9 @@ newroute(int argc, char **argv)
>   break;
>   }
>   oerrno = errno;
> + if ((rtm_addrs & RTA_GATEWAY) == 0 &&
> + so_dst.sa.sa_family != so_gate.sa.sa_family)
> + errx(1, "address family mismatch");
>   if (!qflag && (*cmd != 'g' || ret != 0)) {
>   printf("%s %s %s", cmd, ishost ? "host" : "net", dest);
>   if (*gateway) {
> 
> Index: sys/net/rtsock.c
> ===
> RCS file: /cvs/src/sys/net/rtsock.c,v
> retrieving revision 1.319
> diff -u -p -r1.319 rtsock.c
> --- sys/net/rtsock.c  23 Jun 2021 16:10:45 -  1.319
> +++ sys/net/rtsock.c  26 Aug 2021 09:02:01 -
> @@ -822,6 +822,12 @@ route_output(struct mbuf *m, struct sock
>   error = EINVAL;
>   goto fail;
>   }
> + if ((rtm->rtm_type == RTM_ADD || rtm->rtm_type == RTM_CHANGE) &&
> + info.rti_info[RTAX_DST]->sa_family !=
> + info.rti_info[RTAX_GATEWAY]->sa_family) {
> + error = EINVAL;
> + goto fail;
> + }
> #ifdef MPLS
>   info.rti_mpls = rtm->rtm_mpls;
> #endif
> 
> 

-- 
:wq Claudio



Re: bgpd MRT RFC8050 support (add-path for mrt dumps)

2021-08-18 Thread Claudio Jeker
On Mon, Aug 09, 2021 at 12:17:47PM +0200, Claudio Jeker wrote:
> This diff adds the bits needed to support add-path in MRT dumps.
> The problem here is that MRT as a stateless protocol has no chance
> to know what kind of encoding (add-path or not) is used for the NLRI in
> message dumps. And for table dumps there is a need to add an extra field
> to the dumps to show the path-id.
> 
> There are two issues:
> - for message dumps: it is necessary to peek into UPDATE messages to
>   figure out if that update is using add-path or not. This comes from the
>   fact that the add-path RFC allows to set the option per AFI/SAFI and
>   also per direction. This is a major pain in bgpd since UPDATE messages
>   are actually parsed in the RDE and not in the SE. The SE just does the
>   basic lenght checks (header size, total length). So this peak into the
>   packet needs to be done with some care (especialy for MP encoded
>   UPDATEs).
> 
> - for table dumps the RFC did a major fobar and defined a extra special
>   encoding for non-IPv4/IPv6 prefixes. In the general encoding the path-id
>   is not part of the rib entries sub-struct but is instead part of the
>   NLRI. This encoding is to complex to build into the bgpd codebase and it
>   seems the only other BGP implementation supporting RIB_GENERIC_ADDPATH,
>   gobgp, is also not implementing it according to the RFC but instead is
>   using the same encoding as for the other _ADDPATH types. OpenBGPD will
>   do the same.
> 
> This seems to work for me and results in the right output in bgpctl.
> 
> Please review mrt_bgp_guess_aid() and check if all checks are correct to
> ensure we don't run off the rails.

Ping

-- 
:wq Claudio

Index: mrt.c
===
RCS file: /cvs/src/usr.sbin/bgpd/mrt.c,v
retrieving revision 1.104
diff -u -p -r1.104 mrt.c
--- mrt.c   24 Jun 2021 10:04:05 -  1.104
+++ mrt.c   9 Aug 2021 10:13:43 -
@@ -91,23 +91,128 @@ int mrt_open(struct mrt *, time_t);
x == MRT_TABLE_DUMP_V2) ? RDEIDX : SEIDX\
)
 
-void
-mrt_dump_bgp_msg(struct mrt *mrt, void *pkg, u_int16_t pkglen,
-struct peer *peer)
+static u_int8_t
+mrt_bgp_guess_aid(u_int8_t *pkg, u_int16_t pkglen)
+{
+   u_int16_t wlen, alen, len, afi;
+   u_int8_t type, aid;
+
+   pkg += MSGSIZE_HEADER;
+   pkglen -= MSGSIZE_HEADER;
+
+   if (pkglen < 4)
+   goto bad;
+
+   memcpy(, pkg, 2);
+   wlen = ntohs(wlen);
+   pkg += 2;
+   pkglen -= 2;
+
+   memcpy(, pkg, 2);
+   alen = ntohs(alen);
+   pkg += 2;
+   pkglen -= 2;
+
+   if (wlen > 0 || alen < pkglen || (wlen == 0 && alen == 0)) {
+   /* UPDATE has withdraw or NLRI prefixes or IPv4 EoR */
+   return AID_INET;
+   }
+
+   /* bad attribute length */
+   if (alen > pkglen)
+   goto bad;
+
+   /* try to extract AFI/SAFI from the MP attributes */
+   while (alen > 0) {
+   if (alen < 3)
+   goto bad;
+   type = pkg[1];
+   if (pkg[0] & ATTR_EXTLEN) {
+   if (alen < 4)
+   goto bad;
+   memcpy(, pkg + 2, 2);
+   len = ntohs(len);
+   pkg += 4;
+   alen -= 4;
+   } else {
+   len = pkg[2];
+   pkg += 3;
+   alen -= 3;
+   }
+   if (len > alen)
+   goto bad;
+
+   if (type == ATTR_MP_REACH_NLRI ||
+   type == ATTR_MP_UNREACH_NLRI) {
+   if (alen < 3)
+   goto bad;
+   memcpy(, pkg, 2);
+   afi = ntohs(afi);
+   if (afi2aid(afi, pkg[2], ) == -1)
+   goto bad;
+   return aid;
+   }
+
+   pkg += len;
+   alen -= len;
+   }
+
+bad:
+   return AID_UNSPEC;
+}
+
+static u_int16_t
+mrt_bgp_msg_subtype(struct mrt *mrt, void *pkg, u_int16_t pkglen,
+struct peer *peer, enum msg_type msgtype, int in)
 {
-   struct ibuf *buf;
-   int  incoming = 0;
u_int16_tsubtype = BGP4MP_MESSAGE;
+   u_int8_t aid, mask;
 
if (peer->capa.neg.as4byte)
subtype = BGP4MP_MESSAGE_AS4;
 
+   if (msgtype != UPDATE)
+   return subtype;
+
+   /*
+* RFC8050 adjust types for add-path enabled sessions.
+* It is necessary to extract the AID from UPDATES to decide
+* if the add-path types are needed or not. The ADDPATH
+* subtypes only matter for BGP UPDATES.
+  

Re: libedit: stop ignoring SIGINT

2021-08-09 Thread Claudio Jeker
On Mon, Aug 09, 2021 at 01:19:08PM +0200, Ingo Schwarze wrote:
> Hi,
> 
> as mentioned earlier, deraadt@ reported that sftp(1) ignores Ctrl-C.
> Fixing that without longjmp(3) requires making editline(3) better
> behaved.
> 
> Currently, when read(2) from the terminal gets interrupted by a
> signal, editline(3) ignores the (first) signal and unconditionally
> calls read(2) a second time.  That seems wrong: if the user hits
> Ctrl-C, it is sane to assume that they meant it, not that they
> want it ignored.
> 
> The following patch causes el_gets(3) and el_wgets(3) to return
> failure when read(2)ing from the terminal is interrupted by a
> signal other than SIGCONT or SIGWINCH.  That allows the calling
> program to decide what to do, usually either exit the program or
> provide a fresh prompt to the user.
> 
> If i know how to grep(1), the following programs in the base system
> use -ledit:
> 
>  * bc(1)
>It behaves well with the patch: Ctrl-C discards the current
>input line and starts a new input line.
>The reason why this already works even without the patch
>is that bc(1) does very scary stuff inside the signal handler:
>pass a file-global EditLine pointer on the heap to el_line(3)
>and access fields inside the returned struct.  Needless to
>say that no signal handler should do such things...
> 
>  * cdio(1)
>Behaviour is acceptable and unchanged with the patch:
>Ctrl-C exits cdio(1).
> 
>  * ftp(1)
>It behaves well with the patch: Ctrl-C discards the current
>input line and provides a fresh prompt.
>The reason why it already works without the patch is that ftp(1)
>uses setjmp(3)/longjmp(3) to forcefully grab back control
>from el_gets(3) without waiting for it to return.
> 
>  * lldb(1)
>It misbehaves with or without the patch and ignores Ctrl-C.
>I freely admit that i don't feel too enthusiastic about
>debugging that beast.
> 
>  * sftp(1)
>Behaviour is improved with the patch: Ctrl-C now exits sftp(1).
>If desired, i can supply a very simple follow-up patch to sftp.c
>to instead behave like ftp(1) and bc(1), i.e. discard the
>current input line and provide a fresh prompt.
>Neither doing undue work in the signal handler nor longjmp(3)
>will be required for that (if this patch gets committed).
> 
>  * bgplgsh(8), fsdb(8)
>I have no idea how to test those.  Does anyone think that testing
>either of them would be required?

I had a quick test with bgplgsh(8), hitting Ctrl-C exits bgplgsh. Same
before with or without the diff. Not sure if anyone is actually using
bgplgsh(8) -- I never used it.

-- 
:wq Claudio



bgpd MRT RFC8050 support (add-path for mrt dumps)

2021-08-09 Thread Claudio Jeker
This diff adds the bits needed to support add-path in MRT dumps.
The problem here is that MRT as a stateless protocol has no chance
to know what kind of encoding (add-path or not) is used for the NLRI in
message dumps. And for table dumps there is a need to add an extra field
to the dumps to show the path-id.

There are two issues:
- for message dumps: it is necessary to peek into UPDATE messages to
  figure out if that update is using add-path or not. This comes from the
  fact that the add-path RFC allows to set the option per AFI/SAFI and
  also per direction. This is a major pain in bgpd since UPDATE messages
  are actually parsed in the RDE and not in the SE. The SE just does the
  basic lenght checks (header size, total length). So this peak into the
  packet needs to be done with some care (especialy for MP encoded
  UPDATEs).

- for table dumps the RFC did a major fobar and defined a extra special
  encoding for non-IPv4/IPv6 prefixes. In the general encoding the path-id
  is not part of the rib entries sub-struct but is instead part of the
  NLRI. This encoding is to complex to build into the bgpd codebase and it
  seems the only other BGP implementation supporting RIB_GENERIC_ADDPATH,
  gobgp, is also not implementing it according to the RFC but instead is
  using the same encoding as for the other _ADDPATH types. OpenBGPD will
  do the same.

This seems to work for me and results in the right output in bgpctl.

Please review mrt_bgp_guess_aid() and check if all checks are correct to
ensure we don't run off the rails.
-- 
:wq Claudio

Index: mrt.c
===
RCS file: /cvs/src/usr.sbin/bgpd/mrt.c,v
retrieving revision 1.104
diff -u -p -r1.104 mrt.c
--- mrt.c   24 Jun 2021 10:04:05 -  1.104
+++ mrt.c   9 Aug 2021 10:13:43 -
@@ -91,23 +91,128 @@ int mrt_open(struct mrt *, time_t);
x == MRT_TABLE_DUMP_V2) ? RDEIDX : SEIDX\
)
 
-void
-mrt_dump_bgp_msg(struct mrt *mrt, void *pkg, u_int16_t pkglen,
-struct peer *peer)
+static u_int8_t
+mrt_bgp_guess_aid(u_int8_t *pkg, u_int16_t pkglen)
+{
+   u_int16_t wlen, alen, len, afi;
+   u_int8_t type, aid;
+
+   pkg += MSGSIZE_HEADER;
+   pkglen -= MSGSIZE_HEADER;
+
+   if (pkglen < 4)
+   goto bad;
+
+   memcpy(, pkg, 2);
+   wlen = ntohs(wlen);
+   pkg += 2;
+   pkglen -= 2;
+
+   memcpy(, pkg, 2);
+   alen = ntohs(alen);
+   pkg += 2;
+   pkglen -= 2;
+
+   if (wlen > 0 || alen < pkglen || (wlen == 0 && alen == 0)) {
+   /* UPDATE has withdraw or NLRI prefixes or IPv4 EoR */
+   return AID_INET;
+   }
+
+   /* bad attribute length */
+   if (alen > pkglen)
+   goto bad;
+
+   /* try to extract AFI/SAFI from the MP attributes */
+   while (alen > 0) {
+   if (alen < 3)
+   goto bad;
+   type = pkg[1];
+   if (pkg[0] & ATTR_EXTLEN) {
+   if (alen < 4)
+   goto bad;
+   memcpy(, pkg + 2, 2);
+   len = ntohs(len);
+   pkg += 4;
+   alen -= 4;
+   } else {
+   len = pkg[2];
+   pkg += 3;
+   alen -= 3;
+   }
+   if (len > alen)
+   goto bad;
+
+   if (type == ATTR_MP_REACH_NLRI ||
+   type == ATTR_MP_UNREACH_NLRI) {
+   if (alen < 3)
+   goto bad;
+   memcpy(, pkg, 2);
+   afi = ntohs(afi);
+   if (afi2aid(afi, pkg[2], ) == -1)
+   goto bad;
+   return aid;
+   }
+
+   pkg += len;
+   alen -= len;
+   }
+
+bad:
+   return AID_UNSPEC;
+}
+
+static u_int16_t
+mrt_bgp_msg_subtype(struct mrt *mrt, void *pkg, u_int16_t pkglen,
+struct peer *peer, enum msg_type msgtype, int in)
 {
-   struct ibuf *buf;
-   int  incoming = 0;
u_int16_tsubtype = BGP4MP_MESSAGE;
+   u_int8_t aid, mask;
 
if (peer->capa.neg.as4byte)
subtype = BGP4MP_MESSAGE_AS4;
 
+   if (msgtype != UPDATE)
+   return subtype;
+
+   /*
+* RFC8050 adjust types for add-path enabled sessions.
+* It is necessary to extract the AID from UPDATES to decide
+* if the add-path types are needed or not. The ADDPATH
+* subtypes only matter for BGP UPDATES.
+*/
+
+   mask = in ? CAPA_AP_RECV : CAPA_AP_SEND;
+   /* only guess if add-path could be active */
+   if (peer->capa.neg.add_path[0] & mask) {
+   aid = mrt_bgp_guess_aid(pkg, pkglen);
+   if (aid != AID_UNSPEC &&
+  

Re: bgpd add add-path receive support

2021-08-09 Thread Claudio Jeker
On Fri, Aug 06, 2021 at 08:34:18PM +0200, Sebastian Benoit wrote:
> Claudio Jeker(cje...@diehard.n-r-g.com) on 2021.08.04 17:55:45 +0200:
> > On Fri, Jul 30, 2021 at 12:02:12PM +0200, Claudio Jeker wrote:
> > > This diff implements the bit to support the receive side of
> > > RFC7911 - Advertisement of Multiple Paths in BGP.
> > > 
> > > I did some basic tests and it works for me. People running route
> > > collectors should give this a try. The interaction of Add-Path and bgpctl
> > > probably needs some work. Also the MRT dumper needs to be updated to
> > > support RFC8050. I have a partial diff for that ready as well.
> > > 
> > > Sending out multiple paths will follow in a later step since that is a
> > > bit more complex. I still need to decide how stable I want to make the
> > > assigned path_ids for the multiple paths and then changes to the decision
> > > process and adjrib-out are required to allow multipe paths there.
> > 
> > Updated diff that includes some minimal support for bgpctl.
> > This add 'bgpctl show rib nei foo path-id 42' as a way to limit which
> > paths to show. Now the RFC itself is very flexible in how path-ids are
> > assigned (it is possible that different prefixes have different path-ids)
> > but the assumtion is that most systems assign path-id in a stable way and
> > so it makes sense to allow to filter on path-id.
> > Apart from that not much changed.
> 
> ok benno@

Thanks a lot :)
 
> Only one thing, I worry that using this while the sending side is not working 
> can lead to
> blackholing of prefixes.
> 

Add-path allows for send / recv to be independent and so this is would me
more of a common issue with add-path. In general having more paths to
select from should help to stop oscilation (e.g. with route reflectors)
but I think this is frequently used to collect routes and still get full
views. Not sure if blackholing is possible (in the end there are more
routes to select from available) but route loops could be an issue.

By default add-path is disabled and that will remain. I agree that
operators need to evaluate carefully what add-path will give them.

Also plan is to finish the MRT support for add-path and then work on
add-path send. So this should arrive soon as well :)

-- 
:wq Claudio

> > 
> > -- 
> > :wq Claudio
> > 
> > Index: bgpctl/bgpctl.8
> > ===
> > RCS file: /cvs/src/usr.sbin/bgpctl/bgpctl.8,v
> > retrieving revision 1.99
> > diff -u -p -r1.99 bgpctl.8
> > --- bgpctl/bgpctl.8 16 Jun 2021 16:24:12 -  1.99
> > +++ bgpctl/bgpctl.8 4 Aug 2021 13:15:53 -
> > @@ -348,6 +348,13 @@ Show RIB memory statistics.
> >  Show only entries from the specified peer.
> >  .It Cm neighbor group Ar description
> >  Show only entries from the specified peer group.
> > +.It Cm path-id Ar pathid
> > +Show only entries which match the specified
> > +.Ar pathid .
> > +Must be used together with either
> > +.Cm neighbor
> > +or
> > +.Cm out .
> >  .It Cm peer-as Ar as
> >  Show all entries with
> >  .Ar as
> > Index: bgpctl/bgpctl.c
> > ===
> > RCS file: /cvs/src/usr.sbin/bgpctl/bgpctl.c,v
> > retrieving revision 1.272
> > diff -u -p -r1.272 bgpctl.c
> > --- bgpctl/bgpctl.c 2 Aug 2021 16:51:39 -   1.272
> > +++ bgpctl/bgpctl.c 4 Aug 2021 15:54:25 -
> > @@ -249,6 +249,7 @@ main(int argc, char *argv[])
> > ribreq.neighbor = neighbor;
> > strlcpy(ribreq.rib, res->rib, sizeof(ribreq.rib));
> > ribreq.aid = res->aid;
> > +   ribreq.path_id = res->pathid;
> > ribreq.flags = res->flags;
> > imsg_compose(ibuf, type, 0, 0, -1, , sizeof(ribreq));
> > break;
> > Index: bgpctl/parser.c
> > ===
> > RCS file: /cvs/src/usr.sbin/bgpctl/parser.c,v
> > retrieving revision 1.106
> > diff -u -p -r1.106 parser.c
> > --- bgpctl/parser.c 16 Feb 2021 08:30:21 -  1.106
> > +++ bgpctl/parser.c 4 Aug 2021 13:08:31 -
> > @@ -61,7 +61,8 @@ enum token_type {
> > RD,
> > FAMILY,
> > RTABLE,
> > -   FILENAME
> > +   FILENAME,
> > +   PATHID,
> >  };
> >  
> >  struct token {
> > @@ -114,6 +115,7 @@ static const struct token t_log[];
> >  static const struct token t_fib_table[];
> >  static const struct token t_show_fib_table[];
> >  stat

Re: bgpd add add-path receive support

2021-08-04 Thread Claudio Jeker
On Fri, Jul 30, 2021 at 12:02:12PM +0200, Claudio Jeker wrote:
> This diff implements the bit to support the receive side of
> RFC7911 - Advertisement of Multiple Paths in BGP.
> 
> I did some basic tests and it works for me. People running route
> collectors should give this a try. The interaction of Add-Path and bgpctl
> probably needs some work. Also the MRT dumper needs to be updated to
> support RFC8050. I have a partial diff for that ready as well.
> 
> Sending out multiple paths will follow in a later step since that is a
> bit more complex. I still need to decide how stable I want to make the
> assigned path_ids for the multiple paths and then changes to the decision
> process and adjrib-out are required to allow multipe paths there.

Updated diff that includes some minimal support for bgpctl.
This add 'bgpctl show rib nei foo path-id 42' as a way to limit which
paths to show. Now the RFC itself is very flexible in how path-ids are
assigned (it is possible that different prefixes have different path-ids)
but the assumtion is that most systems assign path-id in a stable way and
so it makes sense to allow to filter on path-id.
Apart from that not much changed.

-- 
:wq Claudio

Index: bgpctl/bgpctl.8
===
RCS file: /cvs/src/usr.sbin/bgpctl/bgpctl.8,v
retrieving revision 1.99
diff -u -p -r1.99 bgpctl.8
--- bgpctl/bgpctl.8 16 Jun 2021 16:24:12 -  1.99
+++ bgpctl/bgpctl.8 4 Aug 2021 13:15:53 -
@@ -348,6 +348,13 @@ Show RIB memory statistics.
 Show only entries from the specified peer.
 .It Cm neighbor group Ar description
 Show only entries from the specified peer group.
+.It Cm path-id Ar pathid
+Show only entries which match the specified
+.Ar pathid .
+Must be used together with either
+.Cm neighbor
+or
+.Cm out .
 .It Cm peer-as Ar as
 Show all entries with
 .Ar as
Index: bgpctl/bgpctl.c
===
RCS file: /cvs/src/usr.sbin/bgpctl/bgpctl.c,v
retrieving revision 1.272
diff -u -p -r1.272 bgpctl.c
--- bgpctl/bgpctl.c 2 Aug 2021 16:51:39 -   1.272
+++ bgpctl/bgpctl.c 4 Aug 2021 15:54:25 -
@@ -249,6 +249,7 @@ main(int argc, char *argv[])
ribreq.neighbor = neighbor;
strlcpy(ribreq.rib, res->rib, sizeof(ribreq.rib));
ribreq.aid = res->aid;
+   ribreq.path_id = res->pathid;
ribreq.flags = res->flags;
imsg_compose(ibuf, type, 0, 0, -1, , sizeof(ribreq));
break;
Index: bgpctl/parser.c
===
RCS file: /cvs/src/usr.sbin/bgpctl/parser.c,v
retrieving revision 1.106
diff -u -p -r1.106 parser.c
--- bgpctl/parser.c 16 Feb 2021 08:30:21 -  1.106
+++ bgpctl/parser.c 4 Aug 2021 13:08:31 -
@@ -61,7 +61,8 @@ enum token_type {
RD,
FAMILY,
RTABLE,
-   FILENAME
+   FILENAME,
+   PATHID,
 };
 
 struct token {
@@ -114,6 +115,7 @@ static const struct token t_log[];
 static const struct token t_fib_table[];
 static const struct token t_show_fib_table[];
 static const struct token t_communication[];
+static const struct token t_show_rib_path[];
 
 static const struct token t_main[] = {
{ KEYWORD,  "reload",   RELOAD, t_communication},
@@ -178,10 +180,11 @@ static const struct token t_show_rib[] =
{ FLAG, "in",   F_CTL_ADJ_IN,   t_show_rib},
{ FLAG, "out",  F_CTL_ADJ_OUT,  t_show_rib},
{ KEYWORD,  "neighbor", NONE,   t_show_rib_neigh},
+   { KEYWORD,  "ovs",  NONE,   t_show_ovs},
+   { KEYWORD,  "path-id",  NONE,   t_show_rib_path},
{ KEYWORD,  "table",NONE,   t_show_rib_rib},
{ KEYWORD,  "summary",  SHOW_SUMMARY,   t_show_summary},
{ KEYWORD,  "memory",   SHOW_RIB_MEM,   NULL},
-   { KEYWORD,  "ovs",  NONE,   t_show_ovs},
{ FAMILY,   "", NONE,   t_show_rib},
{ PREFIX,   "", NONE,   t_show_prefix},
{ ENDTOKEN, "", NONE,   NULL}
@@ -479,6 +482,11 @@ static const struct token t_show_fib_tab
{ ENDTOKEN, "", NONE,   NULL}
 };
 
+static const struct token t_show_rib_path[] = {
+   { PATHID,   "", NONE,   t_show_rib},
+   { ENDTOKEN, "", NONE,   NULL}
+};
+
 static struct parse_result res;
 
 const struct token *match_token(int *argc, char **argv[],
@@ -748,6 +756,7 @@ match_token(int *argc, char **argv[], co
case PREPSELF:

Re: rpki-client support more http status codes

2021-08-04 Thread Claudio Jeker
On Wed, Aug 04, 2021 at 10:53:39AM +0200, Claudio Jeker wrote:
> This adds a few more HTTP Status codes to the mix of the accepted ones.
> Mainly 100, 103 and 203 are now also accepted. All other codes in the 1xx
> and 2xx are still considered an error since they are not expected from the
> GET request made by the http client. This is a minimal HTTP client and it
> should remain minimal. If a server is sending back something unexpected
> just fail and fall back to rsync.


Update with additional comments for the various status codes.

-- 
:wq Claudio

Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.34
diff -u -p -r1.34 http.c
--- http.c  23 Jul 2021 16:03:47 -  1.34
+++ http.c  4 Aug 2021 15:22:22 -
@@ -865,7 +865,9 @@ http_request(struct http_connection *con
 
 /*
  * Parse the HTTP status line.
- * Return 0 for status codes 200, 301-304, 307-308.
+ * Return 0 for status codes 100, 103, 200, 203, 301-304, 307-308.
+ * The other 1xx and 2xx status codes are explicitly not handled and are
+ * considered an error.
  * Failure codes and other errors return -1.
  * The redirect loop limit is enforced here.
  */
@@ -885,7 +887,7 @@ http_parse_status(struct http_connection
cp++;
 
strlcpy(ststr, cp, sizeof(ststr));
-   status = strtonum(ststr, 200, 599, );
+   status = strtonum(ststr, 100, 599, );
if (errstr != NULL) {
strnvis(gerror, cp, sizeof gerror, VIS_SAFE);
warnx("Error retrieving %s: %s", http_info(conn->host),
@@ -894,19 +896,23 @@ http_parse_status(struct http_connection
}
 
switch (status) {
-   case 301:
-   case 302:
-   case 303:
-   case 307:
-   case 308:
+   case 301:   /* Redirect: moved permanently */
+   case 302:   /* Redirect: found / moved temporarily */
+   case 303:   /* Redirect: see other */
+   case 307:   /* Redirect: temporary redirect */
+   case 308:   /* Redirect: permanent redirect */
if (conn->req->redirect_loop++ > 10) {
warnx("%s: Too many redirections requested",
http_info(conn->host));
return -1;
}
/* FALLTHROUGH */
-   case 200:
-   case 304:
+   case 100:   /* Informational: continue (ignored) */
+   case 103:   /* Informational: early hints (ignored) */
+   /* FALLTHROUGH */
+   case 200:   /* Success: OK */
+   case 203:   /* Success: non-authoritative information (proxy) */
+   case 304:   /* Redirect: not modified */
conn->status = status;
break;
default:
@@ -931,6 +937,14 @@ http_isredirect(struct http_connection *
return 0;
 }
 
+static inline int
+http_isok(struct http_connection *conn)
+{
+   if (conn->status >= 200 && conn->status < 300)
+   return 1;
+   return 0;
+}
+
 static void
 http_redirect(struct http_connection *conn)
 {
@@ -1165,7 +1179,7 @@ again:
}
 
/* Check status header and decide what to do next */
-   if (conn->status == 200 || http_isredirect(conn)) {
+   if (http_isok(conn) || http_isredirect(conn)) {
if (http_isredirect(conn))
http_redirect(conn);
 
@@ -1174,6 +1188,8 @@ again:
else
conn->state = STATE_RESPONSE_DATA;
goto again;
+   } else if (conn->status == 100 || conn->status == 103) {
+   conn->state = STATE_RESPONSE_STATUS;
} else if (conn->status == 304) {
return http_done(conn, HTTP_NOT_MOD);
}



rpki-client support more http status codes

2021-08-04 Thread Claudio Jeker
This adds a few more HTTP Status codes to the mix of the accepted ones.
Mainly 100, 103 and 203 are now also accepted. All other codes in the 1xx
and 2xx are still considered an error since they are not expected from the
GET request made by the http client. This is a minimal HTTP client and it
should remain minimal. If a server is sending back something unexpected
just fail and fall back to rsync.

OK?
-- 
:wq Claudio

Index: http.c
===
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
retrieving revision 1.34
diff -u -p -r1.34 http.c
--- http.c  23 Jul 2021 16:03:47 -  1.34
+++ http.c  4 Aug 2021 08:48:01 -
@@ -865,7 +865,9 @@ http_request(struct http_connection *con
 
 /*
  * Parse the HTTP status line.
- * Return 0 for status codes 200, 301-304, 307-308.
+ * Return 0 for status codes 100, 103, 200, 203, 301-304, 307-308.
+ * The other 1xx and 2xx status codes are explicitly not handled and are
+ * considered an error.
  * Failure codes and other errors return -1.
  * The redirect loop limit is enforced here.
  */
@@ -885,7 +887,7 @@ http_parse_status(struct http_connection
cp++;
 
strlcpy(ststr, cp, sizeof(ststr));
-   status = strtonum(ststr, 200, 599, );
+   status = strtonum(ststr, 100, 599, );
if (errstr != NULL) {
strnvis(gerror, cp, sizeof gerror, VIS_SAFE);
warnx("Error retrieving %s: %s", http_info(conn->host),
@@ -905,7 +907,11 @@ http_parse_status(struct http_connection
return -1;
}
/* FALLTHROUGH */
+   case 100:
+   case 103:
+   /* FALLTHROUGH */
case 200:
+   case 203:
case 304:
conn->status = status;
break;
@@ -931,6 +937,14 @@ http_isredirect(struct http_connection *
return 0;
 }
 
+static inline int
+http_isok(struct http_connection *conn)
+{
+   if (conn->status >= 200 && conn->status < 300)
+   return 1;
+   return 0;
+}
+
 static void
 http_redirect(struct http_connection *conn)
 {
@@ -1165,7 +1179,7 @@ again:
}
 
/* Check status header and decide what to do next */
-   if (conn->status == 200 || http_isredirect(conn)) {
+   if (http_isok(conn) || http_isredirect(conn)) {
if (http_isredirect(conn))
http_redirect(conn);
 
@@ -1174,6 +1188,8 @@ again:
else
conn->state = STATE_RESPONSE_DATA;
goto again;
+   } else if (conn->status == 100 || conn->status == 103) {
+   conn->state = STATE_RESPONSE_STATUS;
} else if (conn->status == 304) {
return http_done(conn, HTTP_NOT_MOD);
}



bgpd add add-path receive support

2021-07-30 Thread Claudio Jeker
This diff implements the bit to support the receive side of
RFC7911 - Advertisement of Multiple Paths in BGP.

I did some basic tests and it works for me. People running route
collectors should give this a try. The interaction of Add-Path and bgpctl
probably needs some work. Also the MRT dumper needs to be updated to
support RFC8050. I have a partial diff for that ready as well.

Sending out multiple paths will follow in a later step since that is a
bit more complex. I still need to decide how stable I want to make the
assigned path_ids for the multiple paths and then changes to the decision
process and adjrib-out are required to allow multipe paths there.

-- 
:wq Claudio

Index: parse.y
===
RCS file: /cvs/src/usr.sbin/bgpd/parse.y,v
retrieving revision 1.417
diff -u -p -r1.417 parse.y
--- parse.y 17 Jun 2021 16:05:26 -  1.417
+++ parse.y 22 Jun 2021 10:48:50 -
@@ -204,7 +204,8 @@ typedef struct {
 %token GROUP NEIGHBOR NETWORK
 %token EBGP IBGP
 %token LOCALAS REMOTEAS DESCR LOCALADDR MULTIHOP PASSIVE MAXPREFIX RESTART
-%token ANNOUNCE CAPABILITIES REFRESH AS4BYTE CONNECTRETRY ENHANCED
+%token ANNOUNCE CAPABILITIES REFRESH AS4BYTE CONNECTRETRY ENHANCED ADDPATH
+%token SEND RECV
 %token DEMOTE ENFORCE NEIGHBORAS ASOVERRIDE REFLECTOR DEPEND DOWN
 %token DUMP IN OUT SOCKET RESTRICTED
 %token LOG TRANSPARENT
@@ -1455,6 +1456,16 @@ peeropts : REMOTEAS as4number{
| ANNOUNCE AS4BYTE yesno {
curpeer->conf.capabilities.as4byte = $3;
}
+   | ANNOUNCE ADDPATH RECV yesno {
+   int8_t *ap = curpeer->conf.capabilities.add_path;
+   u_int8_t i;
+
+   for (i = 0; i < AID_MAX; i++)
+   if ($4)
+   *ap++ |= CAPA_AP_RECV;
+   else
+   *ap++ &= ~CAPA_AP_RECV;
+   }
| EXPORT NONE {
curpeer->conf.export_type = EXPORT_NONE;
}
@@ -2878,6 +2889,7 @@ lookup(char *s)
{ "AS", AS},
{ "IPv4",   IPV4},
{ "IPv6",   IPV6},
+   { "add-path",   ADDPATH},
{ "ah", AH},
{ "allow",  ALLOW},
{ "announce",   ANNOUNCE},
@@ -2965,6 +2977,7 @@ lookup(char *s)
{ "quick",  QUICK},
{ "rd", RD},
{ "rde",RDE},
+   { "recv",   RECV},
{ "refresh",REFRESH },
{ "reject", REJECT},
{ "remote-as",  REMOTEAS},
@@ -2978,6 +2991,7 @@ lookup(char *s)
{ "rtlabel",RTLABEL},
{ "rtr",RTR},
{ "self",   SELF},
+   { "send",   SEND},
{ "set",SET},
{ "socket", SOCKET },
{ "source-as",  SOURCEAS},
Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.531
diff -u -p -r1.531 rde.c
--- rde.c   27 Jul 2021 07:50:01 -  1.531
+++ rde.c   30 Jul 2021 09:16:53 -
@@ -50,10 +50,10 @@ void rde_dispatch_imsg_parent(struct i
 voidrde_dispatch_imsg_rtr(struct imsgbuf *);
 voidrde_dispatch_imsg_peer(struct rde_peer *, void *);
 voidrde_update_dispatch(struct rde_peer *, struct imsg *);
-int rde_update_update(struct rde_peer *, struct filterstate *,
+int rde_update_update(struct rde_peer *, u_int32_t,
+ struct filterstate *, struct bgpd_addr *, u_int8_t);
+voidrde_update_withdraw(struct rde_peer *, u_int32_t,
 struct bgpd_addr *, u_int8_t);
-voidrde_update_withdraw(struct rde_peer *, struct bgpd_addr *,
-u_int8_t);
 int rde_attr_parse(u_char *, u_int16_t, struct rde_peer *,
 struct filterstate *, struct mpattr *);
 int rde_attr_add(struct filterstate *, u_char *, u_int16_t);
@@ -1183,7 +1183,7 @@ rde_update_dispatch(struct rde_peer *pee
u_int16_tattrpath_len;
u_int16_tnlri_len;
u_int8_t aid, prefixlen, safi, subtype;
-   u_int32_tfas;
+   u_int32_tfas, pathid;
 
p = imsg->data;
 
@@ -1288,6 +1288,21 @@ rde_update_dispatch(struct rde_peer *pee
goto done;
}
 
+   if (peer_has_add_path(peer, AID_INET, CAPA_AP_RECV)) {
+  

Re: Do not spin on the NET_LOCK() in kqueue

2021-07-29 Thread Claudio Jeker
On Thu, Jul 29, 2021 at 09:51:43AM +0200, Martin Pieuchot wrote:
> On 26/07/21(Mon) 09:23, Martin Pieuchot wrote:
> > On 26/07/21(Mon) 08:55, Martin Pieuchot wrote:
> > > On 21/07/21(Wed) 10:18, Martin Pieuchot wrote:
> > > > On 11/07/21(Sun) 14:45, Visa Hankala wrote:
> > > > > On Sat, Jul 10, 2021 at 05:26:57PM +0200, Martin Pieuchot wrote:
> > > > > > One of the reasons for the drop of performances in the kqueue-based
> > > > > > poll/select is the fact that kqueue filters are called up to 3 times
> > > > > > per syscall and that they all spin on the NET_LOCK() for TCP/UDP
> > > > > > packets.
> > > > > > 
> > > > > > Diff below is a RFC for improving the situation.
> > > > > > 
> > > > > > socket kqueue filters mainly check for the amount of available 
> > > > > > items to
> > > > > > read/write.  This involves comparing various socket buffer fields 
> > > > > > (sb_cc,
> > > > > > sb_lowat, etc).  The diff below introduces a new mutex to serialize
> > > > > > updates of those fields with reads in the kqueue filters.
> > > > > > 
> > > > > > Since these fields are always modified with the socket lock held, 
> > > > > > either
> > > > > > the mutex or the solock are enough to have a coherent view of them.
> > > > > > Note that either of these locks is necessary only if multiple fields
> > > > > > have to be read (like in sbspace()).
> > > > > > 
> > > > > > Other per-socket fields accessed in the kqueue filters are never
> > > > > > combined (with &&) to determine a condition.  So assuming it is 
> > > > > > fine to
> > > > > > read register-sized fields w/o the socket lock we can safely remove 
> > > > > > it
> > > > > > there.
> > > > > > 
> > > > > > Could such mutex also be used to serialize klist updates?
> > > > > 
> > > > > I think the lock should be such that it can serialize socket klists.
> > > > > 
> > > > > As the main motivator for this change is kqueue, the viability of 
> > > > > using
> > > > > the mutex for the klist locking should be checked now. The mutex has 
> > > > > to
> > > > > be held whenever calling KNOTE() on sb_sel.si_note, or selwakeup() on
> > > > > sb_sel. Then the socket f_event callbacks will not need to lock the
> > > > > mutex themselves.
> > > > > 
> > > > > I had a diff that serialized socket klists using solock(). It did not
> > > > > work well because it increased lock contention, especially when using
> > > > > kqueue as backend for poll(2) and select(2). The diff is not even
> > > > > correct any longer since recent changes to socket locking have
> > > > > introduced new lock order constraints that conflict with it.
> > > > 
> > > > Updated diff below does that.  It also uses a single per-socket mutex as
> > > > suggested by bluhm@.
> > > > 
> > > > Note that as long poll(2) & select(2) use the current implementation a
> > > > KERNEL_LOCK()/UNLOCK() dance is necessary in sowakeup().  The goal of
> > > > this change combined with the poll/select rewrite is to get rid of this
> > > > dance.
> > > 
> > > Updated diff after recent commits, more comments?  Oks?
> > 
> > Previous diff had a double mtx_enter() in filt_fifowrite_common(), this
> > one use the *locked() version of sbspace() to prevent it.
> 
> New diff fixing a locking dance pointed out by visa@.

I think the diff is fine. It does show a few places that should be
improved on. e.g. some of those macros in socketvar.h should be
implemented as functions or inline functions or the fact that high level
code (e.g. TCP) is directly manipulating socket buffer internals.

I'm not a big fan of either-this-lock-or-that-lock locking schemes. They
are just confusing. I guess in the long run the mutex should be used all
the time.

Maybe it is best to commit it and look for any fallout.

> Index: kern/uipc_socket.c
> ===
> RCS file: /cvs/src/sys/kern/uipc_socket.c,v
> retrieving revision 1.264
> diff -u -p -r1.264 uipc_socket.c
> --- kern/uipc_socket.c26 Jul 2021 05:51:13 -  1.264
> +++ kern/uipc_socket.c29 Jul 2021 07:31:32 -
> @@ -84,7 +84,7 @@ int filt_solistenprocess(struct knote *k
>  int  filt_solisten_common(struct knote *kn, struct socket *so);
>  
>  const struct filterops solisten_filtops = {
> - .f_flags= FILTEROP_ISFD,
> + .f_flags= FILTEROP_ISFD | FILTEROP_MPSAFE,
>   .f_attach   = NULL,
>   .f_detach   = filt_sordetach,
>   .f_event= filt_solisten,
> @@ -93,7 +93,7 @@ const struct filterops solisten_filtops 
>  };
>  
>  const struct filterops soread_filtops = {
> - .f_flags= FILTEROP_ISFD,
> + .f_flags= FILTEROP_ISFD | FILTEROP_MPSAFE,
>   .f_attach   = NULL,
>   .f_detach   = filt_sordetach,
>   .f_event= filt_soread,
> @@ -102,7 +102,7 @@ const struct filterops soread_filtops = 
>  };
>  
>  const struct filterops sowrite_filtops = {
> - .f_flags= FILTEROP_ISFD,
> + .f_flags= 

Re: rpki-client: adjust HTTP/1.1 request string

2021-07-23 Thread Claudio Jeker
On Fri, Jul 23, 2021 at 05:28:33PM +0200, Sebastian Benoit wrote:
> Job Snijders(j...@openbsd.org) on 2021.07.23 15:23:49 +:
> > Hi all,
> > 
> > Based on suggestions from Julian Reschke.
> > 
> > * "Connection: keep-alive" isn't needed, as the HTTP 1.1 default is to
> >   use persistent connections (RFC 7230, section 6.3).
> > 
> > * "Host" is recommended to be in the front.
> > 
> > * "Accept-Encoding: identity" makes it clear to the server compression
> >   encodings are not supported.
> >   https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding
> > 
> > OK?
> 
> reasonable.
> ok benno@

Agreed, I think the time of possible 'Connection: close' is over.

The spec for Accept-Encoding is whako, the default is no-preference
and therefor any encoding is OK. Who came up with such a bad default?

Because of this OK claudio@
 
> > 
> > Kind regards,
> > 
> > Job
> > 
> > Index: http.c
> > ===
> > RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
> > retrieving revision 1.33
> > diff -u -p -r1.33 http.c
> > --- http.c  10 May 2021 15:12:33 -  1.33
> > +++ http.c  23 Jul 2021 15:13:35 -
> > @@ -847,9 +847,10 @@ http_request(struct http_connection *con
> > conn->bufpos = 0;
> > if ((r = asprintf(>buf,
> > "GET /%s HTTP/1.1\r\n"
> > -   "Connection: keep-alive\r\n"
> > +   "Host: %s\r\n"
> > +   "Accept-Encoding: identity\r\n"
> > "User-Agent: " HTTP_USER_AGENT "\r\n"
> > -   "Host: %s\r\n%s\r\n",
> > +   "%s\r\n",
> > epath, host,
> > modified_since ? modified_since : "")) == -1)
> > err(1, NULL);
> > 
> 

-- 
:wq Claudio



bgpd support for RFC9072

2021-07-19 Thread Claudio Jeker
This adds support for RFC9072: Extended Optional Parameters Length for BGP
OPEN Message. I did not add any knobs to force the new format. Seems to
work for me (tested both formats with bgpd). The other ususal suspects
have no support yet so lets see.

-- 
:wq Claudio

Index: bgpd.8
===
RCS file: /cvs/src/usr.sbin/bgpd/bgpd.8,v
retrieving revision 1.69
diff -u -p -r1.69 bgpd.8
--- bgpd.8  17 Jun 2021 16:05:25 -  1.69
+++ bgpd.8  19 Jul 2021 15:12:41 -
@@ -458,6 +458,14 @@ has been started.
 .%R RFC 8326
 .%T Graceful BGP Session Shutdown
 .Re
+.Pp
+.Rs
+.%A E. Chen
+.%A J. Scudder
+.%D July 2021
+.%R RFC 9072
+.%T Extended Optional Parameters Length for BGP OPEN Message
+.Re
 .Sh HISTORY
 The
 .Nm
Index: session.c
===
RCS file: /cvs/src/usr.sbin/bgpd/session.c,v
retrieving revision 1.422
diff -u -p -r1.422 session.c
--- session.c   24 Jun 2021 09:26:18 -  1.422
+++ session.c   19 Jul 2021 15:20:21 -
@@ -1413,14 +1413,13 @@ session_open(struct peer *p)
struct bgp_msg  *buf;
struct ibuf *opb;
struct msg_open  msg;
-   u_int16_tlen;
-   u_int8_t i, op_type, optparamlen = 0;
-   int  errs = 0;
+   u_int16_tlen, optparamlen = 0;
+   u_int8_t i, op_type;
+   int  errs = 0, extlen = 0;
int  mpcapa = 0;
 
 
-   if ((opb = ibuf_dynamic(0, UCHAR_MAX - sizeof(op_type) -
-   sizeof(optparamlen))) == NULL) {
+   if ((opb = ibuf_dynamic(0, UINT16_MAX - 3)) == NULL) {
bgp_fsm(p, EVNT_CON_FATAL);
return;
}
@@ -1491,9 +1490,18 @@ session_open(struct peer *p)
if (p->capa.ann.enhanced_rr)/* no data */
errs += session_capa_add(opb, CAPA_ENHANCED_RR, 0);
 
-   if (ibuf_size(opb))
-   optparamlen = ibuf_size(opb) + sizeof(op_type) +
-   sizeof(optparamlen);
+   optparamlen = ibuf_size(opb);
+   if (optparamlen == 0) {
+   /* nothing */
+   } else if (optparamlen + 2 >= 255) {
+   /* RFC9072: 2 byte lenght instead of 1 + 3 byte extra header */
+   optparamlen += sizeof(op_type) + 2 + 3;
+   msg.optparamlen = 255;
+   extlen = 1;
+   } else {
+   optparamlen += sizeof(op_type) + 1;
+   msg.optparamlen = optparamlen;
+   }
 
len = MSGSIZE_OPEN_MIN + optparamlen;
if (errs || (buf = session_newmsg(OPEN, len)) == NULL) {
@@ -1509,19 +1517,34 @@ session_open(struct peer *p)
else
msg.holdtime = htons(conf->holdtime);
msg.bgpid = conf->bgpid;/* is already in network byte order */
-   msg.optparamlen = optparamlen;
 
errs += ibuf_add(buf->buf, , sizeof(msg.version));
errs += ibuf_add(buf->buf, , sizeof(msg.myas));
errs += ibuf_add(buf->buf, , sizeof(msg.holdtime));
errs += ibuf_add(buf->buf, , sizeof(msg.bgpid));
-   errs += ibuf_add(buf->buf, , sizeof(msg.optparamlen));
+   errs += ibuf_add(buf->buf, , 1);
+
+   if (extlen) {
+   /* write RFC9072 extra header */
+   u_int16_t op_extlen = htons(optparamlen - 3);
+   op_type = OPT_PARAM_EXT_LEN;
+   errs += ibuf_add(buf->buf, _type, 1);
+   errs += ibuf_add(buf->buf, _extlen, 2);
+   }
 
if (optparamlen) {
op_type = OPT_PARAM_CAPABILITIES;
-   optparamlen = ibuf_size(opb);
errs += ibuf_add(buf->buf, _type, sizeof(op_type));
-   errs += ibuf_add(buf->buf, , sizeof(optparamlen));
+
+   optparamlen = ibuf_size(opb);
+   if (extlen) {
+   /* RFC9072: 2-byte extended length */
+   u_int16_t op_extlen = htons(optparamlen);
+   errs += ibuf_add(buf->buf, _extlen, 2);
+   } else {
+   u_int8_t op_len = optparamlen;
+   errs += ibuf_add(buf->buf, _len, 1);
+   }
errs += ibuf_add(buf->buf, opb->buf, ibuf_size(opb));
}
 
@@ -2042,8 +2065,8 @@ parse_open(struct peer *peer)
u_int16_tshort_as, msglen;
u_int16_tholdtime, oholdtime, myholdtime;
u_int32_tas, bgpid;
-   u_int8_t optparamlen, plen;
-   u_int8_t op_type, op_len;
+   u_int16_toptparamlen, extlen, plen, op_len;
+   u_int8_t op_type;
 
p = peer->rbuf->rptr;
p += MSGSIZE_HEADER_MARKER;
@@ -2116,41 +2139,56 @@ parse_open(struct peer *peer)
}
peer->remote_bgpid = bgpid;
 
-   memcpy(, p, sizeof(optparamlen));
-   p += sizeof(optparamlen);
+  

Re: bgpd refactor struct prefix

2021-07-14 Thread Claudio Jeker
On Tue, Jun 29, 2021 at 12:00:24PM +0200, Claudio Jeker wrote:
> This diff moves the rib_entry pointer re into the union to safe some
> space. For add-path I need to add a few more u_int32_t and that would
> blow the size of struct prefix from 128 to 132 bytes. malloc would round
> that up to 256bytes and that is bad for the struct that is allocted in
> millions in bgpd.
> 
> To make this somewhat save introduce PREFIX_FLAG_ADJOUT to mark prefixes
> that live in the adj-rib-out. Those prefixes can not access the re pointer
> also use a wrapper prefix_re() which returns the re pointer or NULL.
> Also add some assertions to make sure that prefixes don't end up in the
> wrong tree.
> 
> This change shrinks the struct back to 120bytes and gives me the space
> needed for add-path.
> 
> Please test

Ping

-- 
:wq Claudio

Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.530
diff -u -p -r1.530 rde.c
--- rde.c   25 Jun 2021 09:25:48 -  1.530
+++ rde.c   29 Jun 2021 08:28:33 -
@@ -2298,6 +2298,7 @@ rde_dump_rib_as(struct prefix *p, struct
struct ibuf *wbuf;
struct attr *a;
struct nexthop  *nexthop;
+   struct rib_entry*re;
void*bp;
time_t   staletime;
size_t   aslen;
@@ -2330,7 +2331,8 @@ rde_dump_rib_as(struct prefix *p, struct
rib.origin = asp->origin;
rib.validation_state = p->validation_state;
rib.flags = 0;
-   if (p->re != NULL && p->re->active == p)
+   re = prefix_re(p);
+   if (re != NULL && re->active == p)
rib.flags |= F_PREF_ACTIVE;
if (!prefix_peer(p)->conf.ebgp)
rib.flags |= F_PREF_INTERNAL;
@@ -2412,14 +2414,16 @@ static void
 rde_dump_filter(struct prefix *p, struct ctl_show_rib_request *req)
 {
struct rde_aspath   *asp;
+   struct rib_entry*re;
 
if (!rde_match_peer(prefix_peer(p), >neighbor))
return;
 
asp = prefix_aspath(p);
+   re = prefix_re(p);
if (asp == NULL)/* skip pending withdraw in Adj-RIB-Out */
return;
-   if ((req->flags & F_CTL_ACTIVE) && p->re->active != p)
+   if ((req->flags & F_CTL_ACTIVE) && re != NULL && re->active != p)
return;
if ((req->flags & F_CTL_INVALID) &&
(asp->flags & F_ATTR_PARSE_ERR) == 0)
Index: rde.h
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.h,v
retrieving revision 1.240
diff -u -p -r1.240 rde.h
--- rde.h   17 Jun 2021 16:05:26 -  1.240
+++ rde.h   29 Jun 2021 08:33:18 -
@@ -56,10 +56,10 @@ struct rib {
struct filter_head  *in_rules_tmp;
u_int   rtableid;
u_int   rtableid_tmp;
+   enum reconf_action  state, fibstate;
+   u_int16_t   id;
u_int16_t   flags;
u_int16_t   flags_tmp;
-   u_int16_t   id;
-   enum reconf_action  state, fibstate;
 };
 
 #define RIB_ADJ_IN 0
@@ -317,13 +317,13 @@ struct prefix {
union {
struct {
LIST_ENTRY(prefix)   rib, nexthop;
+   struct rib_entry*re;
} list;
struct {
RB_ENTRY(prefix) index, update;
} tree;
}entry;
struct pt_entry *pt;
-   struct rib_entry*re;
struct rde_aspath   *aspath;
struct rde_community*communities;
struct rde_peer *peer;
@@ -338,6 +338,7 @@ struct prefix {
 #definePREFIX_FLAG_DEAD0x04/* locked but removed */
 #definePREFIX_FLAG_STALE   0x08/* stale entry (graceful 
reload) */
 #definePREFIX_FLAG_MASK0x0f/* mask for the prefix types */
+#definePREFIX_FLAG_ADJOUT  0x10/* prefix is in the adj-out rib 
*/
 #definePREFIX_NEXTHOP_LINKED   0x40/* prefix is linked onto 
nexthop list */
 #definePREFIX_FLAG_LOCKED  0x80/* locked by rib walker */
 };
@@ -637,6 +638,14 @@ static inline u_int8_t
 prefix_vstate(struct prefix *p)
 {
return (p->validation_state & ROA_MASK);
+}
+
+static inline struct rib_entry *
+prefix_re(struct prefix *p)
+{
+   if (p->flags & PREFIX_FLAG_ADJOUT)
+   return NULL;
+   return (p->entry.list.re);
 }
 
 voidnexthop_init(u_int32_t);
Index: rde_rib.c
===

rsync getopt_long cleanup

2021-07-13 Thread Claudio Jeker
I never really liked the getopt_long definitions in rsync. Too much magic
and chaos.

This moves the table out of main to gain some more space and to make it a
proper read-only object. Because of this struct opts also needs to become
a global but that is OK.

Clean up the required_argument options that have no short from. Instead of
small numbers use some defines and make the values larger than any char
value (I chose 1000 and up).

Fix --no-motd, it is just a flag setting a value. So just use the
getopt_long() method for doing that.

Sort the options alphabetically with the exception of no-XYZ options which
I added below the XYZ option itself.

IMO the result is better than what was there before.
-- 
:wq Claudio

Index: main.c
===
RCS file: /cvs/src/usr.bin/rsync/main.c,v
retrieving revision 1.55
diff -u -p -r1.55 main.c
--- main.c  30 Jun 2021 13:10:04 -  1.55
+++ main.c  13 Jul 2021 17:54:13 -
@@ -269,61 +269,67 @@ fargs_parse(size_t argc, char *argv[], s
return f;
 }
 
+static struct opts  opts;
+
+#define OP_ADDRESS 1000
+#define OP_PORT1001
+#define OP_RSYNCPATH   1002
+#define OP_TIMEOUT 1003
+#define OP_VERSION 1004
+
+const struct option lopts[] = {
+{ "address",   required_argument, NULL,OP_ADDRESS },
+{ "archive",   no_argument,NULL,   'a' },
+{ "compress",  no_argument,NULL,   'z' },
+{ "del",   no_argument,,  1 },
+{ "delete",no_argument,,  1 },
+{ "devices",   no_argument,,  1 },
+{ "no-devices",no_argument,,  0 },
+{ "dry-run",   no_argument,_run,  1 },
+{ "group", no_argument,_gids,1 },
+{ "no-group",  no_argument,_gids,0 },
+{ "help",  no_argument,NULL,   'h' },
+{ "links", no_argument,_links,   1 },
+{ "no-links",  no_argument,_links,   0 },
+{ "no-motd",   no_argument,_motd,  1 },
+{ "numeric-ids",   no_argument,_ids,  1 },
+{ "owner", no_argument,_uids,1 },
+{ "no-owner",  no_argument,_uids,0 },
+{ "perms", no_argument,_perms,   1 },
+{ "no-perms",  no_argument,_perms,   0 },
+{ "port",  required_argument, NULL,OP_PORT },
+{ "recursive", no_argument,,1 },
+{ "no-recursive",  no_argument,,0 },
+{ "rsh",   required_argument, NULL,'e' },
+{ "rsync-path",required_argument, NULL,OP_RSYNCPATH },
+{ "sender",no_argument,,   1 },
+{ "server",no_argument,,   1 },
+{ "specials",  no_argument,, 1 },
+{ "no-specials",   no_argument,, 0 },
+{ "timeout",   required_argument, NULL,OP_TIMEOUT },
+{ "times", no_argument,_times,   1 },
+{ "no-times",  no_argument,_times,   0 },
+{ "verbose",   no_argument,,   1 },
+{ "no-verbose",no_argument,,   0 },
+{ "version",   no_argument,NULL,   OP_VERSION },
+{ NULL,0,  NULL,   0 }
+};
+
 int
 main(int argc, char *argv[])
 {
-   struct opts  opts;
pid_tchild;
int  fds[2], sd = -1, rc, c, st, i;
struct sess   sess;
struct fargs*fargs;
char**args;
const char  *errstr;
-   const struct option  lopts[] = {
-   { "port",   required_argument, NULL,3 },
-   { "rsh",required_argument, NULL,'e' },
-   { "rsync-path", required_argument, NULL,1 },
-   { "sender", no_argument,,   1 },
-   { "server", no_argument,,   1 },
-   { "dry-run",no_argument,_run,  1 },
-   { "version",no_argument,NULL,   2 },
-   { "archive",no_argument,NULL,   'a' },
-   { "help",   no_argument,NULL,   'h' },
-   { "compress",   no_argument,NULL,   'z' },
-   { "del",no_argument,,  1 },
-   { "delete", no_argument,,  1 },
-   { "devices",no_argument,,  1 },
-   { "no-devices", no_argument,,  0 },
-   { "group",  no_argument,_gids,1 },
-   { "no-group",   no_argument,_gids,0 },
-   { "links",  

bgpctl add support for RFC8050 (add-path support for MRT parser)

2021-07-13 Thread Claudio Jeker
This diff adds support to read MRT files using the new introduced _ADDPATH
types as defined in RFC8050. I also started adding MRT support to bgpd but
that depends on ADD-PATH itself.

There are a few gotchas, especially the MRT_DUMP_V2 RIB_GENERIC_ADDPATH
handling is different from all other RIB entry handling. This is a major
pain point for bgpd less so for the bgpctl parser.

Some MRT update dumps that can be downloaded and use ADDPATH do actually
use the BGP4MP_MESSAGE _ADDPATH variant for non-addpath enabled sessions.
The update messages can not be parsed because the NLRI encoding is incorrect.

I tested with a few RIB and UPDATE dumps from RIS, route-views and other
open collectors and it works for me.
-- 
:wq Claudio

Index: usr.sbin/bgpctl/bgpctl.c
===
RCS file: /cvs/src/usr.sbin/bgpctl/bgpctl.c,v
retrieving revision 1.269
diff -u -p -r1.269 bgpctl.c
--- usr.sbin/bgpctl/bgpctl.c16 Jun 2021 16:24:11 -  1.269
+++ usr.sbin/bgpctl/bgpctl.c13 Jul 2021 13:20:51 -
@@ -470,7 +470,7 @@ show(struct imsg *imsg, struct parse_res
warnx("bad IMSG_CTL_SHOW_RIB_ATTR received");
break;
}
-   output->attr(imsg->data, ilen, res->flags);
+   output->attr(imsg->data, ilen, res->flags, 0);
break;
case IMSG_CTL_SHOW_RIB_MEM:
if (imsg->hdr.len < IMSG_HEADER_SIZE + sizeof(stats))
@@ -1150,6 +1150,10 @@ show_mrt_dump(struct mrt_rib *mr, struct
ctl.local_pref = mre->local_pref;
ctl.med = mre->med;
/* weight is not part of the mrt dump so it can't be set */
+   if (mr->add_path) {
+   ctl.flags |= F_PREF_PATH_ID;
+   ctl.path_id = mre->path_id;
+   }
 
if (mre->peer_idx < mp->npeers) {
ctl.remote_addr = mp->peers[mre->peer_idx].addr;
@@ -1195,7 +1199,7 @@ show_mrt_dump(struct mrt_rib *mr, struct
if (req->flags & F_CTL_DETAIL) {
for (j = 0; j < mre->nattrs; j++)
output->attr(mre->attrs[j].attr,
-   mre->attrs[j].attr_len, req->flags);
+   mre->attrs[j].attr_len, req->flags, 0);
}
}
 }
@@ -1211,6 +1215,10 @@ network_mrt_dump(struct mrt_rib *mr, str
time_t   now;
u_int16_ti, j;
 
+   /* can't announce more than one path so ignore add-path */
+   if (mr->add_path)
+   return;
+
now = time(NULL);
for (i = 0; i < mr->nentries; i++) {
mre = >entries[i];
@@ -1586,10 +1594,11 @@ show_mrt_notification(u_char *p, u_int16
 
 /* XXX this function does not handle JSON output */
 static void
-show_mrt_update(u_char *p, u_int16_t len, int reqflags)
+show_mrt_update(u_char *p, u_int16_t len, int reqflags, int addpath)
 {
struct bgpd_addr prefix;
int pos;
+   u_int32_t pathid;
u_int16_t wlen, alen;
u_int8_t prefixlen;
 
@@ -1609,12 +1618,25 @@ show_mrt_update(u_char *p, u_int16_t len
if (wlen > 0) {
printf("\n Withdrawn prefixes:");
while (wlen > 0) {
+   if (addpath) {
+   if (wlen <= sizeof(pathid)) {
+   printf("bad withdraw prefix");
+   return;
+   }
+   memcpy(, p, sizeof(pathid));
+   pathid = ntohl(pathid);
+   p += sizeof(pathid);
+   len -= sizeof(pathid);
+   wlen -= sizeof(pathid);
+   }
if ((pos = nlri_get_prefix(p, wlen, ,
)) == -1) {
printf("bad withdraw prefix");
return;
}
printf(" %s/%u", log_addr(), prefixlen);
+   if (addpath)
+   printf(" path-id %u", pathid);
p += pos;
len -= pos;
wlen -= pos;
@@ -1655,7 +1677,7 @@ show_mrt_update(u_char *p, u_int16_t len
attrlen += 1 + 2;
}
 
-   output->attr(p, attrlen, reqflags);
+   output->attr(p, attrlen, reqflags, addpath);
p += attrlen;
alen -= attrlen;
len -= attrlen;
@@ -1664,12 +1686,24 @@ show_mrt_update(u_char *p, u_int16_t len
if (len > 0) {
printf("NLRI prefixes:");
while (len > 0) {
+   if (addpath) {
+ 

Re: rsync fix symlink discovery

2021-07-06 Thread Claudio Jeker
On Mon, Jul 05, 2021 at 08:11:12PM -0900, Philip Guenther wrote:
> Based on the fts_open(3) manpage and other base source usage, shouldn't
> this use fts_accpath instead of fts_name?

Yes this should use fts_accpath.  I knew there was something different
than fts_name. I just missed it when I read the man page.
 
> The use of fts_statp in this code seems a bit loose vs ftp_info: instead of
> using S_ISLNK() on fts_statp I would expect this code to check for fts_info
> == FTS_SL: according the manpage fts_statp's value is undefined for various
> values of fts_info.
 
I'll have a look at this. I do agree that the code should be improved
overall.
 
> Philip Guenther
> 
> On Fri, Jul 2, 2021 at 4:46 AM Claudio Jeker 
> wrote:
> 
> > Hit this today while doing some tests. symlink_read() needs to use just
> > the filename and not the full path because fts_read(3) does chdir
> > internally.
> >
> > Without this I got:
> > openrsync: error: ./obj/openrsync.1: readlink: No such file or directory
> > openrsync: error: symlink_read
> > openrsync: error: flist_gen_dirent
> > openrsync: error: flist_gen
> > openrsync: error: rsync_sender
> >
> > --
> > :wq Claudio
> >
> > Index: flist.c
> > ===
> > RCS file: /cvs/src/usr.bin/rsync/flist.c,v
> > retrieving revision 1.32
> > diff -u -p -r1.32 flist.c
> > --- flist.c 30 Jun 2021 13:10:04 -  1.32
> > +++ flist.c 2 Jul 2021 13:14:01 -
> > @@ -972,7 +972,7 @@ flist_gen_dirent(struct sess *sess, char
> > /* Optionally copy link information. */
> >
> > if (S_ISLNK(ent->fts_statp->st_mode)) {
> > -   f->link = symlink_read(f->path);
> > +   f->link = symlink_read(ent->fts_name);
> > if (f->link == NULL) {
> > ERRX1("symlink_read");
> > goto out;
> >
> >

-- 
:wq Claudio



rsync fix symlink discovery

2021-07-02 Thread Claudio Jeker
Hit this today while doing some tests. symlink_read() needs to use just
the filename and not the full path because fts_read(3) does chdir
internally.

Without this I got:
openrsync: error: ./obj/openrsync.1: readlink: No such file or directory
openrsync: error: symlink_read
openrsync: error: flist_gen_dirent
openrsync: error: flist_gen
openrsync: error: rsync_sender

-- 
:wq Claudio

Index: flist.c
===
RCS file: /cvs/src/usr.bin/rsync/flist.c,v
retrieving revision 1.32
diff -u -p -r1.32 flist.c
--- flist.c 30 Jun 2021 13:10:04 -  1.32
+++ flist.c 2 Jul 2021 13:14:01 -
@@ -972,7 +972,7 @@ flist_gen_dirent(struct sess *sess, char
/* Optionally copy link information. */
 
if (S_ISLNK(ent->fts_statp->st_mode)) {
-   f->link = symlink_read(f->path);
+   f->link = symlink_read(ent->fts_name);
if (f->link == NULL) {
ERRX1("symlink_read");
goto out;



Re: compare-dest support for openrsync

2021-07-02 Thread Claudio Jeker
On Wed, Jun 30, 2021 at 05:47:16PM +0200, Claudio Jeker wrote:
> Thge compare-dest option of rsync is something I would like to use in
> rpki-client. This implements just that and I think after that adding
> copy-dest and link-dest options should be somewhat easy to add as well.
> Lightly tested with my particular need.
 
There is some unveil issues with this diff that need to be worked out.
Until then it people can test by commenting the unveil calls in receiver.c

> -- 
> :wq Claudio
> 
> Index: Makefile
> ===
> RCS file: /cvs/src/usr.bin/rsync/Makefile,v
> retrieving revision 1.10
> diff -u -p -r1.10 Makefile
> --- Makefile  8 May 2019 21:30:11 -   1.10
> +++ Makefile  30 Jun 2021 15:40:19 -
> @@ -1,14 +1,18 @@
>  #$OpenBSD: Makefile,v 1.10 2019/05/08 21:30:11 benno Exp $
>  
>  PROG=openrsync
> -SRCS=blocks.c client.c downloader.c fargs.c flist.c hash.c ids.c \
> +SRCS=blocks.c client.c copy.c downloader.c fargs.c flist.c hash.c 
> ids.c \
>   io.c log.c mkpath.c mktemp.c receiver.c sender.c server.c session.c \
>   socket.c symlinks.c uploader.c main.c misc.c
>  LDADD+= -lcrypto -lm
>  DPADD+= ${LIBCRYPTO} ${LIBM}
>  MAN= openrsync.1
>  
> -CFLAGS+=-g -W -Wall -Wextra
> +CFLAGS+= -Wall -Wextra
> +CFLAGS+= -Wstrict-prototypes -Wmissing-prototypes
> +CFLAGS+= -Wmissing-declarations
> +CFLAGS+= -Wshadow
> +
>  
>  openrsync.1: rsync.1
>   ln -sf ${.CURDIR}/rsync.1 openrsync.1
> Index: copy.c
> ===
> RCS file: copy.c
> diff -N copy.c
> --- /dev/null 1 Jan 1970 00:00:00 -0000
> +++ copy.c30 Jun 2021 15:40:19 -
> @@ -0,0 +1,90 @@
> +/*   $OpenBSD$ */
> +/*
> + * Copyright (c) 2021 Claudio Jeker 
> + *
> + * Permission to use, copy, modify, and distribute this software for any
> + * purpose with or without fee is hereby granted, provided that the above
> + * copyright notice and this permission notice appear in all copies.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
> + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
> + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
> + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
> + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
> + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
> + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
> + */
> +
> +#include/* for MAXBSIZE */
> +
> +#include 
> +#include 
> +#include 
> +
> +#include "extern.h"
> +
> +/*
> + * Return true if all bytes in buffer are zero.
> + * A buffer of zero lenght is also considered a zero buffer.
> + */
> +static int
> +iszero(const void *b, size_t len)
> +{
> + const unsigned char *c = b;
> +
> + for (; len > 0; len--) {
> + if (*c++ != '\0')
> + return 0;
> + }
> + return 1;
> +}
> +
> +static int
> +copy_internal(int fromfd, int tofd)
> +{
> + char buf[MAXBSIZE];
> + ssize_t r, w;
> +
> + while ((r = read(fromfd, buf, sizeof(buf))) > 0) {
> + if (iszero(buf, sizeof(buf))) {
> + if (lseek(tofd, r, SEEK_CUR) == -1)
> + return -1;
> + } else {
> + w = write(tofd, buf, r);
> + if (r != w || w == -1)
> + return -1;
> + }
> + }
> + if (r == -1)
> + return -1;
> + if (ftruncate(tofd, lseek(tofd, 0, SEEK_CUR)) == -1)
> + return -1;
> + return 0;
> +}
> +
> +void
> +copy_file(int rootfd, const char *basedir, const struct flist *f)
> +{
> + int fromfd, tofd, dfd;
> +
> + dfd = openat(rootfd, basedir, O_RDONLY | O_DIRECTORY, 0);
> + if (dfd == -1)
> + err(ERR_FILE_IO, "%s: openat", basedir);
> +
> + fromfd = openat(dfd, f->path, O_RDONLY | O_NOFOLLOW, 0);
> + if (fromfd == -1)
> + err(ERR_FILE_IO, "%s/%s: openat", basedir, f->path);
> + close(dfd);
> +
> + tofd = openat(rootfd, f->path,
> + O_WRONLY | O_NOFOLLOW | O_TRUNC | O_CREAT | O_EXCL,
> + 0600);
> + if (tofd == -1)
> + err(ERR_FILE_IO, "%s: openat", f->path);
> +
> + if (copy_internal(fromfd, tofd) == -1)
> + err(ERR_FILE_IO, "%s: copy file", f->path);
> +
> + close(from

Re: systat(1) counter overflow

2021-07-02 Thread Claudio Jeker
On Fri, Jul 02, 2021 at 01:09:05PM +0200, Martin Pieuchot wrote:
> On 01/07/21(Thu) 13:53, Anindya Mukherjee wrote:
> > Hi,
> > 
> > I noticed that if I leave the system running for more than about a month, 
> > some
> > of the counters in the uvm view of systat(1) overflow and become negative. 
> > This
> > is because the members of struct uvmexp in sys/uvm/uvmexp.h are ints. The
> > kernel's internal counters are of course uint64_t so they don't overflow. It
> > only happens during the uvm_sysctl(9) call which casts the numbers to 
> > integers.
> > The function is uvmexp_read.
> > 
> > In the attached diff I took the path of least resistance and promoted some 
> > of
> > the counters to unsigned int. Ideally I would have liked to use int64_t or 
> > even
> > uint64_t, but I hit an issue in some of the architecture dependent code. An
> > example is:
> > /usr/src/sys/arch/alpha/alpha/trap.c:536 atomic_add_int(, 
> > 1);
> > In other places the ++ operator is used to increment the counters and the 
> > 64 bit
> > types can be used.
> > 
> > I am not completely sure this is the best way to proceed, but even if this 
> > diff
> > is horrifying, I'd appreciate some feedback and advice, thanks!
> 
> I wonder if we shouldn't use uint64_t for those and embrace the ABI
> break, that would at least simplify the kernel side and remove a
> truncation.
> 
> Do you have an idea of the different consumers of the "struct uvmexp"
> apart from systat(1)?  What is the impact in userland & ports of such
> change?

I know that golang has its definition of uvmexp and so if you change the
ABI then you would break at least that. struct uvmexp is used more often
than we would like.

> > Index: sys/uvm/uvm_meter.c
> > ===
> > RCS file: /cvs/src/sys/uvm/uvm_meter.c,v
> > retrieving revision 1.42
> > diff -u -p -r1.42 uvm_meter.c
> > --- sys/uvm/uvm_meter.c 28 Dec 2020 14:01:23 -  1.42
> > +++ sys/uvm/uvm_meter.c 28 Jun 2021 05:24:56 -
> > @@ -329,7 +329,7 @@ uvmexp_read(struct uvmexp *uexp)
> > counters_read(uvmexp_counters, counters, exp_ncounters);
> >  
> > /* stat counters */
> > -   uexp->faults = (int)counters[faults];
> > +   uexp->faults = (unsigned int)counters[faults];
> > uexp->pageins = (int)counters[pageins];
> >  
> > /* fault subcounters */
> > @@ -379,10 +379,10 @@ uvmexp_print(int (*pr)(const char *, ...
> > (*pr)("  freemin=%d, free-target=%d, inactive-target=%d, "
> > "wired-max=%d\n", uexp.freemin, uexp.freetarg, uexp.inactarg,
> > uexp.wiredmax);
> > -   (*pr)("  faults=%d, traps=%d, intrs=%d, ctxswitch=%d fpuswitch=%d\n",
> > +   (*pr)("  faults=%u, traps=%u, intrs=%u, ctxswitch=%u fpuswitch=%d\n",
> > uexp.faults, uexp.traps, uexp.intrs, uexp.swtch,
> > uexp.fpswtch);
> > -   (*pr)("  softint=%d, syscalls=%d, kmapent=%d\n",
> > +   (*pr)("  softint=%u, syscalls=%u, kmapent=%d\n",
> > uexp.softs, uexp.syscalls, uexp.kmapent);
> >  
> > (*pr)("  fault counts:\n");
> > Index: sys/uvm/uvmexp.h
> > ===
> > RCS file: /cvs/src/sys/uvm/uvmexp.h,v
> > retrieving revision 1.9
> > diff -u -p -r1.9 uvmexp.h
> > --- sys/uvm/uvmexp.h4 Mar 2021 09:00:03 -   1.9
> > +++ sys/uvm/uvmexp.h28 Jun 2021 05:24:56 -
> > @@ -90,12 +90,12 @@ struct uvmexp {
> > int unused06;   /* formerly nfreeanon */
> >  
> > /* stat counters */
> > -   int faults; /* page fault count */
> > -   int traps;  /* trap count */
> > -   int intrs;  /* interrupt count */
> > -   int swtch;  /* context switch count */
> > -   int softs;  /* software interrupt count */
> > -   int syscalls;   /* system calls */
> > +   unsigned int faults;/* page fault count */
> > +   unsigned int traps; /* trap count */
> > +   unsigned int intrs; /* interrupt count */
> > +   unsigned int swtch; /* context switch count */
> > +   unsigned int softs; /* software interrupt count */
> > +   unsigned int syscalls;  /* system calls */
> > int pageins;/* pagein operation count */
> > /* pageouts are in pdpageouts below */
> > int unused07;   /* formerly obsolete_swapins */
> > Index: usr.bin/systat/uvm.c
> > ===
> > RCS file: /cvs/src/usr.bin/systat/uvm.c,v
> > retrieving revision 1.5
> > diff -u -p -r1.5 uvm.c
> > --- usr.bin/systat/uvm.c28 Jun 2019 13:35:04 -  1.5
> > +++ usr.bin/systat/uvm.c28 Jun 2021 05:24:57 -
> > @@ -37,22 +37,23 @@ void print_uvm(void);
> >  int  read_uvm(void);
> >  int  select_uvm(void);
> >  
> > -void print_uvmexp_field(field_def *, field_def *, int *, int *, const char 
> > *);
> > +void print_uvmexp_field(field_def *, field_def *, 

compare-dest support for openrsync

2021-06-30 Thread Claudio Jeker
Thge compare-dest option of rsync is something I would like to use in
rpki-client. This implements just that and I think after that adding
copy-dest and link-dest options should be somewhat easy to add as well.
Lightly tested with my particular need.

-- 
:wq Claudio

Index: Makefile
===
RCS file: /cvs/src/usr.bin/rsync/Makefile,v
retrieving revision 1.10
diff -u -p -r1.10 Makefile
--- Makefile8 May 2019 21:30:11 -   1.10
+++ Makefile30 Jun 2021 15:40:19 -
@@ -1,14 +1,18 @@
 #  $OpenBSD: Makefile,v 1.10 2019/05/08 21:30:11 benno Exp $
 
 PROG=  openrsync
-SRCS=  blocks.c client.c downloader.c fargs.c flist.c hash.c ids.c \
+SRCS=  blocks.c client.c copy.c downloader.c fargs.c flist.c hash.c ids.c \
io.c log.c mkpath.c mktemp.c receiver.c sender.c server.c session.c \
socket.c symlinks.c uploader.c main.c misc.c
 LDADD+= -lcrypto -lm
 DPADD+= ${LIBCRYPTO} ${LIBM}
 MAN=   openrsync.1
 
-CFLAGS+=-g -W -Wall -Wextra
+CFLAGS+= -Wall -Wextra
+CFLAGS+= -Wstrict-prototypes -Wmissing-prototypes
+CFLAGS+= -Wmissing-declarations
+CFLAGS+= -Wshadow
+
 
 openrsync.1: rsync.1
ln -sf ${.CURDIR}/rsync.1 openrsync.1
Index: copy.c
===
RCS file: copy.c
diff -N copy.c
--- /dev/null   1 Jan 1970 00:00:00 -
+++ copy.c  30 Jun 2021 15:40:19 -
@@ -0,0 +1,90 @@
+/* $OpenBSD$ */
+/*
+ * Copyright (c) 2021 Claudio Jeker 
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include  /* for MAXBSIZE */
+
+#include 
+#include 
+#include 
+
+#include "extern.h"
+
+/*
+ * Return true if all bytes in buffer are zero.
+ * A buffer of zero lenght is also considered a zero buffer.
+ */
+static int
+iszero(const void *b, size_t len)
+{
+   const unsigned char *c = b;
+
+   for (; len > 0; len--) {
+   if (*c++ != '\0')
+   return 0;
+   }
+   return 1;
+}
+
+static int
+copy_internal(int fromfd, int tofd)
+{
+   char buf[MAXBSIZE];
+   ssize_t r, w;
+
+   while ((r = read(fromfd, buf, sizeof(buf))) > 0) {
+   if (iszero(buf, sizeof(buf))) {
+   if (lseek(tofd, r, SEEK_CUR) == -1)
+   return -1;
+   } else {
+   w = write(tofd, buf, r);
+   if (r != w || w == -1)
+   return -1;
+   }
+   }
+   if (r == -1)
+   return -1;
+   if (ftruncate(tofd, lseek(tofd, 0, SEEK_CUR)) == -1)
+   return -1;
+   return 0;
+}
+
+void
+copy_file(int rootfd, const char *basedir, const struct flist *f)
+{
+   int fromfd, tofd, dfd;
+
+   dfd = openat(rootfd, basedir, O_RDONLY | O_DIRECTORY, 0);
+   if (dfd == -1)
+   err(ERR_FILE_IO, "%s: openat", basedir);
+
+   fromfd = openat(dfd, f->path, O_RDONLY | O_NOFOLLOW, 0);
+   if (fromfd == -1)
+   err(ERR_FILE_IO, "%s/%s: openat", basedir, f->path);
+   close(dfd);
+
+   tofd = openat(rootfd, f->path,
+   O_WRONLY | O_NOFOLLOW | O_TRUNC | O_CREAT | O_EXCL,
+   0600);
+   if (tofd == -1)
+   err(ERR_FILE_IO, "%s: openat", f->path);
+
+   if (copy_internal(fromfd, tofd) == -1)
+   err(ERR_FILE_IO, "%s: copy file", f->path);
+
+   close(fromfd);
+   close(tofd);
+}
Index: extern.h
===
RCS file: /cvs/src/usr.bin/rsync/extern.h,v
retrieving revision 1.39
diff -u -p -r1.39 extern.h
--- extern.h30 Jun 2021 15:24:10 -  1.39
+++ extern.h30 Jun 2021 15:40:19 -
@@ -34,6 +34,15 @@
 #defineBLOCK_SIZE_MIN  (700)
 
 /*
+ * Maximum number of base directories that can be used.
+ */
+#define MAX_BASEDIR20
+
+#define BASE_MODE_COMPARE  1
+#define BASE_MODE_COPY 2
+#define BASE_MODE_LINK 3
+
+/*
  * The sender and receiver use a two-phase synchronisation process.
  * The first uses two-byte hashes; the second, 16-byte.
  * (The second must hold a full MD4 digest.)
@@ -131,10 +140,12 @@ structo

Re: patch: __realpath: no need of LOCKLEAF

2021-06-29 Thread Claudio Jeker
On Sat, Jun 26, 2021 at 07:07:42AM +0200, Sebastien Marie wrote:
> Hi,
> 
> The following diff removes LOCKLEAF from NDINIT. The code doesn't
> doesn't need it: the returned vnode is released immediately. The
> string path is built from the namei() call using REALPATH, during
> directories traversal.
> 
> Without LOCKLEAF, calling vrele() only is enough if namei() found a
> file, instead of calling VOP_UNLOCK() + vrele().
> 
> Comments or OK ?
> -- 
> Sebastien Marie
> 
> diff dca1bd8d5621788b92aad3f944ac965773e2 
> 545f8aacd74a1f793d1289475eb1c3f84d649e06
> blob - 840ea5453e13604cb38b6a5d580370053386ce71
> blob + fe240bb520b142726f358dfec4eaf6f26de151c1
> --- sys/kern/vfs_syscalls.c
> +++ sys/kern/vfs_syscalls.c
> @@ -916,36 +916,35 @@ sys___realpath(struct proc *p, void *v, register_t *re
>   }
>  
>   free(cwdbuf, M_TEMP, cwdlen);
>   }
>  
>   /* find root "/" or "//" */
>   for (c = pathname; *c != '\0'; c++) {
>   if (*c != '/')
>   break;
>   }
> - NDINIT(, LOOKUP, FOLLOW | LOCKLEAF | SAVENAME | REALPATH,
> - UIO_SYSSPACE, pathname, p);
> + NDINIT(, LOOKUP, FOLLOW | SAVENAME | REALPATH, UIO_SYSSPACE,
> + pathname, p);
>  
>   nd.ni_cnd.cn_rpbuf = rpbuf;
>   nd.ni_cnd.cn_rpi = strlen(rpbuf);
>  
>   nd.ni_pledge = PLEDGE_RPATH;
>   nd.ni_unveil = UNVEIL_READ;
>   if ((error = namei()) != 0)
>   goto end;
>  
> - /* release lock and reference from namei */
> - if (nd.ni_vp) {
> - VOP_UNLOCK(nd.ni_vp);
> + /* release reference from namei */
> + if (nd.ni_vp)
>   vrele(nd.ni_vp);
> - }
> +
>   error = copyoutstr(nd.ni_cnd.cn_rpbuf, SCARG(uap, resolved),
>   MAXPATHLEN, NULL);
>  
>  #ifdef KTRACE
>   if (KTRPOINT(p, KTR_NAMEI))
>   ktrnamei(p, nd.ni_cnd.cn_rpbuf);
>  #endif
>   pool_put(_pool, nd.ni_cnd.cn_pnbuf);
>  end:
>   pool_put(_pool, rpbuf);
> 

Makes sense. OK claudio@

-- 
:wq Claudio



bgpd refactor struct prefix

2021-06-29 Thread Claudio Jeker
This diff moves the rib_entry pointer re into the union to safe some
space. For add-path I need to add a few more u_int32_t and that would
blow the size of struct prefix from 128 to 132 bytes. malloc would round
that up to 256bytes and that is bad for the struct that is allocted in
millions in bgpd.

To make this somewhat save introduce PREFIX_FLAG_ADJOUT to mark prefixes
that live in the adj-rib-out. Those prefixes can not access the re pointer
also use a wrapper prefix_re() which returns the re pointer or NULL.
Also add some assertions to make sure that prefixes don't end up in the
wrong tree.

This change shrinks the struct back to 120bytes and gives me the space
needed for add-path.

Please test
-- 
:wq Claudio

Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.530
diff -u -p -r1.530 rde.c
--- rde.c   25 Jun 2021 09:25:48 -  1.530
+++ rde.c   29 Jun 2021 08:28:33 -
@@ -2298,6 +2298,7 @@ rde_dump_rib_as(struct prefix *p, struct
struct ibuf *wbuf;
struct attr *a;
struct nexthop  *nexthop;
+   struct rib_entry*re;
void*bp;
time_t   staletime;
size_t   aslen;
@@ -2330,7 +2331,8 @@ rde_dump_rib_as(struct prefix *p, struct
rib.origin = asp->origin;
rib.validation_state = p->validation_state;
rib.flags = 0;
-   if (p->re != NULL && p->re->active == p)
+   re = prefix_re(p);
+   if (re != NULL && re->active == p)
rib.flags |= F_PREF_ACTIVE;
if (!prefix_peer(p)->conf.ebgp)
rib.flags |= F_PREF_INTERNAL;
@@ -2412,14 +2414,16 @@ static void
 rde_dump_filter(struct prefix *p, struct ctl_show_rib_request *req)
 {
struct rde_aspath   *asp;
+   struct rib_entry*re;
 
if (!rde_match_peer(prefix_peer(p), >neighbor))
return;
 
asp = prefix_aspath(p);
+   re = prefix_re(p);
if (asp == NULL)/* skip pending withdraw in Adj-RIB-Out */
return;
-   if ((req->flags & F_CTL_ACTIVE) && p->re->active != p)
+   if ((req->flags & F_CTL_ACTIVE) && re != NULL && re->active != p)
return;
if ((req->flags & F_CTL_INVALID) &&
(asp->flags & F_ATTR_PARSE_ERR) == 0)
Index: rde.h
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.h,v
retrieving revision 1.240
diff -u -p -r1.240 rde.h
--- rde.h   17 Jun 2021 16:05:26 -  1.240
+++ rde.h   29 Jun 2021 08:33:18 -
@@ -56,10 +56,10 @@ struct rib {
struct filter_head  *in_rules_tmp;
u_int   rtableid;
u_int   rtableid_tmp;
+   enum reconf_action  state, fibstate;
+   u_int16_t   id;
u_int16_t   flags;
u_int16_t   flags_tmp;
-   u_int16_t   id;
-   enum reconf_action  state, fibstate;
 };
 
 #define RIB_ADJ_IN 0
@@ -317,13 +317,13 @@ struct prefix {
union {
struct {
LIST_ENTRY(prefix)   rib, nexthop;
+   struct rib_entry*re;
} list;
struct {
RB_ENTRY(prefix) index, update;
} tree;
}entry;
struct pt_entry *pt;
-   struct rib_entry*re;
struct rde_aspath   *aspath;
struct rde_community*communities;
struct rde_peer *peer;
@@ -338,6 +338,7 @@ struct prefix {
 #definePREFIX_FLAG_DEAD0x04/* locked but removed */
 #definePREFIX_FLAG_STALE   0x08/* stale entry (graceful 
reload) */
 #definePREFIX_FLAG_MASK0x0f/* mask for the prefix types */
+#definePREFIX_FLAG_ADJOUT  0x10/* prefix is in the adj-out rib 
*/
 #definePREFIX_NEXTHOP_LINKED   0x40/* prefix is linked onto 
nexthop list */
 #definePREFIX_FLAG_LOCKED  0x80/* locked by rib walker */
 };
@@ -637,6 +638,14 @@ static inline u_int8_t
 prefix_vstate(struct prefix *p)
 {
return (p->validation_state & ROA_MASK);
+}
+
+static inline struct rib_entry *
+prefix_re(struct prefix *p)
+{
+   if (p->flags & PREFIX_FLAG_ADJOUT)
+   return NULL;
+   return (p->entry.list.re);
 }
 
 voidnexthop_init(u_int32_t);
Index: rde_rib.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde_rib.c,v
retrieving revision 1.222
diff -u -p -r1.222 rde_rib.c
--- rde_rib.c   17 Jun 2021 08:16:04 -  1.222
+++ rde_rib.c   29 Jun 2021 09:41:21 -
@@ -1031,6 +1031,9 @@ prefix_move(struct prefix *p, struct 

Re: recvmsg returns MSG_DONTWAIT

2021-06-27 Thread Claudio Jeker
On Sun, Jun 27, 2021 at 05:28:09PM +0200, Mark Kettenis wrote:
> > Date: Sun, 27 Jun 2021 13:36:03 +
> > From: Klemens Nanni 
> > 
> > On Sat, Jun 12, 2021 at 11:54:58PM -0700, Greg Steuck wrote:
> > > I started with a failing test for Haskell network package on 6.9-current 
> > > amd64
> > > (cabal get network-3.1.2.1 && cabal v2-test)
> > > 
> > > network-3.1.2.1/build/spec/spec --match 
> > > "/Network.Socket.ByteString/recvMsg/works well/"
> > > 
> > >   tests/Network/Socket/ByteStringSpec.hs:209:21: 
> > >   1) Network.Socket.ByteString.recvMsg works well
> > >expected: MsgFlag {fromMsgFlag = 0}
> > > but got: MsgFlag {fromMsgFlag = 128}
> > > 
> > > ktrace says:
> > > 
> > >  47649 spec CALL  sendto(14,0x8a2126e838c,0x16,0,0x8a4a3622df0,0x10)
> > >  47649 spec STRU  struct sockaddr { AF_INET, 127.0.0.1:9486 }
> > >  47649 spec GIO   fd 14 wrote 22 bytes
> > >"This is a test message"
> > >  47649 spec RET   sendto 22/0x16
> > >  47649 spec CALL  
> > > futex(0x8a4aef6f930,0x81,1,0,0)
> > >  47649 spec STRU  struct kevent { ident=13, filter=EVFILT_READ, 
> > > flags=0x11, fflags=0<>, data=38, udata=0x0 }
> > >  47649 spec RET   kevent 1
> > >  47649 spec CALL  recvmsg(13,0x8a4a3622c50,0)
> > >  47649 spec GIO   fd 13 read 22 bytes
> > >"This is a test message"
> > >  47649 spec STRU  struct sockaddr { AF_INET, 127.0.0.1:12293 }
> > >  47649 spec STRU  struct msghdr { name=0x8a4a3622b70, namelen=16, 
> > > iov=0x8a4a3622c30, iovlen=1, control=0x8a4a3622c10, controllen=0, 
> > > flags=0x80 }
> > >  47649 spec STRU  struct iovec { base=0x8a4a3622666, len=1002 }
> > >  47649 spec RET   recvmsg 22/0x16
> > > 
> > > This seems to contradict recvmsg(2) which doesn't list MSG_DONTWAIT as a
> > > possible value of the flags. Would this be useful as a C regress test?
> > 
> > Looks like this was missed in sys/kern/uipc_syscalls.c revision 1.178
> > 
> > date: 2018/07/30 12:22:14;  author: mpi;  state: Exp;  lines: +10 -18;  
> > commitid: K43aQe66cQkEOSbc;
> > Use FNONBLOCK instead of SS_NBIO to check/indicate that the I/O mode
> > for sockets is non-blocking.
> > 
> > This allows us to G/C SS_NBIO.  Having to keep the two flags in sync
> > in a mp-safe way is complicated.
> > 
> > This change introduce a behavior change in sosplice(), it can now
> > always block.  However this should not matter much due to the socket
> > lock being taken beforhand.
> > 
> > bluhm@, benno@, visa@
> > 
> > Wording taken from mpi's commit message which is exactly what recvit()
> > does in one place, i.e.
> > 
> > if (fp->f_flag & FNONBLOCK)
> > mp->msg_flags |= MSG_DONTWAIT;
> > 
> > 
> > Here's the documentation bits for it?
> > Feedback? Objections? OK?
> 
> I think this points out that diff wasn't quite right.  I mean,
> changing the man page doesn't fix the Haskell test does it?

I agree, I think MSG_DONTWAIT should not be returend to userland.

-- 
:wq Claudio



OpenBGPD 7.1 released

2021-06-25 Thread Claudio Jeker
We have released OpenBGPD 7.1, which will be arriving in the
OpenBGPD directory of your local OpenBSD mirror soon.

This release includes the following changes to the previous release:

* OpenBSD 6.9 errata 009
  During bgpd(8) config reloads prefixes of the wrong address family could
  leak to peers resulting in session resets.

* Support for RFC 7313 - Enhanced Route Refresh
  Disabled by default, to enable use 'announce enhanced refresh yes'.

* Improve output of Adj-RIB-Out by updating nexthop and ASPATH before
  adding the prefix to the RIB. This improves `bgpctl show rib out`
  output.

* Add command line option to show the version

OpenBGPD-portable is known to compile and run on FreeBSD, and
the Linux distributions Alpine, Debian, Fedora, RHEL/CentOS and Ubuntu.
It is our hope that packagers take interest and help adapt OpenBGPD-portable
to more distributions.

We welcome feedback and improvements from the broader community.
Thanks to all of the contributors who helped make this release
possible.



bgpd refactor network flush code a bit

2021-06-24 Thread Claudio Jeker
The network flush code only operates on peerself (like all the other
network commands). Instead of passing a peer to the tree walker just
default to peerself in network_flush_upcall().
This makes the code more obivous that it operates on peerself.

-- 
:wq Claudio

Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.528
diff -u -p -r1.528 rde.c
--- rde.c   24 Jun 2021 13:03:31 -  1.528
+++ rde.c   24 Jun 2021 15:00:32 -
@@ -517,7 +517,7 @@ badnetdel:
break;
}
if (rib_dump_new(RIB_ADJ_IN, AID_UNSPEC,
-   RDE_RUNNER_ROUNDS, peerself, network_flush_upcall,
+   RDE_RUNNER_ROUNDS, NULL, network_flush_upcall,
NULL, NULL) == -1)
log_warn("rde_dispatch: IMSG_NETWORK_FLUSH");
break;
@@ -4065,13 +4065,12 @@ network_dump_upcall(struct rib_entry *re
 static void
 network_flush_upcall(struct rib_entry *re, void *ptr)
 {
-   struct rde_peer *peer = ptr;
struct bgpd_addr addr;
struct prefix *p;
u_int32_t i;
u_int8_t prefixlen;
 
-   p = prefix_bypeer(re, peer);
+   p = prefix_bypeer(re, peerself);
if (p == NULL)
return;
if ((prefix_aspath(p)->flags & F_ANN_DYNAMIC) != F_ANN_DYNAMIC)
@@ -4084,14 +4083,14 @@ network_flush_upcall(struct rib_entry *r
struct rib *rib = rib_byid(i);
if (rib == NULL)
continue;
-   if (prefix_withdraw(rib, peer, , prefixlen) == 1)
-   rde_update_log("flush announce", i, peer,
+   if (prefix_withdraw(rib, peerself, , prefixlen) == 1)
+   rde_update_log("flush announce", i, peerself,
NULL, , prefixlen);
}
 
-   if (prefix_withdraw(rib_byid(RIB_ADJ_IN), peer, ,
+   if (prefix_withdraw(rib_byid(RIB_ADJ_IN), peerself, ,
prefixlen) == 1)
-   peer->prefix_cnt--;
+   peerself->prefix_cnt--;
 }
 
 /* clean up */



bgpd shuffle some code around

2021-06-24 Thread Claudio Jeker
In rde_update_dispatch() do the AFI check for IPv4 prefixes before
extracting the prefix. This is similar to what the MP code does and
is also more logical.

OK?
-- 
:wq Claudio

Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.528
diff -u -p -r1.528 rde.c
--- rde.c   24 Jun 2021 13:03:31 -  1.528
+++ rde.c   24 Jun 2021 15:00:10 -
@@ -1280,6 +1280,14 @@ rde_update_dispatch(struct rde_peer *pee
 
/* withdraw prefix */
while (len > 0) {
+   if (peer->capa.mp[AID_INET] == 0) {
+   log_peer_warnx(>conf,
+   "bad withdraw, %s disabled", aid2str(AID_INET));
+   rde_update_err(peer, ERR_UPDATE, ERR_UPD_OPTATTR,
+   NULL, 0);
+   goto done;
+   }
+
if ((pos = nlri_get_prefix(p, len, ,
)) == -1) {
/*
@@ -1294,14 +1302,6 @@ rde_update_dispatch(struct rde_peer *pee
p += pos;
len -= pos;
 
-   if (peer->capa.mp[AID_INET] == 0) {
-   log_peer_warnx(>conf,
-   "bad withdraw, %s disabled", aid2str(AID_INET));
-   rde_update_err(peer, ERR_UPDATE, ERR_UPD_OPTATTR,
-   NULL, 0);
-   goto done;
-   }
-
rde_update_withdraw(peer, , prefixlen);
}
 
@@ -1393,6 +1393,14 @@ rde_update_dispatch(struct rde_peer *pee
 
/* parse nlri prefix */
while (nlri_len > 0) {
+   if (peer->capa.mp[AID_INET] == 0) {
+   log_peer_warnx(>conf,
+   "bad update, %s disabled", aid2str(AID_INET));
+   rde_update_err(peer, ERR_UPDATE, ERR_UPD_OPTATTR,
+   NULL, 0);
+   goto done;
+   }
+
if ((pos = nlri_get_prefix(p, nlri_len, ,
)) == -1) {
log_peer_warnx(>conf, "bad nlri prefix");
@@ -1402,14 +1410,6 @@ rde_update_dispatch(struct rde_peer *pee
}
p += pos;
nlri_len -= pos;
-
-   if (peer->capa.mp[AID_INET] == 0) {
-   log_peer_warnx(>conf,
-   "bad update, %s disabled", aid2str(AID_INET));
-   rde_update_err(peer, ERR_UPDATE, ERR_UPD_OPTATTR,
-   NULL, 0);
-   goto done;
-   }
 
if (rde_update_update(peer, , , prefixlen) == -1)
goto done;



Re: bgpd fix bad free() call when deflating aspath

2021-06-24 Thread Claudio Jeker
On Tue, Jun 22, 2021 at 08:19:22PM +0200, Claudio Jeker wrote:
> I changed the way up_generate_attr() calls aspath_deflate() but did not
> realize that aspath_deflate() frees the pdata at the end. That free should
> no longer happen but for that also the mrt case where aspath_deflate()
> needs to be adjusted.
> 
> With this both the mrt and as0 integration test pass again.

Ping. This crashes bgpd when talking to old 2-byte only BGP speakers. So
I would like to commit this soon.

-- 
:wq Claudio

Index: rde_attr.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde_attr.c,v
retrieving revision 1.124
diff -u -p -r1.124 rde_attr.c
--- rde_attr.c  16 Jan 2021 13:14:54 -  1.124
+++ rde_attr.c  22 Jun 2021 18:13:15 -
@@ -568,7 +568,6 @@ aspath_put(struct aspath *aspath)
 
 /*
  * convert a 4 byte aspath to a 2 byte one.
- * data is freed by aspath_deflate
  */
 u_char *
 aspath_deflate(u_char *data, u_int16_t *len, int *flagnew)
@@ -614,7 +613,6 @@ aspath_deflate(u_char *data, u_int16_t *
}
}
 
-   free(data);
*len = nlen;
return (ndata);
 }
Index: mrt.c
===
RCS file: /cvs/src/usr.sbin/bgpd/mrt.c,v
retrieving revision 1.103
diff -u -p -r1.103 mrt.c
--- mrt.c   9 Jan 2020 11:55:25 -   1.103
+++ mrt.c   22 Jun 2021 18:12:45 -
@@ -160,15 +160,19 @@ mrt_attr_dump(struct ibuf *buf, struct r
return (-1);
 
/* aspath */
-   pdata = aspath_prepend(a->aspath, rde_local_as(), 0, );
+   plen = aspath_length(a->aspath);
+   pdata = aspath_dump(a->aspath);
+
if (!v2)
pdata = aspath_deflate(pdata, , );
if (attr_writebuf(buf, ATTR_WELL_KNOWN, ATTR_ASPATH, pdata,
plen) == -1) {
-   free(pdata);
+   if (!v2)
+   free(pdata);
return (-1);
}
-   free(pdata);
+   if (!v2)
+   free(pdata);
 
if (nexthop && nexthop->aid == AID_INET) {
/* nexthop, already network byte order */




bgpd refactor some common code for add-path

2021-06-22 Thread Claudio Jeker
Adjust the way nlri get extracted from the MP attrs. Instead of
switch statements with a while loop for each case move the while loop out
and only do the nlri_get_* call in the switch statement. The mpp and mplen
adjustmens and the call to rde_update_update and rde_update_withdraw are
also moved to the while loop.

-- 
:wq Claudio

Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.527
diff -u -p -r1.527 rde.c
--- rde.c   17 Jun 2021 16:05:26 -  1.527
+++ rde.c   22 Jun 2021 17:49:22 -
@@ -1338,9 +1338,9 @@ rde_update_dispatch(struct rde_peer *pee
rde_peer_recv_eor(peer, aid);
}
 
-   switch (aid) {
-   case AID_INET6:
-   while (mplen > 0) {
+   while (mplen > 0) {
+   switch (aid) {
+   case AID_INET6:
if ((pos = nlri_get_prefix6(mpp, mplen,
, )) == -1) {
log_peer_warnx(>conf,
@@ -1350,14 +1350,8 @@ rde_update_dispatch(struct rde_peer *pee
mpa.unreach, mpa.unreach_len);
goto done;
}
-   mpp += pos;
-   mplen -= pos;
-
-   rde_update_withdraw(peer, , prefixlen);
-   }
-   break;
-   case AID_VPN_IPv4:
-   while (mplen > 0) {
+   break;
+   case AID_VPN_IPv4:
if ((pos = nlri_get_vpn4(mpp, mplen,
, , 1)) == -1) {
log_peer_warnx(>conf,
@@ -1367,14 +1361,8 @@ rde_update_dispatch(struct rde_peer *pee
mpa.unreach, mpa.unreach_len);
goto done;
}
-   mpp += pos;
-   mplen -= pos;
-
-   rde_update_withdraw(peer, , prefixlen);
-   }
-   break;
-   case AID_VPN_IPv6:
-   while (mplen > 0) {
+   break;
+   case AID_VPN_IPv6:
if ((pos = nlri_get_vpn6(mpp, mplen,
, , 1)) == -1) {
log_peer_warnx(>conf,
@@ -1384,15 +1372,16 @@ rde_update_dispatch(struct rde_peer *pee
mpa.unreach_len);
goto done;
}
-   mpp += pos;
-   mplen -= pos;
-
-   rde_update_withdraw(peer, , prefixlen);
+   break;
+   default:
+   /* ignore unsupported multiprotocol AF */
+   break;
}
-   break;
-   default:
-   /* silently ignore unsupported multiprotocol AF */
-   break;
+
+   mpp += pos;
+   mplen -= pos;
+
+   rde_update_withdraw(peer, , prefixlen);
}
 
if ((state.aspath.flags & ~F_ATTR_MP_UNREACH) == 0)
@@ -1466,9 +1455,9 @@ rde_update_dispatch(struct rde_peer *pee
mpp += pos;
mplen -= pos;
 
-   switch (aid) {
-   case AID_INET6:
-   while (mplen > 0) {
+   while (mplen > 0) {
+   switch (aid) {
+   case AID_INET6:
if ((pos = nlri_get_prefix6(mpp, mplen,
, )) == -1) {
log_peer_warnx(>conf,
@@ -1478,16 +1467,8 @@ rde_update_dispatch(struct rde_peer *pee
mpa.reach, mpa.reach_len);
goto done;
}
-   mpp += pos;
-   mplen -= pos;
-
-   if (rde_update_update(peer, , ,
-   prefixlen) == -1)
-   goto done;
-   }
-   break;
-   case AID_VPN_IPv4:
-   while (mplen > 0) {
+   break;
+   case AID_VPN_IPv4:
if ((pos = nlri_get_vpn4(mpp, 

bgpd fix bad free() call when deflating aspath

2021-06-22 Thread Claudio Jeker
I changed the way up_generate_attr() calls aspath_deflate() but did not
realize that aspath_deflate() frees the pdata at the end. That free should
no longer happen but for that also the mrt case where aspath_deflate()
needs to be adjusted.

With this both the mrt and as0 integration test pass again.
-- 
:wq Claudio

Index: rde_attr.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde_attr.c,v
retrieving revision 1.124
diff -u -p -r1.124 rde_attr.c
--- rde_attr.c  16 Jan 2021 13:14:54 -  1.124
+++ rde_attr.c  22 Jun 2021 18:13:15 -
@@ -568,7 +568,6 @@ aspath_put(struct aspath *aspath)
 
 /*
  * convert a 4 byte aspath to a 2 byte one.
- * data is freed by aspath_deflate
  */
 u_char *
 aspath_deflate(u_char *data, u_int16_t *len, int *flagnew)
@@ -614,7 +613,6 @@ aspath_deflate(u_char *data, u_int16_t *
}
}
 
-   free(data);
*len = nlen;
return (ndata);
 }
Index: mrt.c
===
RCS file: /cvs/src/usr.sbin/bgpd/mrt.c,v
retrieving revision 1.103
diff -u -p -r1.103 mrt.c
--- mrt.c   9 Jan 2020 11:55:25 -   1.103
+++ mrt.c   22 Jun 2021 18:12:45 -
@@ -160,15 +160,19 @@ mrt_attr_dump(struct ibuf *buf, struct r
return (-1);
 
/* aspath */
-   pdata = aspath_prepend(a->aspath, rde_local_as(), 0, );
+   plen = aspath_length(a->aspath);
+   pdata = aspath_dump(a->aspath);
+
if (!v2)
pdata = aspath_deflate(pdata, , );
if (attr_writebuf(buf, ATTR_WELL_KNOWN, ATTR_ASPATH, pdata,
plen) == -1) {
-   free(pdata);
+   if (!v2)
+   free(pdata);
return (-1);
}
-   free(pdata);
+   if (!v2)
+   free(pdata);
 
if (nexthop && nexthop->aid == AID_INET) {
/* nexthop, already network byte order */



bgpd fix add-path capability encoding

2021-06-22 Thread Claudio Jeker
Dumb copy paste error. The add-path capability is 4byte per AFI/SAFI
the 2 + is from graceful restart where two extra bytes are at the front of
the AFI/SAFI list.

-- 
:wq Claudio

Index: session.c
===
RCS file: /cvs/src/usr.sbin/bgpd/session.c,v
retrieving revision 1.421
diff -u -p -r1.421 session.c
--- session.c   17 Jun 2021 16:05:26 -  1.421
+++ session.c   22 Jun 2021 11:50:08 -
@@ -1470,9 +1470,9 @@ session_open(struct peer *p)
u_int8_taplen;
 
if (mpcapa)
-   aplen = 2 + 4 * mpcapa;
+   aplen = 4 * mpcapa;
else/* AID_INET */
-   aplen = 2 + 4;
+   aplen = 4;
errs += session_capa_add(opb, CAPA_ADD_PATH, aplen);
if (mpcapa) {
for (i = AID_MIN; i < AID_MAX; i++) {



Re: setitimer(2): increase interval upper bound to UINT_MAX seconds

2021-06-18 Thread Claudio Jeker
On Thu, Jun 17, 2021 at 08:41:39PM -0500, Scott Cheloha wrote:
> On Fri, Jun 11, 2021 at 12:17:02PM -0500, Scott Cheloha wrote:
> > Hi,
> > 
> > setitimer(2) has a one hundred million second upper bound for timers.
> > Any timer interval larger than this is considered invalid and we set
> > EINVAL.
> > 
> > There is no longer any reason to use this particular limit.  Kclock
> > timeouts support the full range of a timespec, so we can trivially
> > increase the upper bound without any practical risk of overflow.
> > 
> > This patch increases the upper bound to UINT_MAX seconds.
> > 
> > Why UINT_MAX?  UINT_MAX is the largest possible input to alarm(3).  We
> > could then simplify the alarm(3) manpage and the libc alarm.c code in
> > a subsequent patch.  POSIX says alarm(3) "is always successful".  Our
> > implementation can fail.  It would be nicer/simpler if ours were free
> > of failure modes.
> > 
> > ok?
> 
> 1 week bump.
> 
> Updated patch: make the maximum value ("max") static and const.

OK claudio@

I wonder if we need a max at all? I guess there is an upper limit to not
overflow the time_t when calculating the absolute timeout but that is
probably close to LLONG_MAX / 2.

I think a simplified version of alarm(3) that never fails would be nice.
 
> Index: kern_time.c
> ===
> RCS file: /cvs/src/sys/kern/kern_time.c,v
> retrieving revision 1.153
> diff -u -p -r1.153 kern_time.c
> --- kern_time.c   11 Jun 2021 16:36:34 -  1.153
> +++ kern_time.c   18 Jun 2021 01:40:42 -
> @@ -709,15 +709,16 @@ out:
>  int
>  itimerfix(struct itimerval *itv)
>  {
> + static const struct timeval max = { .tv_sec = UINT_MAX, .tv_usec = 0 };
>   struct timeval min_interval = { .tv_sec = 0, .tv_usec = tick };
>  
>   if (itv->it_value.tv_sec < 0 || !timerisvalid(>it_value))
>   return EINVAL;
> - if (itv->it_value.tv_sec > 1)
> + if (timercmp(>it_value, , >))
>   return EINVAL;
>   if (itv->it_interval.tv_sec < 0 || !timerisvalid(>it_interval))
>   return EINVAL;
> - if (itv->it_interval.tv_sec > 1)
> + if (timercmp(>it_interval, , >))
>   return EINVAL;
>  
>   if (!timerisset(>it_value))
> 

-- 
:wq Claudio



Re: bgpd support for enhanced route refresh

2021-06-17 Thread Claudio Jeker
On Thu, Jun 17, 2021 at 01:40:01PM +, Job Snijders wrote:
> On Thu, Jun 17, 2021 at 03:29:38PM +0200, Claudio Jeker wrote:
> > On Thu, Jun 17, 2021 at 01:25:07PM +, Job Snijders wrote:
> > > On Thu, Jun 17, 2021 at 12:24:16PM +0200, Claudio Jeker wrote:
> > > > On Mon, Jun 14, 2021 at 05:10:07PM +0200, Claudio Jeker wrote:
> > > > > On Thu, May 27, 2021 at 06:24:06PM +0200, Claudio Jeker wrote:
> > > > > > Implement RFC 7313 enhanced route refresh.
> > > > > > 
> > > > > > While there also change when graceful restart EoR markers are sent.
> > > > > > In short the graceful restart marker should only be sent initally. 
> > > > > > After
> > > > > > that the End of Route Refresh message should be sent instead.
> > > > > > Because of this track if an EoR marker was received or should be 
> > > > > > sent in
> > > > > > the peer config.
> > > > > > 
> > > > > > For now this setting is off by default but that may be changed at a 
> > > > > > later
> > > > > > state.
> > > > > > 
> > > > > > Please try this out and tell me if it works for you. The message and
> > > > > > prefix/rrefresh counters in bgpctl show nei output help a lot to 
> > > > > > see what
> > > > > > is going on.
> > > > 
> > > > Minor cleanup, instead of this comment use CTASSERT to make sure that 
> > > > the
> > > > recv_eor and send_eor fields are large enough to work as a bitfield for
> > > > AID_MAX
> > > > 
> > > > > -#define  AID_MAX 5
> > > > > +#define  AID_MAX 5   /* check rde_peer.recv_eor when 
> > > > > max reaches 7 */
> > > > 
> > > > Enhanced route refresh is currently off by default. So unless somebody
> > > > is against this I will commit this soon. At least then it gets tested :)
> > > 
> > > Under what circumstances will bgpd(8) send an enhanced route refresh?
> > 
> > The enhanced route refresh messages BoRR and EoRR are sent from the system
> > that got the normal route refresh request. So you need to issue a refresh
> > from the peer to see the new messages.
> 
> Ah... there we go:
> 
> 13:39:14.349247 10.0.0.12.28035 > 10.0.0.1.bgp: P 19:42(23) ack 1 win 256 
> : BGP (ROUTE-REFRESH Request (IPv4 
> Unicast)) (DF) [tos 0xc0]
> 13:39:14.351043 10.0.0.1.bgp > 10.0.0.12.28035: P 1:24(23) ack 42 win 267 
> : BGP (ROUTE-REFRESH BoRR (IPv4 
> Unicast)) [tos 0xc0]
> 13:39:14.548996 10.0.0.12.28035 > 10.0.0.1.bgp: . ack 24 win 256 
>  (DF) [tos 0xc0]
> 13:39:18.855601 10.0.0.1.bgp > 10.0.0.12.28035: P 24:47(23) ack 42 win 267 
> : BGP (ROUTE-REFRESH EoRR (IPv4 
> Unicast)) [tos 0xc0]
> 13:39:19.049045 10.0.0.12.28035 > 10.0.0.1.bgp: . ack 47 win 256 
>  (DF) [tos 0xc0]
> 
> Index: print-bgp.c
> ===
> RCS file: /cvs/src/usr.sbin/tcpdump/print-bgp.c,v
> retrieving revision 1.29
> diff -u -p -r1.29 print-bgp.c
> --- print-bgp.c   3 Jul 2019 03:24:03 -   1.29
> +++ print-bgp.c   17 Jun 2021 13:39:37 -
> @@ -97,7 +97,7 @@ struct bgp_route_refresh {
>   u_int16_t len;
>   u_int8_t type;
>   u_int8_t afi[2]; /* unaligned; should be u_int16_t */
> - u_int8_t res;
> + u_int8_t subtype;
>   u_int8_t safi;
>  };
>  #define BGP_ROUTE_REFRESH_SIZE  23
> @@ -189,6 +189,7 @@ static const char *bgp_capcode[] = {
>   /* 67: [Chen] */ "DYNAMIC_CAPABILITY",
>   /* 68: [Appanna] */ "MULTISESSION",
>   /* 69: [draft-ietf-idr-add-paths] */ "ADD-PATH",
> + /* 70: RFC7313 */ "ENHANCED_ROUTE_REFRESH"
>  };
>  
>  #define bgp_capcode(x) \
> @@ -199,7 +200,7 @@ static const char *bgpnotify_major[] = {
>   NULL, "Message Header Error",
>   "OPEN Message Error", "UPDATE Message Error",
>   "Hold Timer Expired", "Finite State Machine Error",
> - "Cease", "Capability Message Error",
> + "Cease", "ROUTE_REFRESH Message Error",
>  };
>  #define bgp_notify_major(x) \
>   num_or_str(bgpnotify_major, \
> @@ -323,6 +324,11 @@ static const char *afnumber[] = AFNUM_NA
>   num_or_str(afnumber, \
>   sizeof(afnumber)/sizeof(afnumber[0]), (x)))
>  
> +static const char *refreshtype[] = {
> + "Request", "BoRR", "EoRR"
> +};
> +#define refresh_subtype(x) \
> + num_or_str(refreshtype, sizeof(refreshtype)/sizeof(refreshtype[0]), (x))
>  
>  static const char *
>  num_or_str(const char **table, size_t siz, int value)
> @@ -1069,7 +1075,8 @@ bgp_route_refresh_print(const u_char *da
>  
>   bgp_route_refresh_header = (const struct bgp_route_refresh *)dat;
>  
> - printf(" (%s %s)",
> + printf(" %s (%s %s)",
> + refresh_subtype(bgp_route_refresh_header->subtype),
>   af_name(EXTRACT_16BITS(_route_refresh_header->afi)),
>   bgp_attr_nlri_safi(bgp_route_refresh_header->safi));
>  
> 

Not a tcpdump expert but this diff looks OK.

-- 
:wq Claudio



Re: bgpd support for enhanced route refresh

2021-06-17 Thread Claudio Jeker
On Thu, Jun 17, 2021 at 01:25:07PM +, Job Snijders wrote:
> On Thu, Jun 17, 2021 at 12:24:16PM +0200, Claudio Jeker wrote:
> > On Mon, Jun 14, 2021 at 05:10:07PM +0200, Claudio Jeker wrote:
> > > On Thu, May 27, 2021 at 06:24:06PM +0200, Claudio Jeker wrote:
> > > > Implement RFC 7313 enhanced route refresh.
> > > > 
> > > > While there also change when graceful restart EoR markers are sent.
> > > > In short the graceful restart marker should only be sent initally. After
> > > > that the End of Route Refresh message should be sent instead.
> > > > Because of this track if an EoR marker was received or should be sent in
> > > > the peer config.
> > > > 
> > > > For now this setting is off by default but that may be changed at a 
> > > > later
> > > > state.
> > > > 
> > > > Please try this out and tell me if it works for you. The message and
> > > > prefix/rrefresh counters in bgpctl show nei output help a lot to see 
> > > > what
> > > > is going on.
> > 
> > Minor cleanup, instead of this comment use CTASSERT to make sure that the
> > recv_eor and send_eor fields are large enough to work as a bitfield for
> > AID_MAX
> > 
> > > -#define  AID_MAX 5
> > > +#define  AID_MAX 5   /* check rde_peer.recv_eor when max 
> > > reaches 7 */
> > 
> > Enhanced route refresh is currently off by default. So unless somebody
> > is against this I will commit this soon. At least then it gets tested :)
> 
> Under what circumstances will bgpd(8) send an enhanced route refresh?

The enhanced route refresh messages BoRR and EoRR are sent from the system
that got the normal route refresh request. So you need to issue a refresh
from the peer to see the new messages.
 
> If I issue 'bgpctl neighbor xxx refresh' on a session where Enhanced
> Refresh is negotiated
> 
>   Negotiated capabilities:
> Multiprotocol extensions: IPv4 unicast
> 4-byte AS numbers
> Route Refresh
> Enhanced Route Refresh
> Graceful Restart
> 
> I see in tshark:
> 
>Border Gateway Protocol - ROUTE-REFRESH Message
>Marker: 
>Length: 23
>Type: ROUTE-REFRESH Message (5)
>Address family identifier (AFI): IPv4 (1)
>Subtype: Normal route refresh request [RFC2918] with/without ORF 
> [RFC5291] (0)
>Subsequent address family identifier (SAFI): Unicast (1)
> 
> Attached below a tcpdump diff that perhaps can help improve visibility
> :-)
> 
> Kind regards,
> 
> Job
> 
> Index: print-bgp.c
> ===
> RCS file: /cvs/src/usr.sbin/tcpdump/print-bgp.c,v
> retrieving revision 1.29
> diff -u -p -r1.29 print-bgp.c
> --- print-bgp.c   3 Jul 2019 03:24:03 -   1.29
> +++ print-bgp.c   17 Jun 2021 13:24:33 -
> @@ -97,7 +97,7 @@ struct bgp_route_refresh {
>   u_int16_t len;
>   u_int8_t type;
>   u_int8_t afi[2]; /* unaligned; should be u_int16_t */
> - u_int8_t res;
> + u_int8_t subtype;
>   u_int8_t safi;
>  };
>  #define BGP_ROUTE_REFRESH_SIZE  23
> @@ -189,6 +189,7 @@ static const char *bgp_capcode[] = {
>   /* 67: [Chen] */ "DYNAMIC_CAPABILITY",
>   /* 68: [Appanna] */ "MULTISESSION",
>   /* 69: [draft-ietf-idr-add-paths] */ "ADD-PATH",
> + /* 70: RFC7313 */ "ENHANCED_ROUTE_REFRESH"
>  };
>  
>  #define bgp_capcode(x) \
> @@ -199,7 +200,7 @@ static const char *bgpnotify_major[] = {
>   NULL, "Message Header Error",
>   "OPEN Message Error", "UPDATE Message Error",
>   "Hold Timer Expired", "Finite State Machine Error",
> - "Cease", "Capability Message Error",
> + "Cease", "ROUTE_REFRESH Message Error",
>  };
>  #define bgp_notify_major(x) \
>   num_or_str(bgpnotify_major, \
> @@ -323,6 +324,11 @@ static const char *afnumber[] = AFNUM_NA
>   num_or_str(afnumber, \
>   sizeof(afnumber)/sizeof(afnumber[0]), (x)))
>  
> +static const char *refreshtype[] = {
> + "Normal", "BoRR", "EoRR"
> +};
> +#define refresh_subtype(x) \
> + num_or_str(refreshtype, sizeof(refreshtype)/sizeof(refreshtype[0]), (x))
>  
>  static const char *
>  num_or_str(const char **table, size_t siz, int value)
> @@ -1069,7 +1075,8 @@ bgp_route_refresh_print(const u_char *da
>  
>   bgp_route_refresh_header = (const struct bgp_route_refresh *)dat;
>  
> - printf(" (%s %s)",
> + printf(" %s (%s %s)",
> + refresh_subtype(bgp_route_refresh_header->subtype),
>   af_name(EXTRACT_16BITS(_route_refresh_header->afi)),
>   bgp_attr_nlri_safi(bgp_route_refresh_header->safi));
>  
> 

-- 
:wq Claudio



Re: bgpd support for enhanced route refresh

2021-06-17 Thread Claudio Jeker
On Mon, Jun 14, 2021 at 05:10:07PM +0200, Claudio Jeker wrote:
> On Thu, May 27, 2021 at 06:24:06PM +0200, Claudio Jeker wrote:
> > Implement RFC 7313 enhanced route refresh.
> > 
> > While there also change when graceful restart EoR markers are sent.
> > In short the graceful restart marker should only be sent initally. After
> > that the End of Route Refresh message should be sent instead.
> > Because of this track if an EoR marker was received or should be sent in
> > the peer config.
> > 
> > For now this setting is off by default but that may be changed at a later
> > state.
> > 
> > Please try this out and tell me if it works for you. The message and
> > prefix/rrefresh counters in bgpctl show nei output help a lot to see what
> > is going on.

Minor cleanup, instead of this comment use CTASSERT to make sure that the
recv_eor and send_eor fields are large enough to work as a bitfield for
AID_MAX

> -#define  AID_MAX 5
> +#define  AID_MAX 5   /* check rde_peer.recv_eor when max 
> reaches 7 */

Enhanced route refresh is currently off by default. So unless somebody
is against this I will commit this soon. At least then it gets tested :)

-- 
:wq Claudio


Index: bgpd.8
===
RCS file: /cvs/src/usr.sbin/bgpd/bgpd.8,v
retrieving revision 1.68
diff -u -p -r1.68 bgpd.8
--- bgpd.8  16 Jun 2021 16:24:12 -  1.68
+++ bgpd.8  17 Jun 2021 09:35:52 -
@@ -382,6 +382,15 @@ has been started.
 .Re
 .Pp
 .Rs
+.%A K. Patel
+.%A E. Chen
+.%A B. Venkatachalapathy
+.%D July 2014
+.%R RFC 7313
+.%T Enhanced Route Refresh Capability for BGP-4
+.Re
+.Pp
+.Rs
 .%A W. Kumari
 .%A R. Bush
 .%A H. Schiller
Index: bgpd.conf.5
===
RCS file: /cvs/src/usr.sbin/bgpd/bgpd.conf.5,v
retrieving revision 1.210
diff -u -p -r1.210 bgpd.conf.5
--- bgpd.conf.5 6 May 2021 09:21:35 -   1.210
+++ bgpd.conf.5 27 May 2021 16:03:30 -
@@ -831,6 +831,16 @@ The default is
 .Ic yes .
 .Pp
 .It Xo
+.Ic announce enhanced refresh
+.Pq Ic yes Ns | Ns Ic no
+.Xc
+If set to
+.Ic yes ,
+the enhanced route refresh capability is announced.
+The default is
+.Ic no .
+.Pp
+.It Xo
 .Ic announce refresh
 .Pq Ic yes Ns | Ns Ic no
 .Xc
Index: bgpd.h
===
RCS file: /cvs/src/usr.sbin/bgpd/bgpd.h,v
retrieving revision 1.414
diff -u -p -r1.414 bgpd.h
--- bgpd.h  27 May 2021 08:27:48 -  1.414
+++ bgpd.h  17 Jun 2021 10:13:28 -
@@ -96,6 +96,9 @@
 #defineF_CTL_OVS_NOTFOUND  0x20
 #defineF_CTL_NEIGHBORS 0x40 /* only used by bgpctl */
 
+#define CTASSERT(x)extern char  _ctassert[(x) ? 1 : -1 ] \
+   __attribute__((__unused__))
+
 /*
  * Note that these numeric assignments differ from the numbers commonly
  * used in route origin validation context.
Index: parse.y
===
RCS file: /cvs/src/usr.sbin/bgpd/parse.y,v
retrieving revision 1.416
diff -u -p -r1.416 parse.y
--- parse.y 20 May 2021 10:06:20 -  1.416
+++ parse.y 27 May 2021 16:03:30 -
@@ -204,7 +204,7 @@ typedef struct {
 %token GROUP NEIGHBOR NETWORK
 %token EBGP IBGP
 %token LOCALAS REMOTEAS DESCR LOCALADDR MULTIHOP PASSIVE MAXPREFIX RESTART
-%token ANNOUNCE CAPABILITIES REFRESH AS4BYTE CONNECTRETRY
+%token ANNOUNCE CAPABILITIES REFRESH AS4BYTE CONNECTRETRY ENHANCED
 %token DEMOTE ENFORCE NEIGHBORAS ASOVERRIDE REFLECTOR DEPEND DOWN
 %token DUMP IN OUT SOCKET RESTRICTED
 %token LOG TRANSPARENT
@@ -1446,6 +1446,9 @@ peeropts  : REMOTEAS as4number{
| ANNOUNCE REFRESH yesno {
curpeer->conf.capabilities.refresh = $3;
}
+   | ANNOUNCE ENHANCED REFRESH yesno {
+   curpeer->conf.capabilities.enhanced_rr = $4;
+   }
| ANNOUNCE RESTART yesno {
curpeer->conf.capabilities.grestart.restart = $3;
}
@@ -2898,6 +2901,7 @@ lookup(char *s)
{ "dump",   DUMP},
{ "ebgp",   EBGP},
{ "enforce",ENFORCE},
+   { "enhanced",   ENHANCED },
{ "esp",ESP},
{ "evaluate",   EVALUATE},
{ "export", EXPORT},
Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.525
diff -u -p -r1.525 rde.c
--- rde.c   17 Jun 2021 08:43:06 -  1.525
+++ rde.c   17 Jun 2021 09:35:52 -
@@ -1068,6 +1068,7 @@ rde_dispatch_imsg_rtr(struct imsgbuf *ib

bgpd refactor common code

2021-06-17 Thread Claudio Jeker
To not recreate the issue of missing another check in one of the
up_generate_updates() call points factor out the common code into
rde_skip_peer().

I hope this way a similar f-up can be avoided
-- 
:wq Claudio

? obj
Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.525
diff -u -p -r1.525 rde.c
--- rde.c   17 Jun 2021 08:43:06 -  1.525
+++ rde.c   17 Jun 2021 08:50:37 -
@@ -2879,6 +2879,28 @@ rde_evaluate_all(void)
return rde_eval_all;
 }
 
+static int
+rde_skip_peer(struct rde_peer *peer, u_int16_t rib_id, u_int8_t aid)
+{
+   /* skip ourself */
+   if (peer == peerself)
+   return 1;
+   if (peer->state != PEER_UP)
+   return 1;
+   /* skip peers using a different rib */
+   if (peer->loc_rib_id != rib_id)
+   return 1;
+   /* check if peer actually supports the address family */
+   if (peer->capa.mp[aid] == 0)
+   return 1;
+   /* skip peers with special export types */
+   if (peer->export_type == EXPORT_NONE ||
+   peer->export_type == EXPORT_DEFAULT_ROUTE)
+   return 1;
+
+   return 0;
+}
+
 void
 rde_generate_updates(struct rib *rib, struct prefix *new, struct prefix *old,
 int eval_all)
@@ -2903,20 +2925,7 @@ rde_generate_updates(struct rib *rib, st
aid = old->pt->aid;
 
LIST_FOREACH(peer, , peer_l) {
-   /* skip ourself */
-   if (peer == peerself)
-   continue;
-   if (peer->state != PEER_UP)
-   continue;
-   /* skip peers using a different rib */
-   if (peer->loc_rib_id != rib->id)
-   continue;
-   /* check if peer actually supports the address family */
-   if (peer->capa.mp[aid] == 0)
-   continue;
-   /* skip peers with special export types */
-   if (peer->export_type == EXPORT_NONE ||
-   peer->export_type == EXPORT_DEFAULT_ROUTE)
+   if (rde_skip_peer(peer, rib->id, aid))
continue;
/* skip regular peers if the best path didn't change */
if ((peer->flags & PEERFLAG_EVALUATE_ALL) == 0 && eval_all)
@@ -3571,20 +3580,7 @@ rde_softreconfig_out(struct rib_entry *r
return;
 
LIST_FOREACH(peer, , peer_l) {
-   /* skip ourself */
-   if (peer == peerself)
-   continue;
-   if (peer->state != PEER_UP)
-   continue;
-   /* skip peers using a different rib */
-   if (peer->loc_rib_id != p->re->rib_id)
-   continue;
-   /* check if peer actually supports the address family */
-   if (peer->capa.mp[aid] == 0)
-   continue;
-   /* skip peers with special export types */
-   if (peer->export_type == EXPORT_NONE ||
-   peer->export_type == EXPORT_DEFAULT_ROUTE)
+   if (rde_skip_peer(peer, re->rib_id, aid))
continue;
/* skip peers which don't need to reconfigure */
if (peer->reconf_out == 0)



  1   2   3   4   5   6   7   8   9   10   >