Re: syn cache hash size sysctl

[Joerg Jung]

> Am 19.07.2016 um 23:16 schrieb Alexander Bluhm :
> 
>> On Tue, Jul 19, 2016 at 09:48:19PM +0100, Jason McIntyre wrote:
>> oh oh. i should have been clearer: they are sorted in sysctl(3), but in
>> sysctl(8) they are merely listed in the order that running "sysctl"
>> dumps them. so no sort neccessary for sysctl(8).
> 
> So now sysctl(8) has all net.inet.tcp in sysctl output order.
> 
> ok?

ok jung@

> bluhm
> 
> Index: lib/libc/gen/sysctl.3
> ===
> RCS file: /data/mirror/openbsd/cvs/src/lib/libc/gen/sysctl.3,v
> retrieving revision 1.266
> diff -u -p -r1.266 sysctl.3
> --- lib/libc/gen/sysctl.314 Jul 2016 17:34:06 -1.266
> +++ lib/libc/gen/sysctl.319 Jul 2016 20:36:19 -
> @@ -1188,6 +1188,7 @@ The currently defined protocols and name
> .It tcp Ta stats Ta structure Ta no
> .It tcp Ta synbucketlimit Ta integer Ta yes
> .It tcp Ta syncachelimit Ta integer Ta yes
> +.It tcp Ta synhashsize Ta integer Ta yes
> .It tcp Ta synuselimit Ta integer Ta yes
> .It udp Ta baddynamic Ta array Ta yes
> .It udp Ta checksum Ta integer Ta yes
> @@ -1617,6 +1618,10 @@ Returns the TCP statistics in a struct t
> The maximum number of entries allowed per hash bucket in the TCP SYN cache.
> .It Li tcp.syncachelimit
> The maximum number of entries allowed in the TCP SYN cache.
> +.It Li tcp.synhashsize
> +The number of buckets in the TCP SYN cache hash array.
> +After the value is set, the actual size changes when the alternative
> +SYN cache becomes empty and both SYN caches are swapped.
> .It Li tcp.synuselimit
> The minimum number of times the hash function for the TCP SYN cache is used
> before it is reseeded.
> Index: sbin/sysctl/sysctl.8
> ===
> RCS file: /data/mirror/openbsd/cvs/src/sbin/sysctl/sysctl.8,v
> retrieving revision 1.202
> diff -u -p -r1.202 sysctl.8
> --- sbin/sysctl/sysctl.85 Jul 2016 17:41:59 -1.202
> +++ sbin/sysctl/sysctl.819 Jul 2016 21:06:02 -
> @@ -256,7 +256,6 @@ and a few require a kernel compiled with
> .It net.inet.tcp.keepinittime Ta integer Ta yes
> .It net.inet.tcp.keepidle Ta integer Ta yes
> .It net.inet.tcp.keepintvl Ta integer Ta yes
> -.It net.inet.tcp.always_keepalive Ta integer Ta yes
> .It net.inet.tcp.slowhz Ta integer Ta no
> .It net.inet.tcp.baddynamic Ta array Ta yes
> .It net.inet.tcp.sack Ta integer Ta yes
> @@ -266,10 +265,13 @@ and a few require a kernel compiled with
> .It net.inet.tcp.ecn Ta integer Ta yes
> .It net.inet.tcp.syncachelimit Ta integer Ta yes
> .It net.inet.tcp.synbucketlimit Ta integer Ta yes
> -.It net.inet.tcp.synuselimit Ta integer Ta yes
> .It net.inet.tcp.rfc3390 Ta integer Ta yes
> .It net.inet.tcp.reasslimit Ta integer Ta yes
> +.It net.inet.tcp.sackholelimit Ta integer Ta yes
> +.It net.inet.tcp.always_keepalive Ta integer Ta yes
> +.It net.inet.tcp.synuselimit Ta integer Ta yes
> .It net.inet.tcp.rootonly Ta array Ta yes
> +.It net.inet.tcp.synhashsize Ta integer Ta yes
> .It net.inet.udp.checksum Ta integer Ta yes
> .It net.inet.udp.baddynamic Ta array Ta yes
> .It net.inet.udp.recvspace Ta integer Ta yes
> 

arm/pmap7 _enter and PV_BEEN_xD()

[Artturi Alm]

Hi,

#define PV_BEEN_EXECD(f)  (((f) & (PVF_REF | PVF_EXEC)) == (PVF_REF | PVF_EXEC))
#define PV_BEEN_REFD(f)   (((f) & PVF_REF) != 0)

and from pmap.h:
 * The PVF_MOD and PVF_REF flags are stored in the mdpage for each
 * page.  PVF_WIRED, PVF_WRITE, and PVF_NC are kept in individual
 * pv_entry's for each page.  They live in the same "namespace" so
 * that we can clear multiple attributes at a time.

well, this isn't stricly about clearing, but using PVF_REF in those macros
does mean you are likely to want | attrs too.

-Artturi

diff --git a/sys/arch/arm/arm/pmap7.c b/sys/arch/arm/arm/pmap7.c
index 0d32bf9..64c1227 100644
--- a/sys/arch/arm/arm/pmap7.c
+++ b/sys/arch/arm/arm/pmap7.c
@@ -1247,8 +1247,7 @@ pmap_enter(pmap_t pm, vaddr_t va, paddr_t pa, vm_prot_t 
prot, int flags)
struct vm_page *pg, *opg;
struct pv_entry *pve;
pt_entry_t *ptep, npte, opte;
-   u_int nflags;
-   u_int oflags;
+   u_int nflags, oflags, oattrs;
int mapped = 1;
 
NPDEBUG(PDB_ENTER, printf("pmap_enter: pm %p va 0x%lx pa 0x%lx prot %x 
flag %x\n", pm, va, pa, prot, flags));
@@ -1287,7 +1286,7 @@ pmap_enter(pmap_t pm, vaddr_t va, paddr_t pa, vm_prot_t 
prot, int flags)
ptep = >l2b_kva[l2pte_index(va)];
opte = *ptep;
npte = pa;
-   oflags = 0;
+   oflags = oattrs = 0;
 
if (opte != 0) {/* not l2pte_valid!!! MIOD */
/*
@@ -1346,6 +1345,7 @@ pmap_enter(pmap_t pm, vaddr_t va, paddr_t pa, vm_prot_t 
prot, int flags)
/*
 * We're changing the attrs of an existing mapping.
 */
+   oattrs = opg->mdpage.pvh_attrs;
oflags = pmap_modify_pv(pg, pm, va,
PVF_WRITE | PVF_EXEC | PVF_WIRED |
PVF_MOD | PVF_REF, nflags);
@@ -1372,6 +1372,7 @@ pmap_enter(pmap_t pm, vaddr_t va, paddr_t pa, vm_prot_t 
prot, int flags)
 * It is part of our managed memory so we
 * must remove it from the PV list
 */
+   oattrs = opg->mdpage.pvh_attrs;
pve = pmap_remove_pv(opg, pm, va);
oflags = pve->pv_flags;
} else
@@ -1402,6 +1403,7 @@ pmap_enter(pmap_t pm, vaddr_t va, paddr_t pa, vm_prot_t 
prot, int flags)
 * Looks like there's an existing 'managed' mapping
 * at this address.
 */
+   oattrs = opg->mdpage.pvh_attrs;
pve = pmap_remove_pv(opg, pm, va);
oflags = pve->pv_flags;
 
@@ -1455,10 +1457,10 @@ pmap_enter(pmap_t pm, vaddr_t va, paddr_t pa, vm_prot_t 
prot, int flags)
}
}
 
-   if (PV_BEEN_EXECD(oflags))
+   if (PV_BEEN_EXECD(oflags | oattrs))
pmap_tlb_flushID_SE(pm, va);
else
-   if (PV_BEEN_REFD(oflags))
+   if (PV_BEEN_REFD(oflags | oattrs))
pmap_tlb_flushD_SE(pm, va);
}
 

Re: tcp output mbuf leak

[Mike Belopuhov]

On 19 July 2016 at 23:00, Alexander Bluhm  wrote:
> Hi,
>
> When looking at the error paths in tcp_output() I have found these
> returns that look like mbuf leaks.
>
> ok?
>
> bluhm
>

looks correct indeed. ok mikeb

Re: syn cache hash size sysctl

[Alexander Bluhm]

On Tue, Jul 19, 2016 at 09:48:19PM +0100, Jason McIntyre wrote:
> oh oh. i should have been clearer: they are sorted in sysctl(3), but in
> sysctl(8) they are merely listed in the order that running "sysctl"
> dumps them. so no sort neccessary for sysctl(8).

So now sysctl(8) has all net.inet.tcp in sysctl output order.

ok?

bluhm

Index: lib/libc/gen/sysctl.3
===
RCS file: /data/mirror/openbsd/cvs/src/lib/libc/gen/sysctl.3,v
retrieving revision 1.266
diff -u -p -r1.266 sysctl.3
--- lib/libc/gen/sysctl.3   14 Jul 2016 17:34:06 -  1.266
+++ lib/libc/gen/sysctl.3   19 Jul 2016 20:36:19 -
@@ -1188,6 +1188,7 @@ The currently defined protocols and name
 .It tcp Ta stats Ta structure Ta no
 .It tcp Ta synbucketlimit Ta integer Ta yes
 .It tcp Ta syncachelimit Ta integer Ta yes
+.It tcp Ta synhashsize Ta integer Ta yes
 .It tcp Ta synuselimit Ta integer Ta yes
 .It udp Ta baddynamic Ta array Ta yes
 .It udp Ta checksum Ta integer Ta yes
@@ -1617,6 +1618,10 @@ Returns the TCP statistics in a struct t
 The maximum number of entries allowed per hash bucket in the TCP SYN cache.
 .It Li tcp.syncachelimit
 The maximum number of entries allowed in the TCP SYN cache.
+.It Li tcp.synhashsize
+The number of buckets in the TCP SYN cache hash array.
+After the value is set, the actual size changes when the alternative
+SYN cache becomes empty and both SYN caches are swapped.
 .It Li tcp.synuselimit
 The minimum number of times the hash function for the TCP SYN cache is used
 before it is reseeded.
Index: sbin/sysctl/sysctl.8
===
RCS file: /data/mirror/openbsd/cvs/src/sbin/sysctl/sysctl.8,v
retrieving revision 1.202
diff -u -p -r1.202 sysctl.8
--- sbin/sysctl/sysctl.85 Jul 2016 17:41:59 -   1.202
+++ sbin/sysctl/sysctl.819 Jul 2016 21:06:02 -
@@ -256,7 +256,6 @@ and a few require a kernel compiled with
 .It net.inet.tcp.keepinittime Ta integer Ta yes
 .It net.inet.tcp.keepidle Ta integer Ta yes
 .It net.inet.tcp.keepintvl Ta integer Ta yes
-.It net.inet.tcp.always_keepalive Ta integer Ta yes
 .It net.inet.tcp.slowhz Ta integer Ta no
 .It net.inet.tcp.baddynamic Ta array Ta yes
 .It net.inet.tcp.sack Ta integer Ta yes
@@ -266,10 +265,13 @@ and a few require a kernel compiled with
 .It net.inet.tcp.ecn Ta integer Ta yes
 .It net.inet.tcp.syncachelimit Ta integer Ta yes
 .It net.inet.tcp.synbucketlimit Ta integer Ta yes
-.It net.inet.tcp.synuselimit Ta integer Ta yes
 .It net.inet.tcp.rfc3390 Ta integer Ta yes
 .It net.inet.tcp.reasslimit Ta integer Ta yes
+.It net.inet.tcp.sackholelimit Ta integer Ta yes
+.It net.inet.tcp.always_keepalive Ta integer Ta yes
+.It net.inet.tcp.synuselimit Ta integer Ta yes
 .It net.inet.tcp.rootonly Ta array Ta yes
+.It net.inet.tcp.synhashsize Ta integer Ta yes
 .It net.inet.udp.checksum Ta integer Ta yes
 .It net.inet.udp.baddynamic Ta array Ta yes
 .It net.inet.udp.recvspace Ta integer Ta yes

Re: tcp output mbuf leak

[Claudio Jeker]

On Tue, Jul 19, 2016 at 11:00:04PM +0200, Alexander Bluhm wrote:
> Hi,
> 
> When looking at the error paths in tcp_output() I have found these
> returns that look like mbuf leaks.
> 
> ok?

Indeed. OK claudio@

Looking at tcp_signature() I actually think it can not fail but better
safe than sorry.
 
> bluhm
> 
> Index: netinet/tcp_output.c
> ===
> RCS file: /data/mirror/openbsd/cvs/src/sys/netinet/tcp_output.c,v
> retrieving revision 1.117
> diff -u -p -r1.117 tcp_output.c
> --- netinet/tcp_output.c  13 Jun 2016 21:24:43 -  1.117
> +++ netinet/tcp_output.c  19 Jul 2016 20:56:51 -
> @@ -931,12 +931,16 @@ send:
>  
>   tdb = gettdbbysrcdst(rtable_l2(tp->t_inpcb->inp_rtableid),
>   0, , , IPPROTO_TCP);
> - if (tdb == NULL)
> + if (tdb == NULL) {
> + m_freem(m);
>   return (EPERM);
> + }
>  
>   if (tcp_signature(tdb, tp->pf, m, th, iphlen, 0,
> - mtod(m, caddr_t) + hdrlen - optlen + sigoff) < 0)
> + mtod(m, caddr_t) + hdrlen - optlen + sigoff) < 0) {
> + m_freem(m);
>   return (EINVAL);
> + }
>   }
>  #endif /* TCP_SIGNATURE */
>  
> 

-- 
:wq Claudio

Re: syn cache hash size sysctl

[Claudio Jeker]

On Tue, Jul 19, 2016 at 10:40:14PM +0200, Alexander Bluhm wrote:
> On Tue, Jul 19, 2016 at 09:19:25PM +0100, Jason McIntyre wrote:
> > On Tue, Jul 19, 2016 at 10:09:47PM +0200, Alexander Bluhm wrote:
> > > On Tue, Jul 19, 2016 at 08:55:58PM +0200, Joerg Jung wrote:
> > > > Please, also document it, at least in sysctl(8).
> 
> Next try, with input from jmc@
> 
> bluhm
> 
> Index: lib/libc/gen/sysctl.3
> ===
> RCS file: /data/mirror/openbsd/cvs/src/lib/libc/gen/sysctl.3,v
> retrieving revision 1.266
> diff -u -p -r1.266 sysctl.3
> --- lib/libc/gen/sysctl.3 14 Jul 2016 17:34:06 -  1.266
> +++ lib/libc/gen/sysctl.3 19 Jul 2016 20:36:19 -
> @@ -1188,6 +1188,7 @@ The currently defined protocols and name
>  .It tcp Ta stats Ta structure Ta no
>  .It tcp Ta synbucketlimit Ta integer Ta yes
>  .It tcp Ta syncachelimit Ta integer Ta yes
> +.It tcp Ta synhashsize Ta integer Ta yes
>  .It tcp Ta synuselimit Ta integer Ta yes
>  .It udp Ta baddynamic Ta array Ta yes
>  .It udp Ta checksum Ta integer Ta yes
> @@ -1617,6 +1618,10 @@ Returns the TCP statistics in a struct t
>  The maximum number of entries allowed per hash bucket in the TCP SYN cache.
>  .It Li tcp.syncachelimit
>  The maximum number of entries allowed in the TCP SYN cache.
> +.It Li tcp.synhashsize
> +The number of buckets in the TCP SYN cache hash array.
> +After the value is set, the actual size changes when the alternative
> +SYN cache becomes empty and both SYN caches are swapped.
>  .It Li tcp.synuselimit
>  The minimum number of times the hash function for the TCP SYN cache is used
>  before it is reseeded.
> Index: sbin/sysctl/sysctl.8
> ===
> RCS file: /data/mirror/openbsd/cvs/src/sbin/sysctl/sysctl.8,v
> retrieving revision 1.202
> diff -u -p -r1.202 sysctl.8
> --- sbin/sysctl/sysctl.8  5 Jul 2016 17:41:59 -   1.202
> +++ sbin/sysctl/sysctl.8  19 Jul 2016 20:31:39 -
> @@ -264,8 +264,9 @@ and a few require a kernel compiled with
>  .It net.inet.tcp.rstppslimit Ta integer Ta yes
>  .It net.inet.tcp.ackonpush Ta integer Ta yes
>  .It net.inet.tcp.ecn Ta integer Ta yes
> -.It net.inet.tcp.syncachelimit Ta integer Ta yes
>  .It net.inet.tcp.synbucketlimit Ta integer Ta yes
> +.It net.inet.tcp.syncachelimit Ta integer Ta yes
> +.It net.inet.tcp.synhashsize Ta integer Ta yes
>  .It net.inet.tcp.synuselimit Ta integer Ta yes
>  .It net.inet.tcp.rfc3390 Ta integer Ta yes
>  .It net.inet.tcp.reasslimit Ta integer Ta yes
> 

Reads OK. I would love to actually reduce the syncache knobs to one or two
and have the kernel select the other two values based on the first two.
As in keep synuselimit and syncachelimit and ditch synbucketlimit and
synhashsize. I always have to look up the code to make usefull decisions
about these values.

-- 
:wq Claudio

tcp output mbuf leak

[Alexander Bluhm]

Hi,

When looking at the error paths in tcp_output() I have found these
returns that look like mbuf leaks.

ok?

bluhm

Index: netinet/tcp_output.c
===
RCS file: /data/mirror/openbsd/cvs/src/sys/netinet/tcp_output.c,v
retrieving revision 1.117
diff -u -p -r1.117 tcp_output.c
--- netinet/tcp_output.c13 Jun 2016 21:24:43 -  1.117
+++ netinet/tcp_output.c19 Jul 2016 20:56:51 -
@@ -931,12 +931,16 @@ send:
 
tdb = gettdbbysrcdst(rtable_l2(tp->t_inpcb->inp_rtableid),
0, , , IPPROTO_TCP);
-   if (tdb == NULL)
+   if (tdb == NULL) {
+   m_freem(m);
return (EPERM);
+   }
 
if (tcp_signature(tdb, tp->pf, m, th, iphlen, 0,
-   mtod(m, caddr_t) + hdrlen - optlen + sigoff) < 0)
+   mtod(m, caddr_t) + hdrlen - optlen + sigoff) < 0) {
+   m_freem(m);
return (EINVAL);
+   }
}
 #endif /* TCP_SIGNATURE */
 

Re: syn cache hash size sysctl

[Alexander Bluhm]

On Tue, Jul 19, 2016 at 09:19:25PM +0100, Jason McIntyre wrote:
> On Tue, Jul 19, 2016 at 10:09:47PM +0200, Alexander Bluhm wrote:
> > On Tue, Jul 19, 2016 at 08:55:58PM +0200, Joerg Jung wrote:
> > > Please, also document it, at least in sysctl(8).

Next try, with input from jmc@

bluhm

Index: lib/libc/gen/sysctl.3
===
RCS file: /data/mirror/openbsd/cvs/src/lib/libc/gen/sysctl.3,v
retrieving revision 1.266
diff -u -p -r1.266 sysctl.3
--- lib/libc/gen/sysctl.3   14 Jul 2016 17:34:06 -  1.266
+++ lib/libc/gen/sysctl.3   19 Jul 2016 20:36:19 -
@@ -1188,6 +1188,7 @@ The currently defined protocols and name
 .It tcp Ta stats Ta structure Ta no
 .It tcp Ta synbucketlimit Ta integer Ta yes
 .It tcp Ta syncachelimit Ta integer Ta yes
+.It tcp Ta synhashsize Ta integer Ta yes
 .It tcp Ta synuselimit Ta integer Ta yes
 .It udp Ta baddynamic Ta array Ta yes
 .It udp Ta checksum Ta integer Ta yes
@@ -1617,6 +1618,10 @@ Returns the TCP statistics in a struct t
 The maximum number of entries allowed per hash bucket in the TCP SYN cache.
 .It Li tcp.syncachelimit
 The maximum number of entries allowed in the TCP SYN cache.
+.It Li tcp.synhashsize
+The number of buckets in the TCP SYN cache hash array.
+After the value is set, the actual size changes when the alternative
+SYN cache becomes empty and both SYN caches are swapped.
 .It Li tcp.synuselimit
 The minimum number of times the hash function for the TCP SYN cache is used
 before it is reseeded.
Index: sbin/sysctl/sysctl.8
===
RCS file: /data/mirror/openbsd/cvs/src/sbin/sysctl/sysctl.8,v
retrieving revision 1.202
diff -u -p -r1.202 sysctl.8
--- sbin/sysctl/sysctl.85 Jul 2016 17:41:59 -   1.202
+++ sbin/sysctl/sysctl.819 Jul 2016 20:31:39 -
@@ -264,8 +264,9 @@ and a few require a kernel compiled with
 .It net.inet.tcp.rstppslimit Ta integer Ta yes
 .It net.inet.tcp.ackonpush Ta integer Ta yes
 .It net.inet.tcp.ecn Ta integer Ta yes
-.It net.inet.tcp.syncachelimit Ta integer Ta yes
 .It net.inet.tcp.synbucketlimit Ta integer Ta yes
+.It net.inet.tcp.syncachelimit Ta integer Ta yes
+.It net.inet.tcp.synhashsize Ta integer Ta yes
 .It net.inet.tcp.synuselimit Ta integer Ta yes
 .It net.inet.tcp.rfc3390 Ta integer Ta yes
 .It net.inet.tcp.reasslimit Ta integer Ta yes

Re: syn cache hash size sysctl

[Alexander Bluhm]

On Tue, Jul 19, 2016 at 08:55:58PM +0200, Joerg Jung wrote:
> Please, also document it, at least in sysctl(8).

like this?

bluhm

Index: lib/libc/gen/sysctl.3
===
RCS file: /data/mirror/openbsd/cvs/src/lib/libc/gen/sysctl.3,v
retrieving revision 1.266
diff -u -p -r1.266 sysctl.3
--- lib/libc/gen/sysctl.3   14 Jul 2016 17:34:06 -  1.266
+++ lib/libc/gen/sysctl.3   19 Jul 2016 20:06:54 -
@@ -1189,6 +1189,7 @@ The currently defined protocols and name
 .It tcp Ta synbucketlimit Ta integer Ta yes
 .It tcp Ta syncachelimit Ta integer Ta yes
 .It tcp Ta synuselimit Ta integer Ta yes
+.It tcp Ta synhashsize Ta integer Ta yes
 .It udp Ta baddynamic Ta array Ta yes
 .It udp Ta checksum Ta integer Ta yes
 .It udp Ta recvspace Ta integer Ta yes
@@ -1619,7 +1620,11 @@ The maximum number of entries allowed pe
 The maximum number of entries allowed in the TCP SYN cache.
 .It Li tcp.synuselimit
 The minimum number of times the hash function for the TCP SYN cache is used
-before it is reseeded.
+before it gets reseeded.
+.It Li tcp.synhashsize
+The number of buckets in the TCP SYN cache hash array.
+After the value is set, the actual size changes when the alternative
+SYN cache gets empty and both SYN caches are swapped.
 .It Li udp.baddynamic
 Analogous to
 .Li tcp.baddynamic
Index: sbin/sysctl/sysctl.8
===
RCS file: /data/mirror/openbsd/cvs/src/sbin/sysctl/sysctl.8,v
retrieving revision 1.202
diff -u -p -r1.202 sysctl.8
--- sbin/sysctl/sysctl.85 Jul 2016 17:41:59 -   1.202
+++ sbin/sysctl/sysctl.819 Jul 2016 19:53:16 -
@@ -267,6 +267,7 @@ and a few require a kernel compiled with
 .It net.inet.tcp.syncachelimit Ta integer Ta yes
 .It net.inet.tcp.synbucketlimit Ta integer Ta yes
 .It net.inet.tcp.synuselimit Ta integer Ta yes
+.It net.inet.tcp.synhashsize Ta integer Ta yes
 .It net.inet.tcp.rfc3390 Ta integer Ta yes
 .It net.inet.tcp.reasslimit Ta integer Ta yes
 .It net.inet.tcp.rootonly Ta array Ta yes

Re: syn cache hash size sysctl

[Joerg Jung]

On Tue, Jul 19, 2016 at 06:13:42PM +0200, Alexander Bluhm wrote:
> Hi,
> 
> claudio@ suggested to have a tunable size for the syn cache hash
> array.  As we are swapping between two syn caches for random reseeding
> anyway, this feature can be added easily.  When the cache is empty,
> we can change the hash size.
> 
> This allows an admin under SYN flood attack to tune his machine.
> sysctl net.inet.tcp.synhashsize=1

Makes sense to me and I like this.
 
> ok?

ok jung@

Please, also document it, at least in sysctl(8).
 
> bluhm
> 
> Index: netinet/tcp_input.c
> ===
> RCS file: /data/mirror/openbsd/cvs/src/sys/netinet/tcp_input.c,v
> retrieving revision 1.324
> diff -u -p -r1.324 tcp_input.c
> --- netinet/tcp_input.c   1 Jul 2016 18:37:15 -   1.324
> +++ netinet/tcp_input.c   19 Jul 2016 15:02:35 -
> @@ -3266,7 +3266,7 @@ tcp_mss_adv(struct mbuf *m, int af)
>   */
>  
>  /* syn hash parameters */
> -int  tcp_syn_cache_size = TCP_SYN_HASH_SIZE;
> +int  tcp_syn_hash_size = TCP_SYN_HASH_SIZE;
>  int  tcp_syn_cache_limit = TCP_SYN_HASH_SIZE*TCP_SYN_BUCKET_SIZE;
>  int  tcp_syn_bucket_limit = 3*TCP_SYN_BUCKET_SIZE;
>  int  tcp_syn_use_limit = 10;
> @@ -3360,7 +3360,13 @@ syn_cache_init(void)
>   int i;
>  
>   /* Initialize the hash buckets. */
> - for (i = 0; i < tcp_syn_cache_size; i++) {
> + tcp_syn_cache[0].scs_buckethead = mallocarray(tcp_syn_hash_size,
> + sizeof(struct syn_cache_head), M_SYNCACHE, M_WAITOK|M_ZERO);
> + tcp_syn_cache[1].scs_buckethead = mallocarray(tcp_syn_hash_size,
> + sizeof(struct syn_cache_head), M_SYNCACHE, M_WAITOK|M_ZERO);
> + tcp_syn_cache[0].scs_size = tcp_syn_hash_size;
> + tcp_syn_cache[1].scs_size = tcp_syn_hash_size;
> + for (i = 0; i < tcp_syn_hash_size; i++) {
>   TAILQ_INIT(_syn_cache[0].scs_buckethead[i].sch_bucket);
>   TAILQ_INIT(_syn_cache[1].scs_buckethead[i].sch_bucket);
>   }
> @@ -3377,7 +3383,7 @@ syn_cache_insert(struct syn_cache *sc, s
>   struct syn_cache_set *set = _syn_cache[tcp_syn_cache_active];
>   struct syn_cache_head *scp;
>   struct syn_cache *sc2;
> - int s;
> + int i, s;
>  
>   s = splsoftnet();
>  
> @@ -3385,16 +3391,33 @@ syn_cache_insert(struct syn_cache *sc, s
>* If there are no entries in the hash table, reinitialize
>* the hash secrets.  To avoid useless cache swaps and
>* reinitialization, use it until the limit is reached.
> +  * An emtpy cache is also the oportunity to resize the hash.
>*/
>   if (set->scs_count == 0 && set->scs_use <= 0) {
> - arc4random_buf(set->scs_random, sizeof(set->scs_random));
>   set->scs_use = tcp_syn_use_limit;
> + if (set->scs_size != tcp_syn_hash_size) {
> + scp = mallocarray(tcp_syn_hash_size, sizeof(struct
> + syn_cache_head), M_SYNCACHE, M_NOWAIT|M_ZERO);
> + if (scp == NULL) {
> + /* Try again next time. */
> + set->scs_use = 0;
> + } else {
> + free(set->scs_buckethead, M_SYNCACHE,
> + set->scs_size *
> + sizeof(struct syn_cache_head));
> + set->scs_buckethead = scp;
> + set->scs_size = tcp_syn_hash_size;
> + for (i = 0; i < tcp_syn_hash_size; i++)
> + TAILQ_INIT([i].sch_bucket);
> + }
> + }
> + arc4random_buf(set->scs_random, sizeof(set->scs_random));
>   tcpstat.tcps_sc_seedrandom++;
>   }
>  
>   SYN_HASHALL(sc->sc_hash, >sc_src.sa, >sc_dst.sa,
>   set->scs_random);
> - scp = >scs_buckethead[sc->sc_hash % tcp_syn_cache_size];
> + scp = >scs_buckethead[sc->sc_hash % set->scs_size];
>   sc->sc_buckethead = scp;
>  
>   /*
> @@ -3437,7 +3460,7 @@ syn_cache_insert(struct syn_cache *sc, s
>*/
>   scp2 = scp;
>   if (TAILQ_EMPTY(>sch_bucket)) {
> - sce = >scs_buckethead[tcp_syn_cache_size];
> + sce = >scs_buckethead[set->scs_size];
>   for (++scp2; scp2 != scp; scp2++) {
>   if (scp2 >= sce)
>   scp2 = >scs_buckethead[0];
> @@ -3595,7 +3618,7 @@ syn_cache_lookup(struct sockaddr *src, s
>   if (sets[i]->scs_count == 0)
>   continue;
>   SYN_HASHALL(hash, src, dst, sets[i]->scs_random);
> - scp = [i]->scs_buckethead[hash % tcp_syn_cache_size];
> + scp = [i]->scs_buckethead[hash % sets[i]->scs_size];
>   *headp = scp;
>   TAILQ_FOREACH(sc, >sch_bucket, sc_bucketq) 

syn cache hash size sysctl

[Alexander Bluhm]

Hi,

claudio@ suggested to have a tunable size for the syn cache hash
array.  As we are swapping between two syn caches for random reseeding
anyway, this feature can be added easily.  When the cache is empty,
we can change the hash size.

This allows an admin under SYN flood attack to tune his machine.
sysctl net.inet.tcp.synhashsize=1

ok?

bluhm

Index: netinet/tcp_input.c
===
RCS file: /data/mirror/openbsd/cvs/src/sys/netinet/tcp_input.c,v
retrieving revision 1.324
diff -u -p -r1.324 tcp_input.c
--- netinet/tcp_input.c 1 Jul 2016 18:37:15 -   1.324
+++ netinet/tcp_input.c 19 Jul 2016 15:02:35 -
@@ -3266,7 +3266,7 @@ tcp_mss_adv(struct mbuf *m, int af)
  */
 
 /* syn hash parameters */
-inttcp_syn_cache_size = TCP_SYN_HASH_SIZE;
+inttcp_syn_hash_size = TCP_SYN_HASH_SIZE;
 inttcp_syn_cache_limit = TCP_SYN_HASH_SIZE*TCP_SYN_BUCKET_SIZE;
 inttcp_syn_bucket_limit = 3*TCP_SYN_BUCKET_SIZE;
 inttcp_syn_use_limit = 10;
@@ -3360,7 +3360,13 @@ syn_cache_init(void)
int i;
 
/* Initialize the hash buckets. */
-   for (i = 0; i < tcp_syn_cache_size; i++) {
+   tcp_syn_cache[0].scs_buckethead = mallocarray(tcp_syn_hash_size,
+   sizeof(struct syn_cache_head), M_SYNCACHE, M_WAITOK|M_ZERO);
+   tcp_syn_cache[1].scs_buckethead = mallocarray(tcp_syn_hash_size,
+   sizeof(struct syn_cache_head), M_SYNCACHE, M_WAITOK|M_ZERO);
+   tcp_syn_cache[0].scs_size = tcp_syn_hash_size;
+   tcp_syn_cache[1].scs_size = tcp_syn_hash_size;
+   for (i = 0; i < tcp_syn_hash_size; i++) {
TAILQ_INIT(_syn_cache[0].scs_buckethead[i].sch_bucket);
TAILQ_INIT(_syn_cache[1].scs_buckethead[i].sch_bucket);
}
@@ -3377,7 +3383,7 @@ syn_cache_insert(struct syn_cache *sc, s
struct syn_cache_set *set = _syn_cache[tcp_syn_cache_active];
struct syn_cache_head *scp;
struct syn_cache *sc2;
-   int s;
+   int i, s;
 
s = splsoftnet();
 
@@ -3385,16 +3391,33 @@ syn_cache_insert(struct syn_cache *sc, s
 * If there are no entries in the hash table, reinitialize
 * the hash secrets.  To avoid useless cache swaps and
 * reinitialization, use it until the limit is reached.
+* An emtpy cache is also the oportunity to resize the hash.
 */
if (set->scs_count == 0 && set->scs_use <= 0) {
-   arc4random_buf(set->scs_random, sizeof(set->scs_random));
set->scs_use = tcp_syn_use_limit;
+   if (set->scs_size != tcp_syn_hash_size) {
+   scp = mallocarray(tcp_syn_hash_size, sizeof(struct
+   syn_cache_head), M_SYNCACHE, M_NOWAIT|M_ZERO);
+   if (scp == NULL) {
+   /* Try again next time. */
+   set->scs_use = 0;
+   } else {
+   free(set->scs_buckethead, M_SYNCACHE,
+   set->scs_size *
+   sizeof(struct syn_cache_head));
+   set->scs_buckethead = scp;
+   set->scs_size = tcp_syn_hash_size;
+   for (i = 0; i < tcp_syn_hash_size; i++)
+   TAILQ_INIT([i].sch_bucket);
+   }
+   }
+   arc4random_buf(set->scs_random, sizeof(set->scs_random));
tcpstat.tcps_sc_seedrandom++;
}
 
SYN_HASHALL(sc->sc_hash, >sc_src.sa, >sc_dst.sa,
set->scs_random);
-   scp = >scs_buckethead[sc->sc_hash % tcp_syn_cache_size];
+   scp = >scs_buckethead[sc->sc_hash % set->scs_size];
sc->sc_buckethead = scp;
 
/*
@@ -3437,7 +3460,7 @@ syn_cache_insert(struct syn_cache *sc, s
 */
scp2 = scp;
if (TAILQ_EMPTY(>sch_bucket)) {
-   sce = >scs_buckethead[tcp_syn_cache_size];
+   sce = >scs_buckethead[set->scs_size];
for (++scp2; scp2 != scp; scp2++) {
if (scp2 >= sce)
scp2 = >scs_buckethead[0];
@@ -3595,7 +3618,7 @@ syn_cache_lookup(struct sockaddr *src, s
if (sets[i]->scs_count == 0)
continue;
SYN_HASHALL(hash, src, dst, sets[i]->scs_random);
-   scp = [i]->scs_buckethead[hash % tcp_syn_cache_size];
+   scp = [i]->scs_buckethead[hash % sets[i]->scs_size];
*headp = scp;
TAILQ_FOREACH(sc, >sch_bucket, sc_bucketq) {
if (sc->sc_hash != hash)
Index: netinet/tcp_usrreq.c
===
RCS file: /data/mirror/openbsd/cvs/src/sys/netinet/tcp_usrreq.c,v
retrieving revision 1.132

Re: relayd TLS session caching

[Philip Guenther]

On Tue, 19 Jul 2016, Sebastian Benoit wrote:
> maybe session cache disable should disable tickets too.

Well, what problem are you trying to solve by offering that option?

If it's to save memory, because original-flavor session caching requires 
state and overhead on the server, then you should only disable it and not 
tickets.

If it's because this relayed is for handling some resource which is SO 
SPECIAL that you want fresh proof of liveness for every new connection and 
thus totally disable session reuse, then sure, disable tickets too.

Which is the real case?


Maybe you should throw in some stats tracking too see how often the 
different flavors of caching are used in your real-world use.  Maybe 
original-flavor session caching should be disabled because no one real is 
using it and it's just more exposure to leave on...


Philip

Re: relayd TLS session caching

[Sebastian Benoit]

maybe session cache disable should disable tickets too.

some little things below, otherwise ok

Claudio Jeker(cje...@diehard.n-r-g.com) on 2016.07.19 15:32:13 +0200:
> At the moment relayd's TLS session caching is a bit busted because
> the multiple relay processes do not share state.
> The following diff adds SSL session caching and sharing of the TLS ticket
> secrets. Which this openssl s_client -connect W.X.Y.Z:443 -reconnect
> reuses the connection after the first one.
> This should help TLS preformance since no exchange with the ca process is
> needed if there is a cache hit.
> 
> The shared SSL session cache is in a new process (tlsc) but we still use
> the internal cache as well (which is populated on cache miss).
> 
> The TLS token secret are generated on startup and a key rollover happens
> every 4h with a rekey window of 10min. I see no need to add tunables for
> this and also the shared SSL cache will just use the tls session cache
> settings (because OpenSSL will do the garbage collection of the external
> cache via the internal one).
> 
> This is the first version which seems to work for me when hammering relayd
> with openssl s_client.
> -- 
> :wq Claudio
> 
> 
> Index: Makefile
> ===
> RCS file: /cvs/src/usr.sbin/relayd/Makefile,v
> retrieving revision 1.29
> diff -u -p -r1.29 Makefile
> --- Makefile  21 Nov 2015 12:37:42 -  1.29
> +++ Makefile  19 Jul 2016 08:33:26 -
> @@ -6,7 +6,7 @@ SRCS+=agentx.c ca.c carp.c check_icmp.
>   check_tcp.c config.c control.c hce.c log.c name2id.c \
>   pfe.c pfe_filter.c pfe_route.c proc.c \
>   relay.c relay_http.c relay_udp.c relayd.c \
> - shuffle.c snmp.c ssl.c util.c
> + shuffle.c snmp.c ssl.c tlsc.c util.c
>  MAN= relayd.8 relayd.conf.5
>  
>  LDADD=   -levent -lssl -lcrypto -lutil
> Index: ca.c
> ===
> RCS file: /cvs/src/usr.sbin/relayd/ca.c,v
> retrieving revision 1.16
> diff -u -p -r1.16 ca.c
> --- ca.c  5 Dec 2015 13:13:11 -   1.16
> +++ ca.c  19 Jul 2016 13:18:33 -
> @@ -23,6 +23,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  
>  #include 
> @@ -256,6 +257,7 @@ static int
>  rsae_send_imsg(int flen, const u_char *from, u_char *to, RSA *rsa,
>  int padding, u_int cmd)
>  {
> + struct pollfdpfd[1];
>   struct ctl_keyop cko;
>   int  ret = 0;
>   objid_t *id;
> @@ -292,9 +294,21 @@ rsae_send_imsg(int flen, const u_char *f
>* operation in OpenSSL's engine layer.
>*/
>   imsg_composev(ibuf, cmd, 0, 0, -1, iov, cnt);
> - imsg_flush(ibuf);
> + if (imsg_flush(ibuf) == -1)
> + log_warn("rsae_send_imsg: imsg_flush");
>  
> + pfd[0].fd = ibuf->fd;
> + pfd[0].events = POLLIN;
>   while (!done) {
> + switch (poll(pfd, 1, 5 * 1000)) {
> + case -1:
> + fatal("rsae_send_imsg: poll");
> + case 0:
> + log_warnx("rsae_send_imsg: poll timeout");
> + break;
> + default:
> + break;
> + }
>   if ((n = imsg_read(ibuf)) == -1 && errno != EAGAIN)
>   fatalx("imsg_read");
>   if (n == 0)
> Index: config.c
> ===
> RCS file: /cvs/src/usr.sbin/relayd/config.c,v
> retrieving revision 1.27
> diff -u -p -r1.27 config.c
> --- config.c  7 Dec 2015 04:03:27 -   1.27
> +++ config.c  18 Jul 2016 13:01:35 -
> @@ -51,6 +51,7 @@ config_init(struct relayd *env)
>   ps->ps_what[PROC_CA] = CONFIG_RELAYS;
>   ps->ps_what[PROC_RELAY] = CONFIG_RELAYS|
>   CONFIG_TABLES|CONFIG_PROTOS|CONFIG_CA_ENGINE;
> + ps->ps_what[PROC_TLSC] = 0;
>   }
>  
>   /* Other configuration */
> Index: relay.c
> ===
> RCS file: /cvs/src/usr.sbin/relayd/relay.c,v
> retrieving revision 1.206
> diff -u -p -r1.206 relay.c
> --- relay.c   30 Dec 2015 16:00:57 -  1.206
> +++ relay.c   19 Jul 2016 13:17:57 -
> @@ -28,6 +28,7 @@
>  #include 
>  
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -54,6 +55,8 @@ int  relay_dispatch_ca(int, struct priv
>   struct imsg *);
>  int   relay_dispatch_hce(int, struct privsep_proc *,
>   struct imsg *);
> +int   relay_dispatch_tlsc(int, struct privsep_proc *,
> + struct imsg *);
>  void  relay_shutdown(void);
>  
>  void  relay_protodebug(struct relay *);
> @@ -84,6 +87,13 @@ voidrelay_tls_connect(int, short, voi
>  void  relay_tls_connected(struct ctl_relay_event *);
>  void  

relayd TLS session caching

[Claudio Jeker]

At the moment relayd's TLS session caching is a bit busted because
the multiple relay processes do not share state.
The following diff adds SSL session caching and sharing of the TLS ticket
secrets. Which this openssl s_client -connect W.X.Y.Z:443 -reconnect
reuses the connection after the first one.
This should help TLS preformance since no exchange with the ca process is
needed if there is a cache hit.

The shared SSL session cache is in a new process (tlsc) but we still use
the internal cache as well (which is populated on cache miss).

The TLS token secret are generated on startup and a key rollover happens
every 4h with a rekey window of 10min. I see no need to add tunables for
this and also the shared SSL cache will just use the tls session cache
settings (because OpenSSL will do the garbage collection of the external
cache via the internal one).

This is the first version which seems to work for me when hammering relayd
with openssl s_client.
-- 
:wq Claudio


Index: Makefile
===
RCS file: /cvs/src/usr.sbin/relayd/Makefile,v
retrieving revision 1.29
diff -u -p -r1.29 Makefile
--- Makefile21 Nov 2015 12:37:42 -  1.29
+++ Makefile19 Jul 2016 08:33:26 -
@@ -6,7 +6,7 @@ SRCS+=  agentx.c ca.c carp.c check_icmp.
check_tcp.c config.c control.c hce.c log.c name2id.c \
pfe.c pfe_filter.c pfe_route.c proc.c \
relay.c relay_http.c relay_udp.c relayd.c \
-   shuffle.c snmp.c ssl.c util.c
+   shuffle.c snmp.c ssl.c tlsc.c util.c
 MAN=   relayd.8 relayd.conf.5
 
 LDADD= -levent -lssl -lcrypto -lutil
Index: ca.c
===
RCS file: /cvs/src/usr.sbin/relayd/ca.c,v
retrieving revision 1.16
diff -u -p -r1.16 ca.c
--- ca.c5 Dec 2015 13:13:11 -   1.16
+++ ca.c19 Jul 2016 13:18:33 -
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #include 
@@ -256,6 +257,7 @@ static int
 rsae_send_imsg(int flen, const u_char *from, u_char *to, RSA *rsa,
 int padding, u_int cmd)
 {
+   struct pollfdpfd[1];
struct ctl_keyop cko;
int  ret = 0;
objid_t *id;
@@ -292,9 +294,21 @@ rsae_send_imsg(int flen, const u_char *f
 * operation in OpenSSL's engine layer.
 */
imsg_composev(ibuf, cmd, 0, 0, -1, iov, cnt);
-   imsg_flush(ibuf);
+   if (imsg_flush(ibuf) == -1)
+   log_warn("rsae_send_imsg: imsg_flush");
 
+   pfd[0].fd = ibuf->fd;
+   pfd[0].events = POLLIN;
while (!done) {
+   switch (poll(pfd, 1, 5 * 1000)) {
+   case -1:
+   fatal("rsae_send_imsg: poll");
+   case 0:
+   log_warnx("rsae_send_imsg: poll timeout");
+   break;
+   default:
+   break;
+   }
if ((n = imsg_read(ibuf)) == -1 && errno != EAGAIN)
fatalx("imsg_read");
if (n == 0)
Index: config.c
===
RCS file: /cvs/src/usr.sbin/relayd/config.c,v
retrieving revision 1.27
diff -u -p -r1.27 config.c
--- config.c7 Dec 2015 04:03:27 -   1.27
+++ config.c18 Jul 2016 13:01:35 -
@@ -51,6 +51,7 @@ config_init(struct relayd *env)
ps->ps_what[PROC_CA] = CONFIG_RELAYS;
ps->ps_what[PROC_RELAY] = CONFIG_RELAYS|
CONFIG_TABLES|CONFIG_PROTOS|CONFIG_CA_ENGINE;
+   ps->ps_what[PROC_TLSC] = 0;
}
 
/* Other configuration */
Index: relay.c
===
RCS file: /cvs/src/usr.sbin/relayd/relay.c,v
retrieving revision 1.206
diff -u -p -r1.206 relay.c
--- relay.c 30 Dec 2015 16:00:57 -  1.206
+++ relay.c 19 Jul 2016 13:17:57 -
@@ -28,6 +28,7 @@
 #include 
 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -54,6 +55,8 @@ intrelay_dispatch_ca(int, struct priv
struct imsg *);
 int relay_dispatch_hce(int, struct privsep_proc *,
struct imsg *);
+int relay_dispatch_tlsc(int, struct privsep_proc *,
+   struct imsg *);
 voidrelay_shutdown(void);
 
 voidrelay_protodebug(struct relay *);
@@ -84,6 +87,13 @@ void  relay_tls_connect(int, short, voi
 voidrelay_tls_connected(struct ctl_relay_event *);
 voidrelay_tls_readcb(int, short, void *);
 voidrelay_tls_writecb(int, short, void *);
+int relay_tls_new_session(SSL *, SSL_SESSION *);
+SSL_SESSION*relay_tls_get_session(SSL *, unsigned char *, int, int *);
+voidrelay_tls_del_session(SSL_CTX *, SSL_SESSION *);
+
+struct tls_ticket_ctx  

Re: clarity recommendation for faq15.html

[ludovic coues]

faq4.html [1] already deal with the problem.

[1] http://www.openbsd.org/faq/faq4.html#AddThoughts

2016-07-19 3:29 GMT+02:00 Josh Grosse :
> I had a conversation with a new OpenBSD user who thought that he
> may have either misunderstood or been misled by the guidance to unpack
> the ports tree tarball in /usr. He quickly discovered his 2G /usr
> default allocation was too small when he began to build ports.
>
> FAQ 15 doesn't warn users who selected automatic disk allocation that
> the default partition size may be inadequate for all but minor ports building.
>
> This proposed recommendation adds two sentences which may clarify,
> following immediately after the untar example.
>
> Index: faq15.html
> ===
> RCS file: /cvs/www/faq/faq15.html,v
> retrieving revision 1.138
> diff -u -p -r1.138 faq15.html
> --- faq15.html  18 Jun 2016 02:24:32 -  1.138
> +++ faq15.html  19 Jul 2016 01:18:34 -
> @@ -763,6 +763,15 @@ For example:
>  # tar xzf /tmp/ports.tar.gz
>  
>
> +If you installed OpenBSD with automatic disk partition allocations,
> +/usr may be too small to contain the ports tree and
> +any active WRKOBJDIR build directories created in
> +/usr/ports/pobj by default.
> +A larger /usr, a separate /usr/ports partition, a
> +separate /usr/ports/pobj partition, or a non-standard
> +location for WRKOBJDIR may be considered.
> +
> +
>  If you'd like to be able to write to /usr/ports as a regular user,
>  see the next section.
>
>



-- 

Cordialement, Coues Ludovic
+336 148 743 42

Re: Fix begin of word bug in ed

[Theo de Raadt]

> If you feel bold, you can still put it in for 6.0, it's a bugfix
> after all, and i don't consider it very dangerous.  Then again,
> if you want to be extra cautious, nothing would be wrong with
> waiting until after unlock, either.

that's the right attitude.  Around at least every 4-5 hours for
the next month?  If so, commit.  If not, be patient and wait
for unlock :)