Re: ospfd: allow specifying area by number as well as id
yes :D > On 29 May 2019, at 15:05, Remi Locherer wrote: > > Hi David, > > are you going to commit this? > > Remi > > > On Thu, May 16, 2019 at 11:14:55PM +0200, Remi Locherer wrote: >> On Thu, May 16, 2019 at 09:39:37AM +0200, Sebastian Benoit wrote: >>> >>> >>> >>> Remi Locherer(remi.loche...@relo.ch) on 2019.05.15 23:15:03 +0200: On Tue, Apr 30, 2019 at 11:10:37PM +0200, Remi Locherer wrote: > On Mon, Apr 29, 2019 at 11:10:31AM +0100, Stuart Henderson wrote: >> On 2019/04/29 11:58, Sebastian Benoit wrote: >>> David Gwynne(da...@gwynne.id.au) on 2019.04.29 19:36:51 +1000: > On 29 Apr 2019, at 4:59 pm, Remi Locherer > wrote: > > Hi David > > On Mon, Apr 29, 2019 at 11:53:27AM +1000, David Gwynne wrote: >> it's always bothered me that i config areas on a crisco using a >> number, >> but then have to think hard to convert that number to an address for >> use >> in openbsd. eg, i was given area 700 in one place, which is 0.0.2.188 >> as an address. super annoying. >> >> so this changes the ospfd parser so it accepts both a number or >> address. >> i also changed it so it prints the number by default, which may be >> contentious. the manpage is slightly tweaked too. >> >> thoughts? > > I like it to be able to use a number instead of an address! > > It worked fine in my short test I performed. > > The output with the comment looks a bit strange to me. Are you sure it doesn't look... awesome? >>> >>> I like it! >> >> I don't really, but if we change this it needs to be displayed somehow >> and I don't have an idea to make it look nicer than this (cisco's method >> seems pretty horrible and wouldn't work for us anyway - looks like they >> remember which format was used to configure an area and use that as >> the output format...) >> > > Maybe it's better when we just allow both input formats but don't change > any output. Any opinions or comments on this? I think this would be a valuable addition to ospfd. >>> >>> Yes, and diff is ok benno@ >>> >> >> David: ok remi@ for your diff without the printconf part. >> >>> What about ospf6d? >> >> I'll handle that. >> >>> > > Below diff changes ospfctl to accept the address and number format for > "ospfct show database area XXX". > > > Index: parser.c > === > RCS file: /cvs/src/usr.sbin/ospfctl/parser.c,v > retrieving revision 1.20 > diff -u -p -r1.20 parser.c > --- parser.c 9 May 2011 12:25:35 - 1.20 > +++ parser.c 30 Apr 2019 20:28:18 - > @@ -39,7 +39,8 @@ enum token_type { > ADDRESS, > FLAG, > PREFIX, > - IFNAME > + IFNAME, > + AREA > }; > > struct token { > @@ -107,7 +108,7 @@ static const struct token t_show_db[] = > }; > > static const struct token t_show_area[] = { > - {ADDRESS, "", NONE, NULL}, > + {AREA, "", NONE, NULL}, > {ENDTOKEN, "", NONE, NULL} > }; > > @@ -218,6 +219,14 @@ match_token(const char *word, const stru > res->action = t->value; > } > break; > + case AREA: > + if (parse_area(word, >addr)) { > + match++; > + t = [i]; > + if (t->value) > + res->action = t->value; > + } > + break; > case PREFIX: > if (parse_prefix(word, >addr, >prefixlen)) { > match++; > @@ -274,6 +283,9 @@ show_valid_args(const struct token *tabl > case ADDRESS: > fprintf(stderr, " \n"); > break; > + case AREA: > + fprintf(stderr, " \n"); > + break; > case PREFIX: > fprintf(stderr, " [/]\n"); > break; > @@ -298,6 +310,32 @@ parse_addr(const char *word, struct in_a > bzero(, sizeof(ina)); > > if (inet_pton(AF_INET, word, )) { > + addr->s_addr = ina.s_addr; > + return (1); > + } > + > + return (0); > +} > + > +int > +parse_area(const char *word, struct in_addr *addr) > +{ > + struct in_addr ina; > + const char *errstr; > + > + if (word == NULL) > + return (0); > + > + bzero(addr, sizeof(struct in_addr));
hardware packet timestamping for mbufs, then sockets and bpf
nics are starting to offer the ability to timestamp packets when they're received. other systems (eg linux and freebsd) have support for recording that timestamp on mbufs and then using it as the backend for at least the SO_TIMESTAMP socket option instead of a call to microtime(). this implements the above, and additionally supports using the hw timestamp in bpf too. other systems may do the bpf thing too, but i didn't look closely enough to find out. timestamps are recorded as the uptime of the system in nanoseconds in the ph_timestamp field in mbufs. this mirrors the use of ph_timestamp in the fq_codel code to store the uptime in nanoseconds. im using another bit in m_pkthdr.csum_flags to say whether the timestamp is valid or not (M_TIMESTAMP). im arguing that it's another offloading feature and therefore appropriate for the csum offload flags field. this adds some inline functions to time.h for turning ns into a timeval and timespec, which are ns_to_microtime and ns_to_nanotime respectively. i originally wanted ns_to_timeval and ns_to_timespec, but the linux compat stuff in drm already uses those names and ruined the idea. especially since they return the time{val,spec}s as values. the ipv4 SO_TIMESTAMP and bpf code looks at whether M_TIMESTAMP is set, and if so turns ph_timestamp into a timeval before adding it to boottime (which is the wall clock time that uptime starts at), before using it instead of microtime(). the mcx changes are based on what freebsd did to their driver, but simplified a bit. i want this because we're being asked to look at recording network traffic for possible audit use. part of that is having accurate timestamps on received packets, and hopefully it will mitigate against chunks of packets getting reordered or delayed significantly when the box is busy. thoughts? ok? Index: sys/mbuf.h === RCS file: /cvs/src/sys/sys/mbuf.h,v retrieving revision 1.242 diff -u -p -r1.242 mbuf.h --- sys/mbuf.h 11 Feb 2019 00:25:33 - 1.242 +++ sys/mbuf.h 7 Jun 2019 03:27:41 - @@ -226,13 +226,14 @@ struct mbuf { #defineM_ICMP_CSUM_IN_OK 0x0400 /* ICMP/ICMPv6 checksum verified */ #defineM_ICMP_CSUM_IN_BAD 0x0800 /* ICMP/ICMPv6 checksum bad */ #defineM_IPV6_DF_OUT 0x1000 /* don't fragment outgoing IPv6 */ +#defineM_TIMESTAMP 0x2000 /* ph_timestamp is set */ #ifdef _KERNEL #define MCS_BITS \ ("\20\1IPV4_CSUM_OUT\2TCP_CSUM_OUT\3UDP_CSUM_OUT\4IPV4_CSUM_IN_OK" \ "\5IPV4_CSUM_IN_BAD\6TCP_CSUM_IN_OK\7TCP_CSUM_IN_BAD\10UDP_CSUM_IN_OK" \ "\11UDP_CSUM_IN_BAD\12ICMP_CSUM_OUT\13ICMP_CSUM_IN_OK\14ICMP_CSUM_IN_BAD" \ -"\15IPV6_NODF_OUT") +"\15IPV6_NODF_OUT" "\16TIMESTAMP") #endif /* mbuf types */ Index: sys/time.h === RCS file: /cvs/src/sys/sys/time.h,v retrieving revision 1.41 diff -u -p -r1.41 time.h --- sys/time.h 3 Jun 2019 01:27:30 - 1.41 +++ sys/time.h 7 Jun 2019 03:27:41 - @@ -333,6 +333,20 @@ void clock_secs_to_ymdhms(time_t, struct /* Traditional POSIX base year */ #define POSIX_BASE_YEAR 1970 +static __inline void +ns_to_microtime(struct timeval *tv, uint64_t ns) +{ + tv->tv_sec = ns / 10L; + tv->tv_usec = (ns % 10L) / 1000; +} + +static __inline void +ns_to_nanotime(struct timespec *tv, uint64_t ns) +{ + tv->tv_sec = ns / 10L; + tv->tv_nsec = ns % 10L; +} + #else /* !_KERNEL */ #include Index: net/bpf.c === RCS file: /cvs/src/sys/net/bpf.c,v retrieving revision 1.175 diff -u -p -r1.175 bpf.c --- net/bpf.c 18 May 2019 12:59:32 - 1.175 +++ net/bpf.c 7 Jun 2019 03:27:41 - @@ -1284,13 +1284,25 @@ _bpf_mtap(caddr_t arg, const struct mbuf fcode = bps->bps_bf.bf_insns; slen = bpf_mfilter(fcode, m, pktlen); - if (slen == 0) + if (slen == 0) continue; if (d->bd_fildrop != BPF_FILDROP_PASS) drop = 1; if (d->bd_fildrop != BPF_FILDROP_DROP) { - if (!gottime++) - microtime(); + if (!gottime) { + if (ISSET(m->m_flags, M_PKTHDR) && + ISSET(m->m_pkthdr.csum_flags, +M_TIMESTAMP)) { + struct timeval btv; + ns_to_microtime(, + m->m_pkthdr.ph_timestamp); + microboottime(); + timeradd(, , ); + } else + microtime(); + +
Re: 64-bit dma for drm(4) on amd64
On Thu, Jun 06, 2019 at 11:41:07PM +0200, Mark Kettenis wrote: > As a result of a recent discussion with jsg@, I realized that the > graphics drivers are (mostly) allocating memory from the dma region. > Since the the graphics stack can potentially gobble up large amounts > of memory, this means we can run out of dma memory which makes other > parts of our kernel quite unhappy. Most of the supported hardware > actually supports 64-bit DMA just fine, and the drivers already have > code to handle the exceptions. The diff below makes use of this > knowledge to (hopefully) safely allocate from "high" memory when > possible. One big change is that this makes bus_dma(9) 64-bit DMA > aware in the sense that if the BUS_DMA_64BIT flag is used, we skip the > "not dma-reachable" panic. > > It seems to work fine on my Intel Broadwell laptop. I haven't tested > this on radeon(4) yet. So further testing, especially on systems with > 4GB of memory or more is necessary. > > Please test. One of the ways dma32 is set in radeon/amdgpu is along the lines of dma_bits = rdev->need_dma32 ? 32 : 40; r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits)); if (r) { rdev->need_dma32 = true; dma_bits = 32; which we don't handle. That is for the case where the card supports 64 bit dma but the system doesn't? Not something we should be concerned about? Having __GFP_DMA32 set the sign bit on the flags seems like it is asking for trouble. How about 0x4000 / bit 30 instead of 0x8000 / bit 31. Otherwise looks good. > > > Index: arch/amd64/amd64/bus_dma.c > === > RCS file: /cvs/src/sys/arch/amd64/amd64/bus_dma.c,v > retrieving revision 1.50 > diff -u -p -r1.50 bus_dma.c > --- arch/amd64/amd64/bus_dma.c14 Oct 2017 04:44:43 - 1.50 > +++ arch/amd64/amd64/bus_dma.c6 Jun 2019 21:19:21 - > @@ -319,7 +319,8 @@ _bus_dmamap_load_raw(bus_dma_tag_t t, bu > if (plen < sgsize) > sgsize = plen; > > - if (paddr > dma_constraint.ucr_high) > + if (paddr > dma_constraint.ucr_high && > + (map->_dm_flags & BUS_DMA_64BIT) == 0) > panic("Non dma-reachable buffer at paddr > %#lx(raw)", > paddr); > > @@ -583,7 +584,8 @@ _bus_dmamap_load_buffer(bus_dma_tag_t t, >*/ > pmap_extract(pmap, vaddr, (paddr_t *)); > > - if (curaddr > dma_constraint.ucr_high) > + if (curaddr > dma_constraint.ucr_high && > + (map->_dm_flags & BUS_DMA_64BIT) == 0) > panic("Non dma-reachable buffer at curaddr %#lx(raw)", > curaddr); > > Index: dev/pci/drm/drm_linux.c > === > RCS file: /cvs/src/sys/dev/pci/drm/drm_linux.c,v > retrieving revision 1.37 > diff -u -p -r1.37 drm_linux.c > --- dev/pci/drm/drm_linux.c 4 Jun 2019 12:08:22 - 1.37 > +++ dev/pci/drm/drm_linux.c 6 Jun 2019 21:19:21 - > @@ -293,16 +293,19 @@ struct vm_page * > alloc_pages(unsigned int gfp_mask, unsigned int order) > { > int flags = (gfp_mask & M_NOWAIT) ? UVM_PLA_NOWAIT : UVM_PLA_WAITOK; > + struct uvm_constraint_range *constraint = _constraint; > struct pglist mlist; > > if (gfp_mask & M_CANFAIL) > flags |= UVM_PLA_FAILOK; > if (gfp_mask & M_ZERO) > flags |= UVM_PLA_ZERO; > + if (gfp_mask & __GFP_DMA32) > + constraint = _constraint; > > TAILQ_INIT(); > - if (uvm_pglistalloc(PAGE_SIZE << order, dma_constraint.ucr_low, > - dma_constraint.ucr_high, PAGE_SIZE, 0, , 1, flags)) > + if (uvm_pglistalloc(PAGE_SIZE << order, constraint->ucr_low, > + constraint->ucr_high, PAGE_SIZE, 0, , 1, flags)) > return NULL; > return TAILQ_FIRST(); > } > Index: dev/pci/drm/include/linux/gfp.h > === > RCS file: /cvs/src/sys/dev/pci/drm/include/linux/gfp.h,v > retrieving revision 1.1 > diff -u -p -r1.1 gfp.h > --- dev/pci/drm/include/linux/gfp.h 14 Apr 2019 10:14:53 - 1.1 > +++ dev/pci/drm/include/linux/gfp.h 6 Jun 2019 21:19:21 - > @@ -7,24 +7,25 @@ > #include > #include > > -#define GFP_ATOMIC M_NOWAIT > -#define GFP_NOWAIT M_NOWAIT > -#define GFP_KERNEL (M_WAITOK | M_CANFAIL) > -#define GFP_USER (M_WAITOK | M_CANFAIL) > -#define GFP_TEMPORARY(M_WAITOK | M_CANFAIL) > -#define GFP_HIGHUSER 0 > -#define GFP_DMA320 > -#define __GFP_NOWARN 0 > -#define __GFP_NORETRY0 > -#define __GFP_ZERO M_ZERO > +#define __GFP_ZERO M_ZERO > +#define __GFP_DMA32 0x8000 > +#define __GFP_NOWARN 0 > +#define __GFP_NORETRY0 > #define __GFP_RETRY_MAYFAIL 0 >
Re: 64-bit dma for drm(4) on amd64
On Thu, Jun 06, 2019 at 11:41:07PM +0200, Mark Kettenis wrote: [...] > It seems to work fine on my Intel Broadwell laptop. I haven't tested > this on radeon(4) yet. So further testing, especially on systems with > 4GB of memory or more is necessary. > > Please test. Tested on a desktop with 8GB RAM and amdgpu enabled (GPU is a Vega 64, but that may not be relevant). I'm attaching dmesg below. Tested for about 2 hours with some Chrome, some Firefox, mono... No regression noticed. The situations where the display freezes were encountered as before building the kernel with this patch, that is with Chrome after a while with Google Maps (one of the most reliable triggers). Also with `piglit run all results/all` after 39 tests (dmafence on wait channel again). If there's more that would help testing or reporting, I'm happy to. > > > Index: arch/amd64/amd64/bus_dma.c > === > RCS file: /cvs/src/sys/arch/amd64/amd64/bus_dma.c,v > retrieving revision 1.50 > diff -u -p -r1.50 bus_dma.c > --- arch/amd64/amd64/bus_dma.c14 Oct 2017 04:44:43 - 1.50 > +++ arch/amd64/amd64/bus_dma.c6 Jun 2019 21:19:21 - > @@ -319,7 +319,8 @@ _bus_dmamap_load_raw(bus_dma_tag_t t, bu > if (plen < sgsize) > sgsize = plen; > > - if (paddr > dma_constraint.ucr_high) > + if (paddr > dma_constraint.ucr_high && > + (map->_dm_flags & BUS_DMA_64BIT) == 0) > panic("Non dma-reachable buffer at paddr > %#lx(raw)", > paddr); > > @@ -583,7 +584,8 @@ _bus_dmamap_load_buffer(bus_dma_tag_t t, >*/ > pmap_extract(pmap, vaddr, (paddr_t *)); > > - if (curaddr > dma_constraint.ucr_high) > + if (curaddr > dma_constraint.ucr_high && > + (map->_dm_flags & BUS_DMA_64BIT) == 0) > panic("Non dma-reachable buffer at curaddr %#lx(raw)", > curaddr); > > Index: dev/pci/drm/drm_linux.c > === > RCS file: /cvs/src/sys/dev/pci/drm/drm_linux.c,v > retrieving revision 1.37 > diff -u -p -r1.37 drm_linux.c > --- dev/pci/drm/drm_linux.c 4 Jun 2019 12:08:22 - 1.37 > +++ dev/pci/drm/drm_linux.c 6 Jun 2019 21:19:21 - > @@ -293,16 +293,19 @@ struct vm_page * > alloc_pages(unsigned int gfp_mask, unsigned int order) > { > int flags = (gfp_mask & M_NOWAIT) ? UVM_PLA_NOWAIT : UVM_PLA_WAITOK; > + struct uvm_constraint_range *constraint = _constraint; > struct pglist mlist; > > if (gfp_mask & M_CANFAIL) > flags |= UVM_PLA_FAILOK; > if (gfp_mask & M_ZERO) > flags |= UVM_PLA_ZERO; > + if (gfp_mask & __GFP_DMA32) > + constraint = _constraint; > > TAILQ_INIT(); > - if (uvm_pglistalloc(PAGE_SIZE << order, dma_constraint.ucr_low, > - dma_constraint.ucr_high, PAGE_SIZE, 0, , 1, flags)) > + if (uvm_pglistalloc(PAGE_SIZE << order, constraint->ucr_low, > + constraint->ucr_high, PAGE_SIZE, 0, , 1, flags)) > return NULL; > return TAILQ_FIRST(); > } > Index: dev/pci/drm/include/linux/gfp.h > === > RCS file: /cvs/src/sys/dev/pci/drm/include/linux/gfp.h,v > retrieving revision 1.1 > diff -u -p -r1.1 gfp.h > --- dev/pci/drm/include/linux/gfp.h 14 Apr 2019 10:14:53 - 1.1 > +++ dev/pci/drm/include/linux/gfp.h 6 Jun 2019 21:19:21 - > @@ -7,24 +7,25 @@ > #include > #include > > -#define GFP_ATOMIC M_NOWAIT > -#define GFP_NOWAIT M_NOWAIT > -#define GFP_KERNEL (M_WAITOK | M_CANFAIL) > -#define GFP_USER (M_WAITOK | M_CANFAIL) > -#define GFP_TEMPORARY(M_WAITOK | M_CANFAIL) > -#define GFP_HIGHUSER 0 > -#define GFP_DMA320 > -#define __GFP_NOWARN 0 > -#define __GFP_NORETRY0 > -#define __GFP_ZERO M_ZERO > +#define __GFP_ZERO M_ZERO > +#define __GFP_DMA32 0x8000 > +#define __GFP_NOWARN 0 > +#define __GFP_NORETRY0 > #define __GFP_RETRY_MAYFAIL 0 > #define __GFP_MOVABLE0 > #define __GFP_COMP 0 > -#define GFP_TRANSHUGE_LIGHT 0 > #define __GFP_KSWAPD_RECLAIM 0 > #define __GFP_HIGHMEM0 > #define __GFP_RECLAIMABLE0 > -#define __GFP_DMA32 0 > + > +#define GFP_ATOMIC M_NOWAIT > +#define GFP_NOWAIT M_NOWAIT > +#define GFP_KERNEL (M_WAITOK | M_CANFAIL) > +#define GFP_USER (M_WAITOK | M_CANFAIL) > +#define GFP_TEMPORARY(M_WAITOK | M_CANFAIL) > +#define GFP_HIGHUSER 0 > +#define GFP_DMA32__GFP_DMA32 > +#define GFP_TRANSHUGE_LIGHT 0 > > static inline bool > gfpflags_allow_blocking(const unsigned int flags) > Index:
64-bit dma for drm(4) on amd64
As a result of a recent discussion with jsg@, I realized that the graphics drivers are (mostly) allocating memory from the dma region. Since the the graphics stack can potentially gobble up large amounts of memory, this means we can run out of dma memory which makes other parts of our kernel quite unhappy. Most of the supported hardware actually supports 64-bit DMA just fine, and the drivers already have code to handle the exceptions. The diff below makes use of this knowledge to (hopefully) safely allocate from "high" memory when possible. One big change is that this makes bus_dma(9) 64-bit DMA aware in the sense that if the BUS_DMA_64BIT flag is used, we skip the "not dma-reachable" panic. It seems to work fine on my Intel Broadwell laptop. I haven't tested this on radeon(4) yet. So further testing, especially on systems with 4GB of memory or more is necessary. Please test. Index: arch/amd64/amd64/bus_dma.c === RCS file: /cvs/src/sys/arch/amd64/amd64/bus_dma.c,v retrieving revision 1.50 diff -u -p -r1.50 bus_dma.c --- arch/amd64/amd64/bus_dma.c 14 Oct 2017 04:44:43 - 1.50 +++ arch/amd64/amd64/bus_dma.c 6 Jun 2019 21:19:21 - @@ -319,7 +319,8 @@ _bus_dmamap_load_raw(bus_dma_tag_t t, bu if (plen < sgsize) sgsize = plen; - if (paddr > dma_constraint.ucr_high) + if (paddr > dma_constraint.ucr_high && + (map->_dm_flags & BUS_DMA_64BIT) == 0) panic("Non dma-reachable buffer at paddr %#lx(raw)", paddr); @@ -583,7 +584,8 @@ _bus_dmamap_load_buffer(bus_dma_tag_t t, */ pmap_extract(pmap, vaddr, (paddr_t *)); - if (curaddr > dma_constraint.ucr_high) + if (curaddr > dma_constraint.ucr_high && + (map->_dm_flags & BUS_DMA_64BIT) == 0) panic("Non dma-reachable buffer at curaddr %#lx(raw)", curaddr); Index: dev/pci/drm/drm_linux.c === RCS file: /cvs/src/sys/dev/pci/drm/drm_linux.c,v retrieving revision 1.37 diff -u -p -r1.37 drm_linux.c --- dev/pci/drm/drm_linux.c 4 Jun 2019 12:08:22 - 1.37 +++ dev/pci/drm/drm_linux.c 6 Jun 2019 21:19:21 - @@ -293,16 +293,19 @@ struct vm_page * alloc_pages(unsigned int gfp_mask, unsigned int order) { int flags = (gfp_mask & M_NOWAIT) ? UVM_PLA_NOWAIT : UVM_PLA_WAITOK; + struct uvm_constraint_range *constraint = _constraint; struct pglist mlist; if (gfp_mask & M_CANFAIL) flags |= UVM_PLA_FAILOK; if (gfp_mask & M_ZERO) flags |= UVM_PLA_ZERO; + if (gfp_mask & __GFP_DMA32) + constraint = _constraint; TAILQ_INIT(); - if (uvm_pglistalloc(PAGE_SIZE << order, dma_constraint.ucr_low, - dma_constraint.ucr_high, PAGE_SIZE, 0, , 1, flags)) + if (uvm_pglistalloc(PAGE_SIZE << order, constraint->ucr_low, + constraint->ucr_high, PAGE_SIZE, 0, , 1, flags)) return NULL; return TAILQ_FIRST(); } Index: dev/pci/drm/include/linux/gfp.h === RCS file: /cvs/src/sys/dev/pci/drm/include/linux/gfp.h,v retrieving revision 1.1 diff -u -p -r1.1 gfp.h --- dev/pci/drm/include/linux/gfp.h 14 Apr 2019 10:14:53 - 1.1 +++ dev/pci/drm/include/linux/gfp.h 6 Jun 2019 21:19:21 - @@ -7,24 +7,25 @@ #include #include -#define GFP_ATOMIC M_NOWAIT -#define GFP_NOWAIT M_NOWAIT -#define GFP_KERNEL (M_WAITOK | M_CANFAIL) -#define GFP_USER (M_WAITOK | M_CANFAIL) -#define GFP_TEMPORARY (M_WAITOK | M_CANFAIL) -#define GFP_HIGHUSER 0 -#define GFP_DMA32 0 -#define __GFP_NOWARN 0 -#define __GFP_NORETRY 0 -#define __GFP_ZERO M_ZERO +#define __GFP_ZERO M_ZERO +#define __GFP_DMA320x8000 +#define __GFP_NOWARN 0 +#define __GFP_NORETRY 0 #define __GFP_RETRY_MAYFAIL0 #define __GFP_MOVABLE 0 #define __GFP_COMP 0 -#define GFP_TRANSHUGE_LIGHT0 #define __GFP_KSWAPD_RECLAIM 0 #define __GFP_HIGHMEM 0 #define __GFP_RECLAIMABLE 0 -#define __GFP_DMA320 + +#define GFP_ATOMIC M_NOWAIT +#define GFP_NOWAIT M_NOWAIT +#define GFP_KERNEL (M_WAITOK | M_CANFAIL) +#define GFP_USER (M_WAITOK | M_CANFAIL) +#define GFP_TEMPORARY (M_WAITOK | M_CANFAIL) +#define GFP_HIGHUSER 0 +#define GFP_DMA32 __GFP_DMA32 +#define GFP_TRANSHUGE_LIGHT0 static inline bool gfpflags_allow_blocking(const unsigned int flags) Index: dev/pci/drm/ttm/ttm_tt.c
Re: monotonic clock going backwards?
> From: David Hill > Date: Thu, 6 Jun 2019 09:55:00 -0400 > > On 6/6/19 8:49 AM, Mark Kettenis wrote: > >> From: David Hill > >> Date: Thu, 6 Jun 2019 08:22:06 -0400 > >> > >> On 6/6/19 6:56 AM, Scott Cheloha wrote: > >>> On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote: > Hello - > > I noticed some negative roundtrip times when running traceroute, which > uses > the monotonic clock to calculate the RTT. > > If I run the following code, it eventually bombs.?? It bombs quick if I > launch Firefox. > > timespeccmp failed > tp1 s:103780 n:63101042 > tp2 s:103779 n:761117849 > >>> please share your dmesg and the output of > >>> > >>> sysctl kern.clockrate kern.timecounter > >>> > >>> Could you also share the output when you run the following code? > >>> With and without firefox running? Curious about how frequently > >>> this happens. > >>> > >>> #include > >>> > >>> #include > >>> #include > >>> #include > >>> > >>> int > >>> main(void) > >>> { > >>> struct timespec t0, t1, timeout; > >>> > >>> timeout.tv_sec = 0; > >>> timeout.tv_nsec = 10; > >>> > >>> for (;;) { > >>> clock_gettime(CLOCK_MONOTONIC, ); > >>> if (nanosleep(, NULL) == -1) > >>> err(1, "nanosleep"); > >>> clock_gettime(CLOCK_MONOTONIC, ); > >>> if (timespeccmp(, , >)) { > >>> printf("%lld.%09ld precedes %lld.%09ld\n", > >>> t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec); > >>> } > >>> } > >>> > >>> return 0; > >>> } > >>> > >> $ sysctl kern.clockrate kern.timecounter > >> kern.clockrate=tick = 1, tickadj = 40, hz = 100, profhz = 100, > >> stathz = 100 > >> kern.timecounter.tick=1 > >> kern.timecounter.timestepwarnings=0 > >> kern.timecounter.hardware=tsc > >> kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000) acpitimer0(1000) > >> > >> 146259.148543164 precedes 146259.450584680 > >> 146259.488592813 precedes 146259.790556528 > >> 146259.808566236 precedes 146260.110528202 > >> 146261.148476141 precedes 146261.450398822 > >> 146262.888279505 precedes 146263.190240715 > >> 146263.887431808 precedes 146264.189435801 > >> 146274.487159451 precedes 146274.789148187 > >> 146278.186820816 precedes 146278.488803756 > >> 146279.546680181 precedes 146279.848674138 > >> 146284.326305557 precedes 146284.628224696 > >> 146285.426154861 precedes 146285.728066901 > >> 146287.885921929 precedes 146288.187826688 > >> 146289.985763711 precedes 146290.287731233 > >> 146290.985659041 precedes 146291.287640302 > >> 146293.985348615 precedes 146294.287327519 > >> 146295.465186742 precedes 146295.767218516 > >> 146295.885156426 precedes 146296.187140717 > >> 146298.544907956 precedes 146298.846882759 > >> 146298.644894308 precedes 146298.946889142 > > Does this also happen with > > > > kern.timecounter.hardware=acpihpet > > > > ? > > Ran the code with kern.timecounter.hardware=acpihpet0 for 10 minutes > without any output. > > With the code still running, I switched to tsc and starting seeing output: > > 151944.345840447 precedes 151944.647753895 > 151944.525774690 precedes 151944.827681380 > 151946.405553296 precedes 151946.707533012 > > The output stops as soon as I switch back to acpihpet0. > > I will add that to my /etc/sysctl.conf. > > Thank you. So the TSCs on the cores of your brand new Intel Kaby Lake R processor are out of sync. That is the final drop. I disabled the code that promotes the TSC as a high-quality timecounter on MULTIPROCESSOR kernels. It simply doesn't work on a significant fraction of machines and causes all kinds of weird problems. Scott, any change to turn that bit of code into a regress test?
Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority
On 02/06/19(Sun) 16:41, Martin Pieuchot wrote: > On 01/06/19(Sat) 18:55, Martin Pieuchot wrote: > > Diff below exists mainly for documentation and test purposes. If > > you're not interested about how to break the scheduler internals in > > pieces, don't read further and go straight to testing! > > > > - First change is to stop calling tsleep(9) at PUSER. That makes > > it clear that all "sleeping priorities" are smaller than PUSER. > > That's important to understand for the diff below. `p_priority' > > is currently a placeholder for the "sleeping priority" and the > > "runnqueue priority". Both fields are separated by this diff. > > > > - When a thread goes to sleep, the priority argument of tsleep(9) is > > now recorded in `p_slpprio'. This argument can be considered as part > > of the sleep queue. Its purpose is to place the thread into a higher > > runqueue when awoken. > > > > - Currently, for stopped threads, `p_priority' correspond to `p_usrpri'. > > So setrunnable() has been untangled to place SSTOP and SSLEEP threads > > in the preferred queue without having to use `p_priority'. Note that > > `p_usrpri' is still recalculated *after* having called setrunqueue(). > > This is currently fine because setrunnable() is called with SCHED_LOCK() > > but it will be racy when we'll split it. > > > > - A new field, `p_runprio' has been introduced. It should be considered > > as part of the per-CPU runqueues. It indicates where a current thread > > is placed. > > > > - `spc_curpriority' is now updated at every context-switch. That means > >need_resched() won't be called after comparing an out-of-date value. > >At the same time, `p_usrpri' is initialized to the highest possible > >value for idle threads. > > > > - resched_proc() was calling need_resched() in the following conditions: > >- If the SONPROC thread has a higher priority that the current > > running thread (itself). > >- Twice in setrunnable() when we know that p_priority <= p_usrpri. > >- If schedcpu() considered that a thread, after updating its prio, > > should preempt the one running on the CPU pointed by `p_cpu'. > > > > The diff below simplify all of that by calling need_resched() when: > >- A thread is inserted in a CPU runqueue at a higher priority than > > the one SONPROC. > >- schedcpu() decides that a thread in SRUN state should preempt the > > one SONPROC. > > > > - `p_estcpu' `p_usrpri' and `p_slptime' which represent the "priority" > > of a thread are now updated while holding a per-thread mutex. As a > > result schedclock() and donice() no longer takes the SCHED_LOCK(), > > and schedcpu() almost never take it. > > > > - With this diff top(1) and ps(1) will report the "real" `p_usrpi' value > > when displaying priorities. This is helpful to understand what's > > happening: > > > > load averages: 0.99, 0.56, 0.25 two.lab.grenadille.net > > 23:42:10 > > 70 threads: 68 idle, 2 on processorup > > 0:09 > > CPU0: 0.0% user, 0.0% nice, 51.0% sys, 2.0% spin, 0.0% intr, 47.1% idle > > CPU1: 2.0% user, 0.0% nice, 51.0% sys, 3.9% spin, 0.0% intr, 43.1% idle > > Memory: Real: 47M/1005M act/tot Free: 2937M Cache: 812M Swap: 0K/4323M > > > > PID TID PRI NICE SIZE RES STATE WAIT TIMECPU COMMAND > > 81000 145101 7200K 1664K sleep/1 bored 1:15 36.96% softnet > > 47133 244097 730 2984K 4408K sleep/1 netio 1:06 35.06% cvs > > 64749 522184 660 176K 148K onproc/1 - 0:55 28.81% nfsd > > 21615 602473 12700K 1664K sleep/0 - 7:22 0.00% idle0 > > 12413 606242 12700K 1664K sleep/1 - 7:08 0.00% idle1 > > 85778 338258 500 4936K 7308K idle select0:10 0.00% ssh > > 22771 575513 500 176K 148K sleep/0 nfsd 0:02 0.00% nfsd > > > > > > > > - The removal of `p_priority' and the change that makes mi_switch() > > always update `spc_curpriority' might introduce some changes in > > behavior, especially with kernel threads that were not going through > > tsleep(9). We currently have some situations where the priority of > > the running thread isn't correctly reflected. This diff changes that > > which means we should be able to better understand where the problems > > are. > > > > I'd be interested in comments/tests/reviews before continuing in this > > direction. Note that at least part of this diff are required to split > > the accounting apart from the SCHED_LOCK() as well. > > > > I'll also work on exporting scheduler statistics unless somebody wants > > to beat me :) > > Updated diff to use IPL_SCHED and rebased to apply on top of -current :) Updated diff that fixes a pagefault reported by sthen@. Index: arch/amd64/amd64/genassym.cf === RCS file:
Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority
On Thu, Jun 06, 2019 at 02:55:35PM +0100, Stuart Henderson wrote: > I'm testing the "pump my sched" and read/write unlock diffs and ran into > the panic below. Seems more likely that it would be connected with the > sched diff rather than anything else. I'll build a WITNESS kernel and > see if I can get more details if it hapens again. > Funny thing, I ran into the same panic last night after running this diff without issue for several days. Mine didn't include a stack trace though. > ddb{1}> show panic > kernel diagnostic assertion "__mp_lock_held(_lock, curcpu()) == 0" > failed > : file "/sys/kern/kern_lock.c", line 63 > ddb{1}> tr > db_enter() at db_enter+0x10 > panic() at panic+0x128 > __assert(81aebffd,81ac7a49,3f,81b0e315) at > __assert+0x2 > e > _kernel_lock() at _kernel_lock+0xf6 > pageflttrap() at pageflttrap+0x75 > kerntrap(800033ee0ca0) at kerntrap+0x91 > alltraps_kern(6,380,fffb,2,80002201aff0,e0) at alltraps_kern+0x7b > setrunqueue(0,8000342b2ce8,e0) at setrunqueue+0xbc > setrunnable(8000342b2ce8,e0) at setrunnable+0xa5 > ptsignal(8000342b2ce8,13,0) at ptsignal+0x3c4 > sys_kill(800033c2fcc0,800033ee0ec0,800033ee0f20) at sys_kill+0x1c5 > syscall(800033ee0f90) at syscall+0x389 > Xsyscall(6,7a,0,7a,13,7f7e3e18) at Xsyscall+0x128 > end of kernel > end trace frame: 0x7f7e3cb0, count: -13 > ddb{1}> ps >PID TID PPIDUID S FLAGS WAIT COMMAND > 89165 411452 1971667 30x1000b2 poll ping > 1971 11232 86869667 30x82 piperdcheck_ping > 66051 380473 92153667 30x1000b2 poll ping > 92153 480496 86869667 30x82 piperdcheck_ping > 7608 110605 8332667 30x1000b2 poll ping > 8332 284560 86869667 30x82 piperdcheck_ping > 40097 336365 35317667 30x1000b2 poll ping > 35317 385677 86869667 30x82 piperdcheck_ping > 33714 102547 21689667 30x1000b2 poll ping > 21689 210571 86869667 30x82 piperdcheck_ping > 72142 347023 62694667 30x1000b2 poll ping > 62694 439660 86869667 30x82 piperdcheck_ping > 87347 447624 76730 1000 4 0x8000430xscreensaver > 37569 319522 76730 1000 20x800402piecewise > 11841 481452 18848667 30x1000b2 poll ping > 18848 111230 86869667 30x82 piperdcheck_ping > 69292 189646 44902667 30x1000b2 poll ping > 44902 21064 86869667 30x82 piperdcheck_ping > 67905 363250 1761507 30x92 kqreadpickup > 86709 146945 91951 1000 30x92 kqreadimap > 36596 227858 91951666 30x92 kqreadimap-login > 90723 180281 91951 1000 30x92 kqreadimap > 42867 129499 91951 1000 30x92 kqreadimap > 52963 203919 91951 1000 30x92 kqreadimap > 61599 167915 91951666 30x92 kqreadimap-login > 54450 154426 91951666 30x92 kqreadimap-login > 54061 103553 91951666 30x92 kqreadimap-login > 39422 362611 85872715 30x90 netconperl > 49809 260623 85872715 30x90 lockf perl > 3824 92473 85872715 30x90 lockf perl > 30990 398152 85872715 30x90 lockf perl > 3327 264795 85872715 30x90 lockf perl > 24406 217143 85872715 30x90 lockf perl > 52871 404585 85872715 30x90 lockf perl > 58220 118881 85872715 30x90 lockf perl > 32898 307331 85872715 30x90 lockf perl > 50203 396987 85872715 30x90 lockf perl > 93024 24785 1 1000 30x100080 selectssh > 22261 482889 52840 1000 30x100083 poll ssh > 52840 404967 61470 1000 30x10008b pause ksh > 35796 132597 61470 1000 30xb0 netio urxvt > 61470 485028 72107 1000 30xb2 selecturxvt > 72107 196192 1 1000 30x10008a pause sh > 70238 396933 1 1000 30x100080 selectssh > 41840 302658 10362 1000 30x100083 poll ssh > 89638 411799 91951 1000 30x92 kqreadimap > 49250 507579 81821 1000 30x100083 ttyin ksh > 37608 523403 91951666 30x92 kqreadimap-login > 75552 67606 33096 1000 30x100083 poll mutt > 33096 492099 81821 1000 30x10008b pause ksh > 81821 270355 1 1000 30x100080 kqreadtmux > 24522 341281 91861 1000 30x100083 kqreadtmux >
Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority
On 2019/06/06 14:55, Stuart Henderson wrote: > I'm testing the "pump my sched" and read/write unlock diffs and ran into > the panic below. Seems more likely that it would be connected with the > sched diff rather than anything else. I'll build a WITNESS kernel and > see if I can get more details if it hapens again. Easily repeatable: Run xscreensaver. Let it lock. Try to unlock. login: panic: kernel diagnostic assertion "__mp_lock_held(_lock, curcpu()) == 0" failed: file "/src/cvs-openbsd/sys/kern/kern_lock.c", line 63 Stopped at db_enter+0x10: popq%rbp TIDPIDUID PRFLAGS PFLAGS CPU COMMAND 6043 83667 1000 0x2 01 firefox 320664 83667 1000 0x2 0x4802 firefox 523640 70812 1000 0x2 0x4003 firefox *307581 55620 10000x32 00K xscreensaver db_enter() at db_enter+0x10 panic() at panic+0x128 __assert(81af6d92,81afc585,3f,81b1a406) at __assert+0x2 e __mp_lock_held(811362e0,800033cfe5e0) at __mp_lock_held pageflttrap() at pageflttrap+0x78 kerntrap(800033cfe6b0) at kerntrap+0x91 alltraps_kern(6,300,fffb,2,800022009ff0,c0) at alltraps_kern+0x7b setrunqueue(0,8000342572a8,c0) at setrunqueue+0xbc setrunnable(8000342572a8,c0) at setrunnable+0xa5 ptsignal(8000342572a8,13,0) at ptsignal+0x3c4 sys_kill(800033c532a8,800033cfe8d0,800033cfe930) at sys_kill+0x1c5 syscall(800033cfe9a0) at syscall+0x399 Xsyscall(6,7a,0,7a,13,7f7e8cc8) at Xsyscall+0x128 end of kernel end trace frame: 0x7f7e8b60, count: 2 https://www.openbsd.org/ddb.html describes the minimum info required in bug reports. Insufficient info makes it difficult to find and fix bugs. ddb{0}> sh witness Sleep locks: _fpriv->lock (type: rwlock, depth: 0)_priv->fbs_lock (type: rwlock, d epth: 0)futex (type: rwlock, depth: 0) _lock (type: kernel_lock, depth: 6 ) >lock (type: rwlock, depth: 5) _lock (type: kernel_lock, depth: 6 ) -- (already displayed) >pr_lock (type: rwlock, depth: 1) _lock (type: kernel_lock, depth: 6) -- (already displayed) swplk (type: rwlock, depth: 0) >lock (type: rwlock, depth: 5) -- (already d isplayed) >i_lock (type: rrwlock, depth: 2) _lock (type: kernel_lock, depth: 6) -- (already displayed) >lock (type: rwlock, depth: 5) -- (already displayed) >dk_lock (type: rwlock, depth: 3) _lock (type: kernel_lock, d epth: 6) -- (already displayed) _mtx (type: rwlock, depth: 3) >dh_mtx (type: rwlock, depth: 4 ) lockflk (type: rwlock, depth: 3)(type: rwlock, depth: 4) ->pr_lock (type: rwlock, depth: 4) >pr_lock (type: rwlock, depth: 4) ->dh_mtx (type: rwlock, depth: 4) -- (already displayed) >uv_lock (type: rwlock, depth: 3) >uv_lock (type: rwlock, depth: 4) >uv_lock (type: rwlock, depth: 4) -- (already displayed) sysctllk (type: rwlock, depth: 0) _lock (type: kernel_lock, depth: 6) -- (already displayed) >lock (type: rwlock, depth: 5) -- (already displayed) >fd_fd.fd_lock (type: rwlock, depth: 1) _lock (type: kernel_lo ck, depth: 6) -- (already displayed) >lock (type: rwlock, depth: 5) -- (already displayed) netlock (type: rwlock, depth: 2) _lock (type: kernel_lock, depth: 6) -- (already displayed) pools (type: rwlock, depth: 3)_lock (type: kernel_lock, depth: 6) -- (already displayed) >lock (type: rwlock, depth: 5) -- (already displayed) >ar_lock (type: rwlock, depth: 3) vlantag (type: rwlock, depth: 3) lo ckflk (type: rwlock, depth: 3) -- (already displayed) >i_lock (type: rrwlock, depth: 2) -- (already displayed) vfs_stall (type: rwlock, depth: 2) _lock (type: kernel_lock, depth: 6 ) -- (already displayed) ptarrlk (type: rwlock, depth: 2) netlock (type: rwlock, depth: 2) -- (already displayed) tc_lock (type: rwlock, depth: 1) >i_lock (type: rrwlock, depth: 2) -- (alr eady displayed) vfs_stall (type: rwlock, depth: 2) -- (already displayed) sysctldlk (type: rwlock, depth: 1)>hdcp_mutex (type: rwlock, depth: 0)>lock (type: rwlock, depth: 0)>mutex (type: rwlock, depth: 0 ) >mode_config.idr_mutex (type: rwlock, depth: 3)_priv->dpll_lock (typ e: rwlock, depth: 0)taskq (type: rwlock, depth: 0) >struct_mutex (type: rw lock, depth: 3) >lock (type: rwlock, depth: 5) -- (already displayed) (type: rwlock, depth: 4) -- (already displayed) >vm_lock (type: rwlock, depth: 5) _priv->pcu_lock (type: rwlock, de pth: 4) _priv->gt_pm.rps.power.mutex (type: rwlock, depth: 5) _dom ains->lock (type: rwlock, depth: 4) _priv->mm.stolen_lock (type: rwlock, d epth: 4) >lock (type: rwlock, depth: 4) >vm_lock (type: rwlock, de pth: 4) >mm.lock (type: rwlock, depth: 4) >lock (type: rwlock, dep th: 5) -- (already displayed) >vm_lock (type: rwlock, depth: 5) -- (already displayed) >mm.get_page.lock (type: rwlock, depth: 4) >vm_lock (type: rwlock, d epth: 5) -- (already displayed) >mm.lock (type: rwlock, depth: 4) -- (already displayed) _priv->av_mutex (type: rwlock, depth:
RFC 8555 for acme-client(1)
This switches acme-client(1) from draft-03 to the final standard. Let's Encrypt claims - and this seems to be true in my testing - that accounts created on the v01 api (the draf one) work on the v02 api (rfc). This drops support for v01, thinks got shuffled around too much to support both. acme-client(1) complains fairly loudly when it's pointed at an old server: acme-client: https://acme-staging.api.letsencrypt.org/directory: bad CA paths To test it switch the api url: - api url "https://acme-v01.api.letsencrypt.org/directory; + api url "https://acme-v02.api.letsencrypt.org/directory; Tests, Comments, OKs? There is certainly more stuff to do here, but it's already quite big. diff --git etc/examples/acme-client.conf etc/examples/acme-client.conf index 53bc65e17f3..94f1b3b5c72 100644 --- etc/examples/acme-client.conf +++ etc/examples/acme-client.conf @@ -2,12 +2,12 @@ # $OpenBSD: acme-client.conf,v 1.1 2019/01/08 07:14:10 florian Exp $ # authority letsencrypt { - api url "https://acme-v01.api.letsencrypt.org/directory; + api url "https://acme-v02.api.letsencrypt.org/directory; account key "/etc/acme/letsencrypt-privkey.pem" } authority letsencrypt-staging { - api url "https://acme-staging.api.letsencrypt.org/directory; + api url "https://acme-staging-v02.api.letsencrypt.org/directory; account key "/etc/acme/letsencrypt-staging-privkey.pem" } diff --git usr.sbin/acme-client/acctproc.c usr.sbin/acme-client/acctproc.c index 2b71fb2ec79..733cf8a0032 100644 --- usr.sbin/acme-client/acctproc.c +++ usr.sbin/acme-client/acctproc.c @@ -154,18 +154,16 @@ out: } static int -op_sign_rsa(char **head, char **prot, EVP_PKEY *pkey, const char *nonce) +op_sign_rsa(char **prot, EVP_PKEY *pkey, const char *nonce, const char *url) { char*exp = NULL, *mod = NULL; int rc = 0; RSA *r; - *head = NULL; *prot = NULL; /* * First, extract relevant portions of our private key. -* Then construct the public header. * Finally, format the header combined with the nonce. */ @@ -175,9 +173,7 @@ op_sign_rsa(char **head, char **prot, EVP_PKEY *pkey, const char *nonce) warnx("bn2string"); else if ((exp = bn2string(r->e)) == NULL) warnx("bn2string"); - else if ((*head = json_fmt_header_rsa(exp, mod)) == NULL) - warnx("json_fmt_header_rsa"); - else if ((*prot = json_fmt_protected_rsa(exp, mod, nonce)) == NULL) + else if ((*prot = json_fmt_protected_rsa(exp, mod, nonce, url)) == NULL) warnx("json_fmt_protected_rsa"); else rc = 1; @@ -192,11 +188,12 @@ op_sign_rsa(char **head, char **prot, EVP_PKEY *pkey, const char *nonce) * This requires the sender ("fd") to provide the payload and a nonce. */ static int -op_sign(int fd, EVP_PKEY *pkey) +op_sign(int fd, EVP_PKEY *pkey, enum acctop op) { char*nonce = NULL, *pay = NULL, *pay64 = NULL; - char*prot = NULL, *prot64 = NULL, *head = NULL; + char*prot = NULL, *prot64 = NULL; char*sign = NULL, *dig64 = NULL, *fin = NULL; + char*url = NULL, *kid = NULL; unsigned char *dig = NULL; EVP_MD_CTX *ctx = NULL; int cc, rc = 0; @@ -208,6 +205,12 @@ op_sign(int fd, EVP_PKEY *pkey) goto out; else if ((nonce = readstr(fd, COMM_NONCE)) == NULL) goto out; + else if ((url = readstr(fd, COMM_URL)) == NULL) + goto out; + + if (op == ACCT_KID_SIGN) + if ((kid = readstr(fd, COMM_KID)) == NULL) + goto out; /* Base64-encode the payload. */ @@ -216,14 +219,21 @@ op_sign(int fd, EVP_PKEY *pkey) goto out; } - switch (EVP_PKEY_type(pkey->type)) { - case EVP_PKEY_RSA: - if (!op_sign_rsa(, , pkey, nonce)) + if (op == ACCT_KID_SIGN) { + if ((prot = json_fmt_protected_kid(kid, nonce, url)) == NULL) { + warnx("json_fmt_protected_kid"); goto out; - break; - default: - warnx("EVP_PKEY_type"); - goto out; + } + } else { + switch (EVP_PKEY_type(pkey->type)) { + case EVP_PKEY_RSA: + if (!op_sign_rsa(, pkey, nonce, url)) + goto out; + break; + default: + warnx("EVP_PKEY_type"); + goto out; + } } /* The header combined with the nonce, base64. */ @@ -275,7 +285,7 @@ op_sign(int fd, EVP_PKEY *pkey) * when we next enter the read loop). */ - if ((fin = json_fmt_signed(head, prot64, pay64, dig64)) == NULL)
Re: [patch] push the KERNEL_LOCK deeper on read(2) and write(2)
Le 2019-06-05 12:06, Mark Kettenis a écrit : Date: Wed, 5 Jun 2019 10:16:25 +0200 From: Sebastien Marie Hi, I would like to have feedback and testing on this diff. The initial work was done by ian@. Don't forget to run "make syscalls" in sys/kern when building your own kernel with this diff! no problem so far with this diff and the scheduler one.
Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority
I'm testing the "pump my sched" and read/write unlock diffs and ran into the panic below. Seems more likely that it would be connected with the sched diff rather than anything else. I'll build a WITNESS kernel and see if I can get more details if it hapens again. ddb{1}> show panic kernel diagnostic assertion "__mp_lock_held(_lock, curcpu()) == 0" failed : file "/sys/kern/kern_lock.c", line 63 ddb{1}> tr db_enter() at db_enter+0x10 panic() at panic+0x128 __assert(81aebffd,81ac7a49,3f,81b0e315) at __assert+0x2 e _kernel_lock() at _kernel_lock+0xf6 pageflttrap() at pageflttrap+0x75 kerntrap(800033ee0ca0) at kerntrap+0x91 alltraps_kern(6,380,fffb,2,80002201aff0,e0) at alltraps_kern+0x7b setrunqueue(0,8000342b2ce8,e0) at setrunqueue+0xbc setrunnable(8000342b2ce8,e0) at setrunnable+0xa5 ptsignal(8000342b2ce8,13,0) at ptsignal+0x3c4 sys_kill(800033c2fcc0,800033ee0ec0,800033ee0f20) at sys_kill+0x1c5 syscall(800033ee0f90) at syscall+0x389 Xsyscall(6,7a,0,7a,13,7f7e3e18) at Xsyscall+0x128 end of kernel end trace frame: 0x7f7e3cb0, count: -13 ddb{1}> ps PID TID PPIDUID S FLAGS WAIT COMMAND 89165 411452 1971667 30x1000b2 poll ping 1971 11232 86869667 30x82 piperdcheck_ping 66051 380473 92153667 30x1000b2 poll ping 92153 480496 86869667 30x82 piperdcheck_ping 7608 110605 8332667 30x1000b2 poll ping 8332 284560 86869667 30x82 piperdcheck_ping 40097 336365 35317667 30x1000b2 poll ping 35317 385677 86869667 30x82 piperdcheck_ping 33714 102547 21689667 30x1000b2 poll ping 21689 210571 86869667 30x82 piperdcheck_ping 72142 347023 62694667 30x1000b2 poll ping 62694 439660 86869667 30x82 piperdcheck_ping 87347 447624 76730 1000 4 0x8000430xscreensaver 37569 319522 76730 1000 20x800402piecewise 11841 481452 18848667 30x1000b2 poll ping 18848 111230 86869667 30x82 piperdcheck_ping 69292 189646 44902667 30x1000b2 poll ping 44902 21064 86869667 30x82 piperdcheck_ping 67905 363250 1761507 30x92 kqreadpickup 86709 146945 91951 1000 30x92 kqreadimap 36596 227858 91951666 30x92 kqreadimap-login 90723 180281 91951 1000 30x92 kqreadimap 42867 129499 91951 1000 30x92 kqreadimap 52963 203919 91951 1000 30x92 kqreadimap 61599 167915 91951666 30x92 kqreadimap-login 54450 154426 91951666 30x92 kqreadimap-login 54061 103553 91951666 30x92 kqreadimap-login 39422 362611 85872715 30x90 netconperl 49809 260623 85872715 30x90 lockf perl 3824 92473 85872715 30x90 lockf perl 30990 398152 85872715 30x90 lockf perl 3327 264795 85872715 30x90 lockf perl 24406 217143 85872715 30x90 lockf perl 52871 404585 85872715 30x90 lockf perl 58220 118881 85872715 30x90 lockf perl 32898 307331 85872715 30x90 lockf perl 50203 396987 85872715 30x90 lockf perl 93024 24785 1 1000 30x100080 selectssh 22261 482889 52840 1000 30x100083 poll ssh 52840 404967 61470 1000 30x10008b pause ksh 35796 132597 61470 1000 30xb0 netio urxvt 61470 485028 72107 1000 30xb2 selecturxvt 72107 196192 1 1000 30x10008a pause sh 70238 396933 1 1000 30x100080 selectssh 41840 302658 10362 1000 30x100083 poll ssh 89638 411799 91951 1000 30x92 kqreadimap 49250 507579 81821 1000 30x100083 ttyin ksh 37608 523403 91951666 30x92 kqreadimap-login 75552 67606 33096 1000 30x100083 poll mutt 33096 492099 81821 1000 30x10008b pause ksh 81821 270355 1 1000 30x100080 kqreadtmux 24522 341281 91861 1000 30x100083 kqreadtmux 23218 441369 78842 1000 30x82 poll firefox 23218 430421 78842 1000 3 0x482 kqreadfirefox 23218 460884 78842 1000 3 0x482 fsleepfirefox 23218 17619 78842 1000 3 0x482 fsleepfirefox 23218 138796 78842 1000 3 0x482 fsleepfirefox 23218 341276 78842 1000 3 0x482
Re: monotonic clock going backwards?
On 6/6/19 8:49 AM, Mark Kettenis wrote: From: David Hill Date: Thu, 6 Jun 2019 08:22:06 -0400 On 6/6/19 6:56 AM, Scott Cheloha wrote: On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote: Hello - I noticed some negative roundtrip times when running traceroute, which uses the monotonic clock to calculate the RTT. If I run the following code, it eventually bombs.?? It bombs quick if I launch Firefox. timespeccmp failed tp1 s:103780 n:63101042 tp2 s:103779 n:761117849 please share your dmesg and the output of sysctl kern.clockrate kern.timecounter Could you also share the output when you run the following code? With and without firefox running? Curious about how frequently this happens. #include #include #include #include int main(void) { struct timespec t0, t1, timeout; timeout.tv_sec = 0; timeout.tv_nsec = 10; for (;;) { clock_gettime(CLOCK_MONOTONIC, ); if (nanosleep(, NULL) == -1) err(1, "nanosleep"); clock_gettime(CLOCK_MONOTONIC, ); if (timespeccmp(, , >)) { printf("%lld.%09ld precedes %lld.%09ld\n", t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec); } } return 0; } $ sysctl kern.clockrate kern.timecounter kern.clockrate=tick = 1, tickadj = 40, hz = 100, profhz = 100, stathz = 100 kern.timecounter.tick=1 kern.timecounter.timestepwarnings=0 kern.timecounter.hardware=tsc kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000) acpitimer0(1000) 146259.148543164 precedes 146259.450584680 146259.488592813 precedes 146259.790556528 146259.808566236 precedes 146260.110528202 146261.148476141 precedes 146261.450398822 146262.888279505 precedes 146263.190240715 146263.887431808 precedes 146264.189435801 146274.487159451 precedes 146274.789148187 146278.186820816 precedes 146278.488803756 146279.546680181 precedes 146279.848674138 146284.326305557 precedes 146284.628224696 146285.426154861 precedes 146285.728066901 146287.885921929 precedes 146288.187826688 146289.985763711 precedes 146290.287731233 146290.985659041 precedes 146291.287640302 146293.985348615 precedes 146294.287327519 146295.465186742 precedes 146295.767218516 146295.885156426 precedes 146296.187140717 146298.544907956 precedes 146298.846882759 146298.644894308 precedes 146298.946889142 Does this also happen with kern.timecounter.hardware=acpihpet ? Ran the code with kern.timecounter.hardware=acpihpet0 for 10 minutes without any output. With the code still running, I switched to tsc and starting seeing output: 151944.345840447 precedes 151944.647753895 151944.525774690 precedes 151944.827681380 151946.405553296 precedes 151946.707533012 The output stops as soon as I switch back to acpihpet0. I will add that to my /etc/sysctl.conf. Thank you. OpenBSD 6.5-current (GENERIC.MP) #27: Tue Jun 4 15:36:14 EDT 2019 dh...@liver.mindcry.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 8418242560 (8028MB) avail mem = 8152993792 (7775MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x9a565000 (49 entries) bios0: vendor American Megatrends Inc. version "X420UA.204" date 11/16/2018 bios0: ASUSTeK COMPUTER INC. VivoBook_ASUSLaptop X420UA acpi0 at bios0: rev 2 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP APIC FPDT FIDT ECDT MCFG SSDT SSDT SSDT MSDM SSDT HPET SSDT UEFI SSDT LPIT SSDT SSDT SSDT SSDT DBGP DBG2 DMAR NHLT BGRT TPM2 WSMT acpi0: wakeup devices PXSX(S4) RP06(S4) GLAN(S4) XHC_(S3) XDCI(S4) HDAS(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3393.27 MHz, 06-8e-0a cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI 1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 23MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3392.09 MHz, 06-8e-0a cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S
Re: monotonic clock going backwards?
> From: David Hill > Date: Thu, 6 Jun 2019 08:22:06 -0400 > > On 6/6/19 6:56 AM, Scott Cheloha wrote: > > On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote: > >> Hello - > >> > >> I noticed some negative roundtrip times when running traceroute, which uses > >> the monotonic clock to calculate the RTT. > >> > >> If I run the following code, it eventually bombs.?? It bombs quick if I > >> launch Firefox. > >> > >> timespeccmp failed > >> tp1 s:103780 n:63101042 > >> tp2 s:103779 n:761117849 > > please share your dmesg and the output of > > > > sysctl kern.clockrate kern.timecounter > > > > Could you also share the output when you run the following code? > > With and without firefox running? Curious about how frequently > > this happens. > > > > #include > > > > #include > > #include > > #include > > > > int > > main(void) > > { > > struct timespec t0, t1, timeout; > > > > timeout.tv_sec = 0; > > timeout.tv_nsec = 10; > > > > for (;;) { > > clock_gettime(CLOCK_MONOTONIC, ); > > if (nanosleep(, NULL) == -1) > > err(1, "nanosleep"); > > clock_gettime(CLOCK_MONOTONIC, ); > > if (timespeccmp(, , >)) { > > printf("%lld.%09ld precedes %lld.%09ld\n", > > t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec); > > } > > } > > > > return 0; > > } > > > > $ sysctl kern.clockrate kern.timecounter > kern.clockrate=tick = 1, tickadj = 40, hz = 100, profhz = 100, > stathz = 100 > kern.timecounter.tick=1 > kern.timecounter.timestepwarnings=0 > kern.timecounter.hardware=tsc > kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000) acpitimer0(1000) > > 146259.148543164 precedes 146259.450584680 > 146259.488592813 precedes 146259.790556528 > 146259.808566236 precedes 146260.110528202 > 146261.148476141 precedes 146261.450398822 > 146262.888279505 precedes 146263.190240715 > 146263.887431808 precedes 146264.189435801 > 146274.487159451 precedes 146274.789148187 > 146278.186820816 precedes 146278.488803756 > 146279.546680181 precedes 146279.848674138 > 146284.326305557 precedes 146284.628224696 > 146285.426154861 precedes 146285.728066901 > 146287.885921929 precedes 146288.187826688 > 146289.985763711 precedes 146290.287731233 > 146290.985659041 precedes 146291.287640302 > 146293.985348615 precedes 146294.287327519 > 146295.465186742 precedes 146295.767218516 > 146295.885156426 precedes 146296.187140717 > 146298.544907956 precedes 146298.846882759 > 146298.644894308 precedes 146298.946889142 Does this also happen with kern.timecounter.hardware=acpihpet ? > OpenBSD 6.5-current (GENERIC.MP) #27: Tue Jun 4 15:36:14 EDT 2019 > dh...@liver.mindcry.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP > real mem = 8418242560 (8028MB) > avail mem = 8152993792 (7775MB) > mpath0 at root > scsibus0 at mpath0: 256 targets > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x9a565000 (49 entries) > bios0: vendor American Megatrends Inc. version "X420UA.204" date 11/16/2018 > bios0: ASUSTeK COMPUTER INC. VivoBook_ASUSLaptop X420UA > acpi0 at bios0: rev 2 > acpi0: sleep states S0 S3 S4 S5 > acpi0: tables DSDT FACP APIC FPDT FIDT ECDT MCFG SSDT SSDT SSDT MSDM > SSDT HPET SSDT UEFI SSDT LPIT SSDT SSDT SSDT SSDT DBGP DBG2 DMAR NHLT > BGRT TPM2 WSMT > acpi0: wakeup devices PXSX(S4) RP06(S4) GLAN(S4) XHC_(S3) XDCI(S4) HDAS(S4) > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3393.27 MHz, 06-8e-0a > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S > SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI > 1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN > cpu0: 256KB 64b/line 8-way L2 cache > cpu0: smt 0, core 0, package 0 > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges > cpu0: apic clock running at 23MHz > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE > cpu1 at mainbus0: apid 2 (application processor) > cpu1: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3392.09 MHz, 06-8e-0a > cpu1: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S > SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI >
Re: monotonic clock going backwards?
On 6/6/19 6:56 AM, Scott Cheloha wrote: On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote: Hello - I noticed some negative roundtrip times when running traceroute, which uses the monotonic clock to calculate the RTT. If I run the following code, it eventually bombs.?? It bombs quick if I launch Firefox. timespeccmp failed tp1 s:103780 n:63101042 tp2 s:103779 n:761117849 please share your dmesg and the output of sysctl kern.clockrate kern.timecounter Could you also share the output when you run the following code? With and without firefox running? Curious about how frequently this happens. #include #include #include #include int main(void) { struct timespec t0, t1, timeout; timeout.tv_sec = 0; timeout.tv_nsec = 10; for (;;) { clock_gettime(CLOCK_MONOTONIC, ); if (nanosleep(, NULL) == -1) err(1, "nanosleep"); clock_gettime(CLOCK_MONOTONIC, ); if (timespeccmp(, , >)) { printf("%lld.%09ld precedes %lld.%09ld\n", t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec); } } return 0; } $ sysctl kern.clockrate kern.timecounter kern.clockrate=tick = 1, tickadj = 40, hz = 100, profhz = 100, stathz = 100 kern.timecounter.tick=1 kern.timecounter.timestepwarnings=0 kern.timecounter.hardware=tsc kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000) acpitimer0(1000) 146259.148543164 precedes 146259.450584680 146259.488592813 precedes 146259.790556528 146259.808566236 precedes 146260.110528202 146261.148476141 precedes 146261.450398822 146262.888279505 precedes 146263.190240715 146263.887431808 precedes 146264.189435801 146274.487159451 precedes 146274.789148187 146278.186820816 precedes 146278.488803756 146279.546680181 precedes 146279.848674138 146284.326305557 precedes 146284.628224696 146285.426154861 precedes 146285.728066901 146287.885921929 precedes 146288.187826688 146289.985763711 precedes 146290.287731233 146290.985659041 precedes 146291.287640302 146293.985348615 precedes 146294.287327519 146295.465186742 precedes 146295.767218516 146295.885156426 precedes 146296.187140717 146298.544907956 precedes 146298.846882759 146298.644894308 precedes 146298.946889142 OpenBSD 6.5-current (GENERIC.MP) #27: Tue Jun 4 15:36:14 EDT 2019 dh...@liver.mindcry.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 8418242560 (8028MB) avail mem = 8152993792 (7775MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x9a565000 (49 entries) bios0: vendor American Megatrends Inc. version "X420UA.204" date 11/16/2018 bios0: ASUSTeK COMPUTER INC. VivoBook_ASUSLaptop X420UA acpi0 at bios0: rev 2 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP APIC FPDT FIDT ECDT MCFG SSDT SSDT SSDT MSDM SSDT HPET SSDT UEFI SSDT LPIT SSDT SSDT SSDT SSDT DBGP DBG2 DMAR NHLT BGRT TPM2 WSMT acpi0: wakeup devices PXSX(S4) RP06(S4) GLAN(S4) XHC_(S3) XDCI(S4) HDAS(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3393.27 MHz, 06-8e-0a cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI 1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 23MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3392.09 MHz, 06-8e-0a cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI 1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 0 cpu2 at mainbus0: apid 4 (application processor) cpu2: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3392.09 MHz, 06-8e-0a cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S
Re: [patch] rsync: fix free() on uninitialized pointer with -rx and same device
Björn Ketelaars: > Diff below is based on the latest diff from naddy@. Changes: > - reallocarray likes type_t, as such changes type of nxdev and i; > - use reallocarray instead of malloc as xdev is initialised as NULL. ok naddy@ -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: monotonic clock going backwards?
On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote: > Hello - > > I noticed some negative roundtrip times when running traceroute, which uses > the monotonic clock to calculate the RTT. > > If I run the following code, it eventually bombs.?? It bombs quick if I > launch Firefox. > > timespeccmp failed > tp1 s:103780 n:63101042 > tp2 s:103779 n:761117849 please share your dmesg and the output of sysctl kern.clockrate kern.timecounter Could you also share the output when you run the following code? With and without firefox running? Curious about how frequently this happens. #include #include #include #include int main(void) { struct timespec t0, t1, timeout; timeout.tv_sec = 0; timeout.tv_nsec = 10; for (;;) { clock_gettime(CLOCK_MONOTONIC, ); if (nanosleep(, NULL) == -1) err(1, "nanosleep"); clock_gettime(CLOCK_MONOTONIC, ); if (timespeccmp(, , >)) { printf("%lld.%09ld precedes %lld.%09ld\n", t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec); } } return 0; }
Re: [patch] rsync: fix free() on uninitialized pointer with -rx and same device
On Thu 06/06/2019 00:37, Christian Weisgerber wrote: > Christian Weisgerber: > > > > To reproduce: > > > > > > mkdir -p /tmp/test /tmp/plop > > > openrsync -rx /tmp/test/ /tmp/plop/ > > > > > > Result: > > > > > > openrsync(3470) in free(): bogus pointer (double free?) 0x7f7dcdc8 > > > Abort trap (core dumped) > > > > > > The patch below fixes it and simplifies the logic: > > > > I agree. However, the (re)allocation of xdev also looks bogus. > > > > How about this? > > > > (Also, the realloc idiom is exactly the one the man page warns > > against. Do we care here?) > > Well, let's not set a bad example. > Guess I'm late to the party...I only read Hiltjo's mail, and the replies, after being contacted offlist by naddy@. Diff below is based on the latest diff from naddy@. Changes: - reallocarray likes type_t, as such changes type of nxdev and i; - use reallocarray instead of malloc as xdev is initialised as NULL. diff --git flist.c flist.c index e1f41b1a108..4c552ce3c5c 100644 --- flist.c +++ flist.c @@ -803,12 +803,12 @@ flist_gen_dirent(struct sess *sess, char *root, struct flist **fl, size_t *sz, size_t *max) { char*cargv[2], *cp; - int rc = 0, nxdev = 0, flag, i; + int rc = 0, flag; FTS *fts; FTSENT *ent; struct flist*f; - size_t flsz = 0, stripdir; - dev_t *xdev; + size_t i, flsz = 0, nxdev = 0, stripdir; + dev_t *newxdev, *xdev = NULL; struct stat st; cargv[0] = root; @@ -931,11 +931,6 @@ flist_gen_dirent(struct sess *sess, char *root, struct flist **fl, size_t *sz, !S_ISDIR(ent->fts_statp->st_mode)) continue; - if ((xdev = malloc(sizeof(dev_t))) == NULL) { - ERRX1("malloc"); - goto out; - } - flag = 0; for (i = 0; i < nxdev; i++) if (xdev[i] == ent->fts_statp->st_dev) { @@ -945,12 +940,12 @@ flist_gen_dirent(struct sess *sess, char *root, struct flist **fl, size_t *sz, if (flag) continue; - if (nxdev) - if ((xdev = realloc(xdev, sizeof(dev_t))) == - NULL) { - ERRX1("realloc"); - goto out; - } + if ((newxdev = reallocarray(xdev, nxdev + 1, + sizeof(dev_t))) == NULL) { + ERRX1("reallocarray"); + goto out; + } + xdev = newxdev; xdev[nxdev] = ent->fts_statp->st_dev; nxdev++; } @@ -1008,8 +1003,7 @@ flist_gen_dirent(struct sess *sess, char *root, struct flist **fl, size_t *sz, rc = 1; out: fts_close(fts); - if (sess->opts->one_file_system) - free(xdev); + free(xdev); return rc; }
no background scan while waiting for mgt frame
Don't start a background scan while we're waiting for a response from the AP to a management frame we have sent. In particular, the logic which tries to detect dead APs might be waiting for a probe response from the AP. Starting a background scan in this state won't do any good. In case of iwn(4) it seems to result in an endless scan command dance with the firwmare. Associated: Jun 6 10:37:03 jim /bsd: iwn0: missed beacon threshold set to 7 beacons, beacon interval is 100 TU Got "beacon miss" interrupt from firmware; sending probe req to AP to check if AP is still alive: Jun 6 10:37:04 jim /bsd: iwn0: sending probe_req to 00:25:56:73:b7:a2 on channel 1 mode 11n Meanwhile, low signal of received frames kicks off a background scan: Jun 6 10:37:05 jim /bsd: iwn0: begin background scan Response from AP never arrived; interface watchdog moves to SCAN state while background scan isn't done yet: Jun 6 10:37:09 jim /bsd: iwn0: RUN -> SCAN Driver issues SCAN_ABORT and SCAN commands here. Jun 6 10:37:09 jim /bsd: iwn0: end active scan Jun 6 10:37:09 jim /bsd: iwn0: SCAN -> SCAN Jun 6 10:37:09 jim /bsd: iwn0: end active scan Driver issues SCAN command here. Jun 6 10:37:09 jim /bsd: iwn0: SCAN -> SCAN Jun 6 10:37:09 jim /bsd: iwn0: end passive scan Driver issues SCAN command here. Jun 6 10:37:09 jim /bsd: iwn0: SCAN -> SCAN Jun 6 10:37:09 jim /bsd: iwn0: end passive scan Driver issues SCAN command here. Jun 6 10:37:09 jim /bsd: iwn0: SCAN -> SCAN Jun 6 10:37:09 jim /bsd: iwn0: end passive scan Driver issues SCAN command here. These SCAN -> SCAN transitions go on forever until the interface is put down. For some reason we never get scan results from firmware. So this diff is merely masking an actual driver bug; but such bugs are hard to fix without really understanding how firmware's SCAN, SCAN_ABORT commands and "scan done" interrupts interact. I suppose asking Intel for docs would be an exercise in futility... Fortunately, it is easy to avoid this situation in the first place. Note that the inverse case (ignore "missed beacon" events while a background scan is in progress) must be fixed separately in the driver, not the stack. OK? diff c3669ebdb8dc5ae83ec861abde2c7a1cf357a580 /usr/src blob - f0e9b2ba482a96a5d7deb442d094f6ecc63afec0 file + sys/net80211/ieee80211.c --- sys/net80211/ieee80211.c +++ sys/net80211/ieee80211.c @@ -75,7 +75,7 @@ ieee80211_begin_bgscan(struct ifnet *ifp) struct ieee80211com *ic = (void *)ifp; if ((ic->ic_flags & IEEE80211_F_BGSCAN) || - ic->ic_state != IEEE80211_S_RUN) + ic->ic_state != IEEE80211_S_RUN || ic->ic_mgt_timer != 0) return; if (ic->ic_bgscan_start != NULL && ic->ic_bgscan_start(ic) == 0) {
Re: [patch] use acme-client to sign certificated with ecdsa keys
On 6/6/19 10:10 AM, Florian Obser wrote: On Wed, Jun 05, 2019 at 05:37:51PM +0200, Gilles Chehade wrote: On Wed, Jun 05, 2019 at 08:39:51AM +0200, Renaud Allard wrote: On 6/5/19 8:20 AM, Gilles Chehade wrote: On Tue, Jun 04, 2019 at 03:54:11PM +0200, Renaud Allard wrote: On 6/3/19 11:53 AM, Renaud Allard wrote: On 5/29/19 9:58 AM, Florian Obser wrote: why not let acme-client generate the key? Here is a more complete diff where you can use the -E switch to generate a ECDSA key instead of the RSA one. I refined a little bit the patch to not put ecdsa functions into rsa.c. So I renamed rsa.c to key.c and removed the rsa references to functions which apply to both rsa and ecdsa. reads, builds and works fine for me a couple comments inlined I removed the parenthesis and used another wording, removed the RSA from a "Load RSA key" as it might not be RSA and added E to the SYNOPSYS. ok gilles@ We don't do bool in base as far as I know, just use an int. No problem, that's a trivial change. I currently don't have time to review this. I'm busy switching acme-client to the rfc 8555 / letsencrypt v2 api. Doesn't look like this conflicts too badly with my work, but I'd appreciate it if we could hold this off for a bit and rebase it ontop of the v2 work. OK, let's wait till your v2 is done. People have the patch already if they want to try out ecdsa in the meantime. I'd also like to get rid of most of the flags and it feels like RSA vs. ECDSA should be handled in the config file? I think the flags A,D,v and E could indeed belong to the config file. smime.p7s Description: S/MIME Cryptographic Signature
Re: [patch] use acme-client to sign certificated with ecdsa keys
On Wed, Jun 05, 2019 at 05:37:51PM +0200, Gilles Chehade wrote: > On Wed, Jun 05, 2019 at 08:39:51AM +0200, Renaud Allard wrote: > > > > > > On 6/5/19 8:20 AM, Gilles Chehade wrote: > > > On Tue, Jun 04, 2019 at 03:54:11PM +0200, Renaud Allard wrote: > > > > > > > > > > > > On 6/3/19 11:53 AM, Renaud Allard wrote: > > > > > > > > > > > > > > On 5/29/19 9:58 AM, Florian Obser wrote: > > > > > > > > why not let acme-client generate the key? > > > > > > > > > > > > > > > > > > > Here is a more complete diff where you can use the -E switch to > > > > > > generate a ECDSA key instead of the RSA one. > > > > > > > > I refined a little bit the patch to not put ecdsa functions into rsa.c. > > > > So I > > > > renamed rsa.c to key.c and removed the rsa references to functions which > > > > apply to both rsa and ecdsa. > > > > > > > > > > reads, builds and works fine for me > > > > > > a couple comments inlined > > > > > > > I removed the parenthesis and used another wording, removed the RSA from a > > "Load RSA key" as it might not be RSA and added E to the SYNOPSYS. > > > > ok gilles@ We don't do bool in base as far as I know, just use an int. I currently don't have time to review this. I'm busy switching acme-client to the rfc 8555 / letsencrypt v2 api. Doesn't look like this conflicts too badly with my work, but I'd appreciate it if we could hold this off for a bit and rebase it ontop of the v2 work. I'd also like to get rid of most of the flags and it feels like RSA vs. ECDSA should be handled in the config file? > > > Index: Makefile > > === > > RCS file: /cvs/src/usr.sbin/acme-client/Makefile,v > > retrieving revision 1.8 > > diff -u -p -r1.8 Makefile > > --- Makefile3 Jul 2017 22:21:47 - 1.8 > > +++ Makefile5 Jun 2019 06:37:00 - > > @@ -2,7 +2,7 @@ > > PROG= acme-client > > SRCS= acctproc.c base64.c certproc.c chngproc.c dbg.c > > dnsproc.c > > SRCS+= fileproc.c http.c jsmn.c json.c keyproc.c main.c > > netproc.c > > -SRCS+= parse.y revokeproc.c rsa.c util.c > > +SRCS+= parse.y revokeproc.c key.c util.c > > > > MAN= acme-client.1 acme-client.conf.5 > > > > Index: acctproc.c > > === > > RCS file: /cvs/src/usr.sbin/acme-client/acctproc.c,v > > retrieving revision 1.12 > > diff -u -p -r1.12 acctproc.c > > --- acctproc.c 28 Jul 2018 15:25:23 - 1.12 > > +++ acctproc.c 5 Jun 2019 06:37:00 - > > @@ -82,8 +82,8 @@ op_thumb_rsa(EVP_PKEY *pkey) > > warnx("bn2string"); > > else if ((exp = bn2string(r->e)) == NULL) > > warnx("bn2string"); > > - else if ((json = json_fmt_thumb_rsa(exp, mod)) == NULL) > > - warnx("json_fmt_thumb_rsa"); > > + else if ((json = json_fmt_thumb_key(exp, mod)) == NULL) > > + warnx("json_fmt_thumb_key"); > > > > free(exp); > > free(mod); > > @@ -175,10 +175,10 @@ op_sign_rsa(char **head, char **prot, EV > > warnx("bn2string"); > > else if ((exp = bn2string(r->e)) == NULL) > > warnx("bn2string"); > > - else if ((*head = json_fmt_header_rsa(exp, mod)) == NULL) > > - warnx("json_fmt_header_rsa"); > > - else if ((*prot = json_fmt_protected_rsa(exp, mod, nonce)) == NULL) > > - warnx("json_fmt_protected_rsa"); > > + else if ((*head = json_fmt_header_key(exp, mod)) == NULL) > > + warnx("json_fmt_header_key"); > > + else if ((*prot = json_fmt_protected_key(exp, mod, nonce)) == NULL) > > + warnx("json_fmt_protected_key"); > > else > > rc = 1; > > > > @@ -338,7 +338,7 @@ acctproc(int netsock, const char *acctke > > goto out; > > dodbg("%s: generated RSA account key", acctkey); > > } else { > > - if ((pkey = rsa_key_load(f, acctkey)) == NULL) > > + if ((pkey = key_load(f, acctkey)) == NULL) > > goto out; > > doddbg("%s: loaded RSA account key", acctkey); > > } > > Index: acme-client.1 > > === > > RCS file: /cvs/src/usr.sbin/acme-client/acme-client.1,v > > retrieving revision 1.29 > > diff -u -p -r1.29 acme-client.1 > > --- acme-client.1 3 Feb 2019 20:39:35 - 1.29 > > +++ acme-client.1 5 Jun 2019 06:37:00 - > > @@ -22,7 +22,7 @@ > > .Nd ACME client > > .Sh SYNOPSIS > > .Nm acme-client > > -.Op Fl ADFnrv > > +.Op Fl ADEFnrv > > .Op Fl f Ar configfile > > .Ar domain > > .Sh DESCRIPTION > > @@ -79,7 +79,9 @@ The options are as follows: > > .It Fl A > > Create a new RSA account key if one does not already exist. > > .It Fl D > > -Create a new RSA domain key if one does not already exist. > > +Create a new domain key if one does not already exist. Defaults to RSA. > > +.It Fl E > > +Switch the
Re: [patch] rsync: fix free() on uninitialized pointer with -rx and same device
OK On Thu, Jun 06, 2019 at 12:37:13AM +0200, Christian Weisgerber wrote: > Christian Weisgerber: > > > > To reproduce: > > > > > > mkdir -p /tmp/test /tmp/plop > > > openrsync -rx /tmp/test/ /tmp/plop/ > > > > > > Result: > > > > > > openrsync(3470) in free(): bogus pointer (double free?) 0x7f7dcdc8 > > > Abort trap (core dumped) > > > > > > The patch below fixes it and simplifies the logic: > > > > I agree. However, the (re)allocation of xdev also looks bogus. > > > > How about this? > > > > (Also, the realloc idiom is exactly the one the man page warns > > against. Do we care here?) > > Well, let's not set a bad example. > > Index: flist.c > === > RCS file: /cvs/src/usr.bin/rsync/flist.c,v > retrieving revision 1.27 > diff -u -p -r1.27 flist.c > --- flist.c 2 Jun 2019 14:29:58 - 1.27 > +++ flist.c 5 Jun 2019 22:36:28 - > @@ -808,7 +808,7 @@ flist_gen_dirent(struct sess *sess, char > FTSENT *ent; > struct flist*f; > size_t flsz = 0, stripdir; > - dev_t *xdev; > + dev_t *newxdev, *xdev = NULL; > struct stat st; > > cargv[0] = root; > @@ -931,7 +931,8 @@ flist_gen_dirent(struct sess *sess, char > !S_ISDIR(ent->fts_statp->st_mode)) > continue; > > - if ((xdev = malloc(sizeof(dev_t))) == NULL) { > + if (xdev == NULL && > + (xdev = malloc(sizeof(dev_t))) == NULL) { > ERRX1("malloc"); > goto out; > } > @@ -945,12 +946,14 @@ flist_gen_dirent(struct sess *sess, char > if (flag) > continue; > > - if (nxdev) > - if ((xdev = realloc(xdev, sizeof(dev_t))) == > - NULL) { > + if (nxdev) { > + if ((newxdev = reallocarray(xdev, nxdev + 1, > + sizeof(dev_t))) == NULL) { > ERRX1("realloc"); > goto out; > } > + xdev = newxdev; > + } > xdev[nxdev] = ent->fts_statp->st_dev; > nxdev++; > } > @@ -1008,8 +1011,7 @@ flist_gen_dirent(struct sess *sess, char > rc = 1; > out: > fts_close(fts); > - if (sess->opts->one_file_system) > - free(xdev); > + free(xdev); > return rc; > } > > -- > Christian "naddy" Weisgerber na...@mips.inka.de > -- I'm not entirely sure you are real.
Re: [patch] rsync: fix free() on uninitialized pointer with -rx and same device
On Thu, Jun 06, 2019 at 12:37:13AM +0200, Christian Weisgerber wrote: > Christian Weisgerber: > > > > To reproduce: > > > > > > mkdir -p /tmp/test /tmp/plop > > > openrsync -rx /tmp/test/ /tmp/plop/ > > > > > > Result: > > > > > > openrsync(3470) in free(): bogus pointer (double free?) 0x7f7dcdc8 > > > Abort trap (core dumped) > > > > > > The patch below fixes it and simplifies the logic: > > > > I agree. However, the (re)allocation of xdev also looks bogus. > > Hi, I noticed the realloc issue also and wanted to send a separate patch later for it. Nice it is addressed also :) > > How about this? > > > > (Also, the realloc idiom is exactly the one the man page warns > > against. Do we care here?) > > Well, let's not set a bad example. > > Index: flist.c > === > RCS file: /cvs/src/usr.bin/rsync/flist.c,v > retrieving revision 1.27 > diff -u -p -r1.27 flist.c > --- flist.c 2 Jun 2019 14:29:58 - 1.27 > +++ flist.c 5 Jun 2019 22:36:28 - > @@ -808,7 +808,7 @@ flist_gen_dirent(struct sess *sess, char > FTSENT *ent; > struct flist*f; > size_t flsz = 0, stripdir; > - dev_t *xdev; > + dev_t *newxdev, *xdev = NULL; > struct stat st; > > cargv[0] = root; > @@ -931,7 +931,8 @@ flist_gen_dirent(struct sess *sess, char > !S_ISDIR(ent->fts_statp->st_mode)) > continue; > > - if ((xdev = malloc(sizeof(dev_t))) == NULL) { > + if (xdev == NULL && > + (xdev = malloc(sizeof(dev_t))) == NULL) { > ERRX1("malloc"); > goto out; > } > @@ -945,12 +946,14 @@ flist_gen_dirent(struct sess *sess, char > if (flag) > continue; > > - if (nxdev) > - if ((xdev = realloc(xdev, sizeof(dev_t))) == > - NULL) { > + if (nxdev) { > + if ((newxdev = reallocarray(xdev, nxdev + 1, > + sizeof(dev_t))) == NULL) { Also could the first malloc and realloc maybe get simplified to reallocarray(NULL, ...) ? > ERRX1("realloc"); > goto out; > } > + xdev = newxdev; > + } > xdev[nxdev] = ent->fts_statp->st_dev; > nxdev++; > } > @@ -1008,8 +1011,7 @@ flist_gen_dirent(struct sess *sess, char > rc = 1; > out: > fts_close(fts); > - if (sess->opts->one_file_system) > - free(xdev); > + free(xdev); > return rc; > } > > -- > Christian "naddy" Weisgerber na...@mips.inka.de > -- Kind regards, Hiltjo