Re: ospfd: allow specifying area by number as well as id

2019-06-06 Thread David Gwynne
yes :D

> On 29 May 2019, at 15:05, Remi Locherer  wrote:
> 
> Hi David,
> 
> are you going to commit this?
> 
> Remi
> 
> 
> On Thu, May 16, 2019 at 11:14:55PM +0200, Remi Locherer wrote:
>> On Thu, May 16, 2019 at 09:39:37AM +0200, Sebastian Benoit wrote:
>>> 
>>> 
>>> 
>>> Remi Locherer(remi.loche...@relo.ch) on 2019.05.15 23:15:03 +0200:
 On Tue, Apr 30, 2019 at 11:10:37PM +0200, Remi Locherer wrote:
> On Mon, Apr 29, 2019 at 11:10:31AM +0100, Stuart Henderson wrote:
>> On 2019/04/29 11:58, Sebastian Benoit wrote:
>>> David Gwynne(da...@gwynne.id.au) on 2019.04.29 19:36:51 +1000:
 
 
> On 29 Apr 2019, at 4:59 pm, Remi Locherer  
> wrote:
> 
> Hi David
> 
> On Mon, Apr 29, 2019 at 11:53:27AM +1000, David Gwynne wrote:
>> it's always bothered me that i config areas on a crisco using a 
>> number,
>> but then have to think hard to convert that number to an address for 
>> use
>> in openbsd. eg, i was given area 700 in one place, which is 0.0.2.188
>> as an address. super annoying.
>> 
>> so this changes the ospfd parser so it accepts both a number or 
>> address.
>> i also changed it so it prints the number by default, which may be
>> contentious. the manpage is slightly tweaked too.
>> 
>> thoughts?
> 
> I like it to be able to use a number instead of an address!
> 
> It worked fine in my short test I performed.
> 
> The output with the comment looks a bit strange to me.
 
 Are you sure it doesn't look... awesome?
>>> 
>>> I like it!
>> 
>> I don't really, but if we change this it needs to be displayed somehow
>> and I don't have an idea to make it look nicer than this (cisco's method
>> seems pretty horrible and wouldn't work for us anyway - looks like they
>> remember which format was used to configure an area and use that as
>> the output format...)
>> 
> 
> Maybe it's better when we just allow both input formats but don't change
> any output.
 
 Any opinions or comments on this? I think this would be a valuable addition
 to ospfd.
>>> 
>>> Yes, and diff is ok benno@
>>> 
>> 
>> David: ok remi@ for your diff without the printconf part.
>> 
>>> What about ospf6d?
>> 
>> I'll handle that.
>> 
>>> 
> 
> Below diff changes ospfctl to accept the address and number format for
> "ospfct show database area XXX".
> 
> 
> Index: parser.c
> ===
> RCS file: /cvs/src/usr.sbin/ospfctl/parser.c,v
> retrieving revision 1.20
> diff -u -p -r1.20 parser.c
> --- parser.c  9 May 2011 12:25:35 -   1.20
> +++ parser.c  30 Apr 2019 20:28:18 -
> @@ -39,7 +39,8 @@ enum token_type {
>   ADDRESS,
>   FLAG,
>   PREFIX,
> - IFNAME
> + IFNAME,
> + AREA
> };
> 
> struct token {
> @@ -107,7 +108,7 @@ static const struct token t_show_db[] = 
> };
> 
> static const struct token t_show_area[] = {
> - {ADDRESS,   "", NONE,   NULL},
> + {AREA,  "", NONE,   NULL},
>   {ENDTOKEN,  "", NONE,   NULL}
> };
> 
> @@ -218,6 +219,14 @@ match_token(const char *word, const stru
>   res->action = t->value;
>   }
>   break;
> + case AREA:
> + if (parse_area(word, >addr)) {
> + match++;
> + t = [i];
> + if (t->value)
> + res->action = t->value;
> + }
> + break;
>   case PREFIX:
>   if (parse_prefix(word, >addr, >prefixlen)) {
>   match++;
> @@ -274,6 +283,9 @@ show_valid_args(const struct token *tabl
>   case ADDRESS:
>   fprintf(stderr, "  \n");
>   break;
> + case AREA:
> + fprintf(stderr, "  \n");
> + break;
>   case PREFIX:
>   fprintf(stderr, "  [/]\n");
>   break;
> @@ -298,6 +310,32 @@ parse_addr(const char *word, struct in_a
>   bzero(, sizeof(ina));
> 
>   if (inet_pton(AF_INET, word, )) {
> + addr->s_addr = ina.s_addr;
> + return (1);
> + }
> +
> + return (0);
> +}
> +
> +int
> +parse_area(const char *word, struct in_addr *addr)
> +{
> + struct in_addr   ina;
> + const char  *errstr;
> +
> + if (word == NULL)
> + return (0);
> +
> + bzero(addr, sizeof(struct in_addr));

hardware packet timestamping for mbufs, then sockets and bpf

2019-06-06 Thread David Gwynne
nics are starting to offer the ability to timestamp packets when
they're received. other systems (eg linux and freebsd) have support
for recording that timestamp on mbufs and then using it as the
backend for at least the SO_TIMESTAMP socket option instead of a
call to microtime().

this implements the above, and additionally supports using the hw
timestamp in bpf too. other systems may do the bpf thing too, but i
didn't look closely enough to find out.

timestamps are recorded as the uptime of the system in nanoseconds
in the ph_timestamp field in mbufs. this mirrors the use of
ph_timestamp in the fq_codel code to store the uptime in nanoseconds.
im using another bit in m_pkthdr.csum_flags to say whether the timestamp
is valid or not (M_TIMESTAMP). im arguing that it's another offloading
feature and therefore appropriate for the csum offload flags field.

this adds some inline functions to time.h for turning ns into a timeval
and timespec, which are ns_to_microtime and ns_to_nanotime respectively.
i originally wanted ns_to_timeval and ns_to_timespec, but the linux
compat stuff in drm already uses those names and ruined the idea.
especially since they return the time{val,spec}s as values.

the ipv4 SO_TIMESTAMP and bpf code looks at whether M_TIMESTAMP is set,
and if so turns ph_timestamp into a timeval before adding it to boottime
(which is the wall clock time that uptime starts at), before using it
instead of microtime().

the mcx changes are based on what freebsd did to their driver, but
simplified a bit.

i want this because we're being asked to look at recording network
traffic for possible audit use. part of that is having accurate
timestamps on received packets, and hopefully it will mitigate against
chunks of packets getting reordered or delayed significantly when the
box is busy.

thoughts? ok?

Index: sys/mbuf.h
===
RCS file: /cvs/src/sys/sys/mbuf.h,v
retrieving revision 1.242
diff -u -p -r1.242 mbuf.h
--- sys/mbuf.h  11 Feb 2019 00:25:33 -  1.242
+++ sys/mbuf.h  7 Jun 2019 03:27:41 -
@@ -226,13 +226,14 @@ struct mbuf {
 #defineM_ICMP_CSUM_IN_OK   0x0400  /* ICMP/ICMPv6 checksum 
verified */
 #defineM_ICMP_CSUM_IN_BAD  0x0800  /* ICMP/ICMPv6 checksum bad */
 #defineM_IPV6_DF_OUT   0x1000  /* don't fragment outgoing IPv6 
*/
+#defineM_TIMESTAMP 0x2000  /* ph_timestamp is set */
 
 #ifdef _KERNEL
 #define MCS_BITS \
 ("\20\1IPV4_CSUM_OUT\2TCP_CSUM_OUT\3UDP_CSUM_OUT\4IPV4_CSUM_IN_OK" \
 "\5IPV4_CSUM_IN_BAD\6TCP_CSUM_IN_OK\7TCP_CSUM_IN_BAD\10UDP_CSUM_IN_OK" \
 "\11UDP_CSUM_IN_BAD\12ICMP_CSUM_OUT\13ICMP_CSUM_IN_OK\14ICMP_CSUM_IN_BAD" \
-"\15IPV6_NODF_OUT")
+"\15IPV6_NODF_OUT" "\16TIMESTAMP")
 #endif
 
 /* mbuf types */
Index: sys/time.h
===
RCS file: /cvs/src/sys/sys/time.h,v
retrieving revision 1.41
diff -u -p -r1.41 time.h
--- sys/time.h  3 Jun 2019 01:27:30 -   1.41
+++ sys/time.h  7 Jun 2019 03:27:41 -
@@ -333,6 +333,20 @@ void clock_secs_to_ymdhms(time_t, struct
 /* Traditional POSIX base year */
 #define POSIX_BASE_YEAR 1970
 
+static __inline void
+ns_to_microtime(struct timeval *tv, uint64_t ns)
+{
+   tv->tv_sec = ns / 10L;
+   tv->tv_usec = (ns % 10L) / 1000;
+}
+
+static __inline void
+ns_to_nanotime(struct timespec *tv, uint64_t ns)
+{
+   tv->tv_sec = ns / 10L;
+   tv->tv_nsec = ns % 10L;
+}
+
 #else /* !_KERNEL */
 #include 
 
Index: net/bpf.c
===
RCS file: /cvs/src/sys/net/bpf.c,v
retrieving revision 1.175
diff -u -p -r1.175 bpf.c
--- net/bpf.c   18 May 2019 12:59:32 -  1.175
+++ net/bpf.c   7 Jun 2019 03:27:41 -
@@ -1284,13 +1284,25 @@ _bpf_mtap(caddr_t arg, const struct mbuf
fcode = bps->bps_bf.bf_insns;
slen = bpf_mfilter(fcode, m, pktlen);
 
-   if (slen == 0)
+   if (slen == 0)
continue;
if (d->bd_fildrop != BPF_FILDROP_PASS)
drop = 1;
if (d->bd_fildrop != BPF_FILDROP_DROP) {
-   if (!gottime++)
-   microtime();
+   if (!gottime) {
+   if (ISSET(m->m_flags, M_PKTHDR) &&
+   ISSET(m->m_pkthdr.csum_flags,
+M_TIMESTAMP)) {
+   struct timeval btv;
+   ns_to_microtime(,
+   m->m_pkthdr.ph_timestamp);
+   microboottime();
+   timeradd(, , );
+   } else
+   microtime();
+
+  

Re: 64-bit dma for drm(4) on amd64

2019-06-06 Thread Jonathan Gray
On Thu, Jun 06, 2019 at 11:41:07PM +0200, Mark Kettenis wrote:
> As a result of a recent discussion with jsg@, I realized that the
> graphics drivers are (mostly) allocating memory from the dma region.
> Since the the graphics stack can potentially gobble up large amounts
> of memory, this means we can run out of dma memory which makes other
> parts of our kernel quite unhappy.  Most of the supported hardware
> actually supports 64-bit DMA just fine, and the drivers already have
> code to handle the exceptions.  The diff below makes use of this
> knowledge to (hopefully) safely allocate from "high" memory when
> possible.  One big change is that this makes bus_dma(9) 64-bit DMA
> aware in the sense that if the BUS_DMA_64BIT flag is used, we skip the
> "not dma-reachable" panic.
> 
> It seems to work fine on my Intel Broadwell laptop.  I haven't tested
> this on radeon(4) yet.  So further testing, especially on systems with
> 4GB of memory or more is necessary.
> 
> Please test.

One of the ways dma32 is set in radeon/amdgpu is along the lines of

dma_bits = rdev->need_dma32 ? 32 : 40;
r = pci_set_dma_mask(rdev->pdev, DMA_BIT_MASK(dma_bits));
if (r) {
rdev->need_dma32 = true;
dma_bits = 32;

which we don't handle.  That is for the case where the card supports
64 bit dma but the system doesn't?  Not something we should be
concerned about?

Having __GFP_DMA32 set the sign bit on the flags seems like it is asking
for trouble.  How about 0x4000 / bit 30 instead of
0x8000 / bit 31.

Otherwise looks good.

> 
> 
> Index: arch/amd64/amd64/bus_dma.c
> ===
> RCS file: /cvs/src/sys/arch/amd64/amd64/bus_dma.c,v
> retrieving revision 1.50
> diff -u -p -r1.50 bus_dma.c
> --- arch/amd64/amd64/bus_dma.c14 Oct 2017 04:44:43 -  1.50
> +++ arch/amd64/amd64/bus_dma.c6 Jun 2019 21:19:21 -
> @@ -319,7 +319,8 @@ _bus_dmamap_load_raw(bus_dma_tag_t t, bu
>   if (plen < sgsize)
>   sgsize = plen;
>  
> - if (paddr > dma_constraint.ucr_high)
> + if (paddr > dma_constraint.ucr_high &&
> + (map->_dm_flags & BUS_DMA_64BIT) == 0)
>   panic("Non dma-reachable buffer at paddr 
> %#lx(raw)",
>   paddr);
>  
> @@ -583,7 +584,8 @@ _bus_dmamap_load_buffer(bus_dma_tag_t t,
>*/
>   pmap_extract(pmap, vaddr, (paddr_t *));
>  
> - if (curaddr > dma_constraint.ucr_high)
> + if (curaddr > dma_constraint.ucr_high &&
> + (map->_dm_flags & BUS_DMA_64BIT) == 0)
>   panic("Non dma-reachable buffer at curaddr %#lx(raw)",
>   curaddr);
>  
> Index: dev/pci/drm/drm_linux.c
> ===
> RCS file: /cvs/src/sys/dev/pci/drm/drm_linux.c,v
> retrieving revision 1.37
> diff -u -p -r1.37 drm_linux.c
> --- dev/pci/drm/drm_linux.c   4 Jun 2019 12:08:22 -   1.37
> +++ dev/pci/drm/drm_linux.c   6 Jun 2019 21:19:21 -
> @@ -293,16 +293,19 @@ struct vm_page *
>  alloc_pages(unsigned int gfp_mask, unsigned int order)
>  {
>   int flags = (gfp_mask & M_NOWAIT) ? UVM_PLA_NOWAIT : UVM_PLA_WAITOK;
> + struct uvm_constraint_range *constraint = _constraint;
>   struct pglist mlist;
>  
>   if (gfp_mask & M_CANFAIL)
>   flags |= UVM_PLA_FAILOK;
>   if (gfp_mask & M_ZERO)
>   flags |= UVM_PLA_ZERO;
> + if (gfp_mask & __GFP_DMA32)
> + constraint = _constraint;
>  
>   TAILQ_INIT();
> - if (uvm_pglistalloc(PAGE_SIZE << order, dma_constraint.ucr_low,
> - dma_constraint.ucr_high, PAGE_SIZE, 0, , 1, flags))
> + if (uvm_pglistalloc(PAGE_SIZE << order, constraint->ucr_low,
> + constraint->ucr_high, PAGE_SIZE, 0, , 1, flags))
>   return NULL;
>   return TAILQ_FIRST();
>  }
> Index: dev/pci/drm/include/linux/gfp.h
> ===
> RCS file: /cvs/src/sys/dev/pci/drm/include/linux/gfp.h,v
> retrieving revision 1.1
> diff -u -p -r1.1 gfp.h
> --- dev/pci/drm/include/linux/gfp.h   14 Apr 2019 10:14:53 -  1.1
> +++ dev/pci/drm/include/linux/gfp.h   6 Jun 2019 21:19:21 -
> @@ -7,24 +7,25 @@
>  #include 
>  #include 
>  
> -#define GFP_ATOMIC   M_NOWAIT
> -#define GFP_NOWAIT   M_NOWAIT
> -#define GFP_KERNEL   (M_WAITOK | M_CANFAIL)
> -#define GFP_USER (M_WAITOK | M_CANFAIL)
> -#define GFP_TEMPORARY(M_WAITOK | M_CANFAIL)
> -#define GFP_HIGHUSER 0
> -#define GFP_DMA320
> -#define __GFP_NOWARN 0
> -#define __GFP_NORETRY0
> -#define __GFP_ZERO   M_ZERO
> +#define __GFP_ZERO   M_ZERO
> +#define __GFP_DMA32  0x8000
> +#define __GFP_NOWARN 0
> +#define __GFP_NORETRY0
>  #define __GFP_RETRY_MAYFAIL  0
>  

Re: 64-bit dma for drm(4) on amd64

2019-06-06 Thread Thomas Frohwein
On Thu, Jun 06, 2019 at 11:41:07PM +0200, Mark Kettenis wrote:
[...]
> It seems to work fine on my Intel Broadwell laptop.  I haven't tested
> this on radeon(4) yet.  So further testing, especially on systems with
> 4GB of memory or more is necessary.
> 
> Please test.

Tested on a desktop with 8GB RAM and amdgpu enabled (GPU is a Vega 64,
but that may not be relevant). I'm attaching dmesg below. Tested for
about 2 hours with some Chrome, some Firefox, mono... No regression
noticed. The situations where the display freezes were encountered as
before building the kernel with this patch, that is with Chrome after a
while with Google Maps (one of the most reliable triggers). Also with
`piglit run all results/all` after 39 tests (dmafence on wait channel
again).

If there's more that would help testing or reporting, I'm happy to.

> 
> 
> Index: arch/amd64/amd64/bus_dma.c
> ===
> RCS file: /cvs/src/sys/arch/amd64/amd64/bus_dma.c,v
> retrieving revision 1.50
> diff -u -p -r1.50 bus_dma.c
> --- arch/amd64/amd64/bus_dma.c14 Oct 2017 04:44:43 -  1.50
> +++ arch/amd64/amd64/bus_dma.c6 Jun 2019 21:19:21 -
> @@ -319,7 +319,8 @@ _bus_dmamap_load_raw(bus_dma_tag_t t, bu
>   if (plen < sgsize)
>   sgsize = plen;
>  
> - if (paddr > dma_constraint.ucr_high)
> + if (paddr > dma_constraint.ucr_high &&
> + (map->_dm_flags & BUS_DMA_64BIT) == 0)
>   panic("Non dma-reachable buffer at paddr 
> %#lx(raw)",
>   paddr);
>  
> @@ -583,7 +584,8 @@ _bus_dmamap_load_buffer(bus_dma_tag_t t,
>*/
>   pmap_extract(pmap, vaddr, (paddr_t *));
>  
> - if (curaddr > dma_constraint.ucr_high)
> + if (curaddr > dma_constraint.ucr_high &&
> + (map->_dm_flags & BUS_DMA_64BIT) == 0)
>   panic("Non dma-reachable buffer at curaddr %#lx(raw)",
>   curaddr);
>  
> Index: dev/pci/drm/drm_linux.c
> ===
> RCS file: /cvs/src/sys/dev/pci/drm/drm_linux.c,v
> retrieving revision 1.37
> diff -u -p -r1.37 drm_linux.c
> --- dev/pci/drm/drm_linux.c   4 Jun 2019 12:08:22 -   1.37
> +++ dev/pci/drm/drm_linux.c   6 Jun 2019 21:19:21 -
> @@ -293,16 +293,19 @@ struct vm_page *
>  alloc_pages(unsigned int gfp_mask, unsigned int order)
>  {
>   int flags = (gfp_mask & M_NOWAIT) ? UVM_PLA_NOWAIT : UVM_PLA_WAITOK;
> + struct uvm_constraint_range *constraint = _constraint;
>   struct pglist mlist;
>  
>   if (gfp_mask & M_CANFAIL)
>   flags |= UVM_PLA_FAILOK;
>   if (gfp_mask & M_ZERO)
>   flags |= UVM_PLA_ZERO;
> + if (gfp_mask & __GFP_DMA32)
> + constraint = _constraint;
>  
>   TAILQ_INIT();
> - if (uvm_pglistalloc(PAGE_SIZE << order, dma_constraint.ucr_low,
> - dma_constraint.ucr_high, PAGE_SIZE, 0, , 1, flags))
> + if (uvm_pglistalloc(PAGE_SIZE << order, constraint->ucr_low,
> + constraint->ucr_high, PAGE_SIZE, 0, , 1, flags))
>   return NULL;
>   return TAILQ_FIRST();
>  }
> Index: dev/pci/drm/include/linux/gfp.h
> ===
> RCS file: /cvs/src/sys/dev/pci/drm/include/linux/gfp.h,v
> retrieving revision 1.1
> diff -u -p -r1.1 gfp.h
> --- dev/pci/drm/include/linux/gfp.h   14 Apr 2019 10:14:53 -  1.1
> +++ dev/pci/drm/include/linux/gfp.h   6 Jun 2019 21:19:21 -
> @@ -7,24 +7,25 @@
>  #include 
>  #include 
>  
> -#define GFP_ATOMIC   M_NOWAIT
> -#define GFP_NOWAIT   M_NOWAIT
> -#define GFP_KERNEL   (M_WAITOK | M_CANFAIL)
> -#define GFP_USER (M_WAITOK | M_CANFAIL)
> -#define GFP_TEMPORARY(M_WAITOK | M_CANFAIL)
> -#define GFP_HIGHUSER 0
> -#define GFP_DMA320
> -#define __GFP_NOWARN 0
> -#define __GFP_NORETRY0
> -#define __GFP_ZERO   M_ZERO
> +#define __GFP_ZERO   M_ZERO
> +#define __GFP_DMA32  0x8000
> +#define __GFP_NOWARN 0
> +#define __GFP_NORETRY0
>  #define __GFP_RETRY_MAYFAIL  0
>  #define __GFP_MOVABLE0
>  #define __GFP_COMP   0
> -#define GFP_TRANSHUGE_LIGHT  0
>  #define __GFP_KSWAPD_RECLAIM 0
>  #define __GFP_HIGHMEM0
>  #define __GFP_RECLAIMABLE0
> -#define __GFP_DMA32  0
> +
> +#define GFP_ATOMIC   M_NOWAIT
> +#define GFP_NOWAIT   M_NOWAIT
> +#define GFP_KERNEL   (M_WAITOK | M_CANFAIL)
> +#define GFP_USER (M_WAITOK | M_CANFAIL)
> +#define GFP_TEMPORARY(M_WAITOK | M_CANFAIL)
> +#define GFP_HIGHUSER 0
> +#define GFP_DMA32__GFP_DMA32
> +#define GFP_TRANSHUGE_LIGHT  0
>  
>  static inline bool
>  gfpflags_allow_blocking(const unsigned int flags)
> Index: 

64-bit dma for drm(4) on amd64

2019-06-06 Thread Mark Kettenis
As a result of a recent discussion with jsg@, I realized that the
graphics drivers are (mostly) allocating memory from the dma region.
Since the the graphics stack can potentially gobble up large amounts
of memory, this means we can run out of dma memory which makes other
parts of our kernel quite unhappy.  Most of the supported hardware
actually supports 64-bit DMA just fine, and the drivers already have
code to handle the exceptions.  The diff below makes use of this
knowledge to (hopefully) safely allocate from "high" memory when
possible.  One big change is that this makes bus_dma(9) 64-bit DMA
aware in the sense that if the BUS_DMA_64BIT flag is used, we skip the
"not dma-reachable" panic.

It seems to work fine on my Intel Broadwell laptop.  I haven't tested
this on radeon(4) yet.  So further testing, especially on systems with
4GB of memory or more is necessary.

Please test.


Index: arch/amd64/amd64/bus_dma.c
===
RCS file: /cvs/src/sys/arch/amd64/amd64/bus_dma.c,v
retrieving revision 1.50
diff -u -p -r1.50 bus_dma.c
--- arch/amd64/amd64/bus_dma.c  14 Oct 2017 04:44:43 -  1.50
+++ arch/amd64/amd64/bus_dma.c  6 Jun 2019 21:19:21 -
@@ -319,7 +319,8 @@ _bus_dmamap_load_raw(bus_dma_tag_t t, bu
if (plen < sgsize)
sgsize = plen;
 
-   if (paddr > dma_constraint.ucr_high)
+   if (paddr > dma_constraint.ucr_high &&
+   (map->_dm_flags & BUS_DMA_64BIT) == 0)
panic("Non dma-reachable buffer at paddr 
%#lx(raw)",
paddr);
 
@@ -583,7 +584,8 @@ _bus_dmamap_load_buffer(bus_dma_tag_t t,
 */
pmap_extract(pmap, vaddr, (paddr_t *));
 
-   if (curaddr > dma_constraint.ucr_high)
+   if (curaddr > dma_constraint.ucr_high &&
+   (map->_dm_flags & BUS_DMA_64BIT) == 0)
panic("Non dma-reachable buffer at curaddr %#lx(raw)",
curaddr);
 
Index: dev/pci/drm/drm_linux.c
===
RCS file: /cvs/src/sys/dev/pci/drm/drm_linux.c,v
retrieving revision 1.37
diff -u -p -r1.37 drm_linux.c
--- dev/pci/drm/drm_linux.c 4 Jun 2019 12:08:22 -   1.37
+++ dev/pci/drm/drm_linux.c 6 Jun 2019 21:19:21 -
@@ -293,16 +293,19 @@ struct vm_page *
 alloc_pages(unsigned int gfp_mask, unsigned int order)
 {
int flags = (gfp_mask & M_NOWAIT) ? UVM_PLA_NOWAIT : UVM_PLA_WAITOK;
+   struct uvm_constraint_range *constraint = _constraint;
struct pglist mlist;
 
if (gfp_mask & M_CANFAIL)
flags |= UVM_PLA_FAILOK;
if (gfp_mask & M_ZERO)
flags |= UVM_PLA_ZERO;
+   if (gfp_mask & __GFP_DMA32)
+   constraint = _constraint;
 
TAILQ_INIT();
-   if (uvm_pglistalloc(PAGE_SIZE << order, dma_constraint.ucr_low,
-   dma_constraint.ucr_high, PAGE_SIZE, 0, , 1, flags))
+   if (uvm_pglistalloc(PAGE_SIZE << order, constraint->ucr_low,
+   constraint->ucr_high, PAGE_SIZE, 0, , 1, flags))
return NULL;
return TAILQ_FIRST();
 }
Index: dev/pci/drm/include/linux/gfp.h
===
RCS file: /cvs/src/sys/dev/pci/drm/include/linux/gfp.h,v
retrieving revision 1.1
diff -u -p -r1.1 gfp.h
--- dev/pci/drm/include/linux/gfp.h 14 Apr 2019 10:14:53 -  1.1
+++ dev/pci/drm/include/linux/gfp.h 6 Jun 2019 21:19:21 -
@@ -7,24 +7,25 @@
 #include 
 #include 
 
-#define GFP_ATOMIC M_NOWAIT
-#define GFP_NOWAIT M_NOWAIT
-#define GFP_KERNEL (M_WAITOK | M_CANFAIL)
-#define GFP_USER   (M_WAITOK | M_CANFAIL)
-#define GFP_TEMPORARY  (M_WAITOK | M_CANFAIL)
-#define GFP_HIGHUSER   0
-#define GFP_DMA32  0
-#define __GFP_NOWARN   0
-#define __GFP_NORETRY  0
-#define __GFP_ZERO M_ZERO
+#define __GFP_ZERO M_ZERO
+#define __GFP_DMA320x8000
+#define __GFP_NOWARN   0
+#define __GFP_NORETRY  0
 #define __GFP_RETRY_MAYFAIL0
 #define __GFP_MOVABLE  0
 #define __GFP_COMP 0
-#define GFP_TRANSHUGE_LIGHT0
 #define __GFP_KSWAPD_RECLAIM   0
 #define __GFP_HIGHMEM  0
 #define __GFP_RECLAIMABLE  0
-#define __GFP_DMA320
+
+#define GFP_ATOMIC M_NOWAIT
+#define GFP_NOWAIT M_NOWAIT
+#define GFP_KERNEL (M_WAITOK | M_CANFAIL)
+#define GFP_USER   (M_WAITOK | M_CANFAIL)
+#define GFP_TEMPORARY  (M_WAITOK | M_CANFAIL)
+#define GFP_HIGHUSER   0
+#define GFP_DMA32  __GFP_DMA32
+#define GFP_TRANSHUGE_LIGHT0
 
 static inline bool
 gfpflags_allow_blocking(const unsigned int flags)
Index: dev/pci/drm/ttm/ttm_tt.c

Re: monotonic clock going backwards?

2019-06-06 Thread Mark Kettenis
> From: David Hill 
> Date: Thu, 6 Jun 2019 09:55:00 -0400
> 
> On 6/6/19 8:49 AM, Mark Kettenis wrote:
> >> From: David Hill 
> >> Date: Thu, 6 Jun 2019 08:22:06 -0400
> >>
> >> On 6/6/19 6:56 AM, Scott Cheloha wrote:
> >>> On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote:
>  Hello -
> 
>  I noticed some negative roundtrip times when running traceroute, which 
>  uses
>  the monotonic clock to calculate the RTT.
> 
>  If I run the following code, it eventually bombs.?? It bombs quick if I
>  launch Firefox.
> 
>  timespeccmp failed
>  tp1 s:103780 n:63101042
>  tp2 s:103779 n:761117849
> >>> please share your dmesg and the output of
> >>>
> >>>   sysctl kern.clockrate kern.timecounter
> >>>
> >>> Could you also share the output when you run the following code?
> >>> With and without firefox running?  Curious about how frequently
> >>> this happens.
> >>>
> >>> #include 
> >>>
> >>> #include 
> >>> #include 
> >>> #include 
> >>>
> >>> int
> >>> main(void)
> >>> {
> >>>   struct timespec t0, t1, timeout;
> >>>
> >>>   timeout.tv_sec = 0;
> >>>   timeout.tv_nsec = 10;
> >>>
> >>>   for (;;) {
> >>>   clock_gettime(CLOCK_MONOTONIC, );
> >>>   if (nanosleep(, NULL) == -1)
> >>>   err(1, "nanosleep");
> >>>   clock_gettime(CLOCK_MONOTONIC, );
> >>>   if (timespeccmp(, , >)) {
> >>>   printf("%lld.%09ld precedes %lld.%09ld\n",
> >>>   t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec);
> >>>   }
> >>>   }
> >>>
> >>>   return 0;
> >>> }
> >>>
> >> $ sysctl kern.clockrate kern.timecounter
> >> kern.clockrate=tick = 1, tickadj = 40, hz = 100, profhz = 100,
> >> stathz = 100
> >> kern.timecounter.tick=1
> >> kern.timecounter.timestepwarnings=0
> >> kern.timecounter.hardware=tsc
> >> kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000) acpitimer0(1000)
> >>
> >> 146259.148543164 precedes 146259.450584680
> >> 146259.488592813 precedes 146259.790556528
> >> 146259.808566236 precedes 146260.110528202
> >> 146261.148476141 precedes 146261.450398822
> >> 146262.888279505 precedes 146263.190240715
> >> 146263.887431808 precedes 146264.189435801
> >> 146274.487159451 precedes 146274.789148187
> >> 146278.186820816 precedes 146278.488803756
> >> 146279.546680181 precedes 146279.848674138
> >> 146284.326305557 precedes 146284.628224696
> >> 146285.426154861 precedes 146285.728066901
> >> 146287.885921929 precedes 146288.187826688
> >> 146289.985763711 precedes 146290.287731233
> >> 146290.985659041 precedes 146291.287640302
> >> 146293.985348615 precedes 146294.287327519
> >> 146295.465186742 precedes 146295.767218516
> >> 146295.885156426 precedes 146296.187140717
> >> 146298.544907956 precedes 146298.846882759
> >> 146298.644894308 precedes 146298.946889142
> > Does this also happen with
> >
> > kern.timecounter.hardware=acpihpet
> >
> > ?
> 
> Ran the code with kern.timecounter.hardware=acpihpet0 for 10 minutes 
> without any output.
> 
> With the code still running, I switched to tsc and starting seeing output:
> 
> 151944.345840447 precedes 151944.647753895
> 151944.525774690 precedes 151944.827681380
> 151946.405553296 precedes 151946.707533012
> 
> The output stops as soon as I switch back to acpihpet0.
> 
> I will add that to my /etc/sysctl.conf.
> 
> Thank you.

So the TSCs on the cores of your brand new Intel Kaby Lake R processor
are out of sync.  That is the final drop.  I disabled the code that
promotes the TSC as a high-quality timecounter on MULTIPROCESSOR
kernels.  It simply doesn't work on a significant fraction of machines
and causes all kinds of weird problems.

Scott, any change to turn that bit of code into a regress test?



Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority

2019-06-06 Thread Martin Pieuchot
On 02/06/19(Sun) 16:41, Martin Pieuchot wrote:
> On 01/06/19(Sat) 18:55, Martin Pieuchot wrote:
> > Diff below exists mainly for documentation and test purposes.  If
> > you're not interested about how to break the scheduler internals in
> > pieces, don't read further and go straight to testing!
> > 
> > - First change is to stop calling tsleep(9) at PUSER.  That makes
> >   it clear that all "sleeping priorities" are smaller than PUSER.
> >   That's important to understand for the diff below.  `p_priority'
> >   is currently a placeholder for the "sleeping priority" and the
> >   "runnqueue priority".  Both fields are separated by this diff.
> > 
> > - When a thread goes to sleep, the priority argument of tsleep(9) is
> >   now recorded in `p_slpprio'.  This argument can be considered as part
> >   of the sleep queue.  Its purpose is to place the thread into a higher
> >   runqueue when awoken.
> > 
> > - Currently, for stopped threads, `p_priority' correspond to `p_usrpri'. 
> >   So setrunnable() has been untangled to place SSTOP and SSLEEP threads
> >   in the preferred queue without having to use `p_priority'.  Note that
> >   `p_usrpri' is still recalculated *after* having called setrunqueue().
> >   This is currently fine because setrunnable() is called with SCHED_LOCK() 
> >   but it will be racy when we'll split it.
> > 
> > - A new field, `p_runprio' has been introduced.  It should be considered
> >   as part of the per-CPU runqueues.  It indicates where a current thread
> >   is placed.
> > 
> > - `spc_curpriority' is now updated at every context-switch.  That means
> >need_resched() won't be called after comparing an out-of-date value.
> >At the same time, `p_usrpri' is initialized to the highest possible
> >value for idle threads.
> > 
> > - resched_proc() was calling need_resched() in the following conditions:
> >- If the SONPROC thread has a higher priority that the current
> >  running thread (itself).
> >- Twice in setrunnable() when we know that p_priority <= p_usrpri.
> >- If schedcpu() considered that a thread, after updating its prio,
> >  should preempt the one running on the CPU pointed by `p_cpu'. 
> > 
> >   The diff below simplify all of that by calling need_resched() when:
> >- A thread is inserted in a CPU runqueue at a higher priority than
> >  the one SONPROC.
> >- schedcpu() decides that a thread in SRUN state should preempt the
> >  one SONPROC.
> > 
> > - `p_estcpu' `p_usrpri' and `p_slptime' which represent the "priority"
> >   of a thread are now updated while holding a per-thread mutex.  As a
> >   result schedclock() and donice() no longer takes the SCHED_LOCK(),
> >   and schedcpu() almost never take it.
> > 
> > - With this diff top(1) and ps(1) will report the "real" `p_usrpi' value
> >   when displaying priorities.  This is helpful to understand what's
> >   happening:
> > 
> > load averages:  0.99,  0.56,  0.25   two.lab.grenadille.net 
> > 23:42:10
> > 70 threads: 68 idle, 2 on processorup  
> > 0:09
> > CPU0:  0.0% user,  0.0% nice, 51.0% sys,  2.0% spin,  0.0% intr, 47.1% idle
> > CPU1:  2.0% user,  0.0% nice, 51.0% sys,  3.9% spin,  0.0% intr, 43.1% idle
> > Memory: Real: 47M/1005M act/tot Free: 2937M Cache: 812M Swap: 0K/4323M
> > 
> >   PID  TID PRI NICE  SIZE   RES STATE WAIT  TIMECPU COMMAND
> > 81000   145101  7200K 1664K sleep/1   bored 1:15 36.96% softnet
> > 47133   244097  730 2984K 4408K sleep/1   netio 1:06 35.06% cvs 
> > 64749   522184  660  176K  148K onproc/1  - 0:55 28.81% nfsd
> > 21615   602473 12700K 1664K sleep/0   - 7:22  0.00% idle0  
> > 12413   606242 12700K 1664K sleep/1   - 7:08  0.00% idle1
> > 85778   338258  500 4936K 7308K idle  select0:10  0.00% ssh  
> > 22771   575513  500  176K  148K sleep/0   nfsd  0:02  0.00% nfsd 
> > 
> > 
> > 
> > - The removal of `p_priority' and the change that makes mi_switch()
> >   always update `spc_curpriority' might introduce some changes in
> >   behavior, especially with kernel threads that were not going through
> >   tsleep(9).  We currently have some situations where the priority of
> >   the running thread isn't correctly reflected.  This diff changes that
> >   which means we should be able to better understand where the problems
> >   are.
> > 
> > I'd be interested in comments/tests/reviews before continuing in this
> > direction.  Note that at least part of this diff are required to split
> > the accounting apart from the SCHED_LOCK() as well.
> > 
> > I'll also work on exporting scheduler statistics unless somebody wants
> > to beat me :)
> 
> Updated diff to use IPL_SCHED and rebased to apply on top of -current :) 

Updated diff that fixes a pagefault reported by sthen@.

Index: arch/amd64/amd64/genassym.cf
===
RCS file: 

Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority

2019-06-06 Thread Mike Larkin
On Thu, Jun 06, 2019 at 02:55:35PM +0100, Stuart Henderson wrote:
> I'm testing the "pump my sched" and read/write unlock diffs and ran into
> the panic below. Seems more likely that it would be connected with the
> sched diff rather than anything else. I'll build a WITNESS kernel and
> see if I can get more details if it hapens again.
> 

Funny thing, I ran into the same panic last night after running this diff
without issue for several days. Mine didn't include a stack trace though.

> ddb{1}> show panic
> kernel diagnostic assertion "__mp_lock_held(_lock, curcpu()) == 0" 
> failed
> : file "/sys/kern/kern_lock.c", line 63
> ddb{1}> tr
> db_enter() at db_enter+0x10
> panic() at panic+0x128
> __assert(81aebffd,81ac7a49,3f,81b0e315) at 
> __assert+0x2
> e
> _kernel_lock() at _kernel_lock+0xf6
> pageflttrap() at pageflttrap+0x75
> kerntrap(800033ee0ca0) at kerntrap+0x91
> alltraps_kern(6,380,fffb,2,80002201aff0,e0) at alltraps_kern+0x7b
> setrunqueue(0,8000342b2ce8,e0) at setrunqueue+0xbc
> setrunnable(8000342b2ce8,e0) at setrunnable+0xa5
> ptsignal(8000342b2ce8,13,0) at ptsignal+0x3c4
> sys_kill(800033c2fcc0,800033ee0ec0,800033ee0f20) at sys_kill+0x1c5
> syscall(800033ee0f90) at syscall+0x389
> Xsyscall(6,7a,0,7a,13,7f7e3e18) at Xsyscall+0x128
> end of kernel
> end trace frame: 0x7f7e3cb0, count: -13
> ddb{1}> ps
>PID TID   PPIDUID  S   FLAGS  WAIT  COMMAND
>  89165  411452   1971667  30x1000b2  poll  ping
>   1971   11232  86869667  30x82  piperdcheck_ping
>  66051  380473  92153667  30x1000b2  poll  ping
>  92153  480496  86869667  30x82  piperdcheck_ping
>   7608  110605   8332667  30x1000b2  poll  ping
>   8332  284560  86869667  30x82  piperdcheck_ping
>  40097  336365  35317667  30x1000b2  poll  ping
>  35317  385677  86869667  30x82  piperdcheck_ping
>  33714  102547  21689667  30x1000b2  poll  ping
>  21689  210571  86869667  30x82  piperdcheck_ping
>  72142  347023  62694667  30x1000b2  poll  ping
>  62694  439660  86869667  30x82  piperdcheck_ping
>  87347  447624  76730   1000  4   0x8000430xscreensaver
>  37569  319522  76730   1000  20x800402piecewise
>  11841  481452  18848667  30x1000b2  poll  ping
>  18848  111230  86869667  30x82  piperdcheck_ping
>  69292  189646  44902667  30x1000b2  poll  ping
>  44902   21064  86869667  30x82  piperdcheck_ping
>  67905  363250   1761507  30x92  kqreadpickup
>  86709  146945  91951   1000  30x92  kqreadimap
>  36596  227858  91951666  30x92  kqreadimap-login
>  90723  180281  91951   1000  30x92  kqreadimap
>  42867  129499  91951   1000  30x92  kqreadimap
>  52963  203919  91951   1000  30x92  kqreadimap
>  61599  167915  91951666  30x92  kqreadimap-login
>  54450  154426  91951666  30x92  kqreadimap-login
>  54061  103553  91951666  30x92  kqreadimap-login
>  39422  362611  85872715  30x90  netconperl
>  49809  260623  85872715  30x90  lockf perl
>   3824   92473  85872715  30x90  lockf perl
>  30990  398152  85872715  30x90  lockf perl
>   3327  264795  85872715  30x90  lockf perl
>  24406  217143  85872715  30x90  lockf perl
>  52871  404585  85872715  30x90  lockf perl
>  58220  118881  85872715  30x90  lockf perl
>  32898  307331  85872715  30x90  lockf perl
>  50203  396987  85872715  30x90  lockf perl
>  93024   24785  1   1000  30x100080  selectssh
>  22261  482889  52840   1000  30x100083  poll  ssh
>  52840  404967  61470   1000  30x10008b  pause ksh
>  35796  132597  61470   1000  30xb0  netio urxvt
>  61470  485028  72107   1000  30xb2  selecturxvt
>  72107  196192  1   1000  30x10008a  pause sh
>  70238  396933  1   1000  30x100080  selectssh
>  41840  302658  10362   1000  30x100083  poll  ssh
>  89638  411799  91951   1000  30x92  kqreadimap
>  49250  507579  81821   1000  30x100083  ttyin ksh
>  37608  523403  91951666  30x92  kqreadimap-login
>  75552   67606  33096   1000  30x100083  poll  mutt
>  33096  492099  81821   1000  30x10008b  pause ksh
>  81821  270355  1   1000  30x100080  kqreadtmux
>  24522  341281  91861   1000  30x100083  kqreadtmux
>  

Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority

2019-06-06 Thread Stuart Henderson
On 2019/06/06 14:55, Stuart Henderson wrote:
> I'm testing the "pump my sched" and read/write unlock diffs and ran into
> the panic below. Seems more likely that it would be connected with the
> sched diff rather than anything else. I'll build a WITNESS kernel and
> see if I can get more details if it hapens again.

Easily repeatable: Run xscreensaver. Let it lock. Try to unlock.

login: panic: kernel diagnostic assertion "__mp_lock_held(_lock, 
curcpu()) == 0" failed: file "/src/cvs-openbsd/sys/kern/kern_lock.c", line 63
Stopped at  db_enter+0x10:  popq%rbp
TIDPIDUID PRFLAGS PFLAGS  CPU  COMMAND
   6043  83667   1000 0x2  01  firefox
 320664  83667   1000 0x2  0x4802  firefox
 523640  70812   1000 0x2  0x4003  firefox
*307581  55620   10000x32  00K xscreensaver
db_enter() at db_enter+0x10
panic() at panic+0x128
__assert(81af6d92,81afc585,3f,81b1a406) at __assert+0x2
e
__mp_lock_held(811362e0,800033cfe5e0) at __mp_lock_held
pageflttrap() at pageflttrap+0x78
kerntrap(800033cfe6b0) at kerntrap+0x91
alltraps_kern(6,300,fffb,2,800022009ff0,c0) at alltraps_kern+0x7b
setrunqueue(0,8000342572a8,c0) at setrunqueue+0xbc
setrunnable(8000342572a8,c0) at setrunnable+0xa5
ptsignal(8000342572a8,13,0) at ptsignal+0x3c4
sys_kill(800033c532a8,800033cfe8d0,800033cfe930) at sys_kill+0x1c5
syscall(800033cfe9a0) at syscall+0x399
Xsyscall(6,7a,0,7a,13,7f7e8cc8) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7e8b60, count: 2
https://www.openbsd.org/ddb.html describes the minimum info required in bug
reports.  Insufficient info makes it difficult to find and fix bugs.
ddb{0}> sh witness
Sleep locks:
_fpriv->lock (type: rwlock, depth: 0)_priv->fbs_lock (type: rwlock, d
epth: 0)futex (type: rwlock, depth: 0) _lock (type: kernel_lock, depth: 6
) >lock (type: rwlock, depth: 5)  _lock (type: kernel_lock, depth: 6
) -- (already displayed)
 >pr_lock (type: rwlock, depth: 1)  _lock (type: kernel_lock, depth:
 6) -- (already displayed)
swplk (type: rwlock, depth: 0) >lock (type: rwlock, depth: 5) -- (already d
isplayed)
 >i_lock (type: rrwlock, depth: 2)  _lock (type: kernel_lock, depth:
 6) -- (already displayed)
  >lock (type: rwlock, depth: 5) -- (already displayed)
  >dk_lock (type: rwlock, depth: 3)   _lock (type: kernel_lock, d
epth: 6) -- (already displayed)
  _mtx (type: rwlock, depth: 3)   >dh_mtx (type: rwlock, depth: 4
)  lockflk (type: rwlock, depth: 3)(type: rwlock, depth: 4)   
->pr_lock (type: rwlock, depth: 4)   >pr_lock (type: rwlock, depth: 4)  
->dh_mtx (type: rwlock, depth: 4) -- (already displayed)
  >uv_lock (type: rwlock, depth: 3)   >uv_lock (type: rwlock, depth: 4)
  >uv_lock (type: rwlock, depth: 4) -- (already displayed)
sysctllk (type: rwlock, depth: 0) _lock (type: kernel_lock, depth: 6) --
 (already displayed)
 >lock (type: rwlock, depth: 5) -- (already displayed)
 >fd_fd.fd_lock (type: rwlock, depth: 1)  _lock (type: kernel_lo
ck, depth: 6) -- (already displayed)
  >lock (type: rwlock, depth: 5) -- (already displayed)
  netlock (type: rwlock, depth: 2)   _lock (type: kernel_lock, depth: 6)
 -- (already displayed)
   pools (type: rwlock, depth: 3)_lock (type: kernel_lock, depth: 6)
 -- (already displayed)
   >lock (type: rwlock, depth: 5) -- (already displayed)
   >ar_lock (type: rwlock, depth: 3)   vlantag (type: rwlock, depth: 3)  lo
ckflk (type: rwlock, depth: 3) -- (already displayed)
  >i_lock (type: rrwlock, depth: 2) -- (already displayed)
  vfs_stall (type: rwlock, depth: 2)   _lock (type: kernel_lock, depth: 6
) -- (already displayed)
  ptarrlk (type: rwlock, depth: 2) netlock (type: rwlock, depth: 2) -- (already
 displayed)
 tc_lock (type: rwlock, depth: 1) >i_lock (type: rrwlock, depth: 2) -- (alr
eady displayed)
 vfs_stall (type: rwlock, depth: 2) -- (already displayed)
 sysctldlk (type: rwlock, depth: 1)>hdcp_mutex (type: rwlock, depth:
 0)>lock (type: rwlock, depth: 0)>mutex (type: rwlock, depth: 0
) >mode_config.idr_mutex (type: rwlock, depth: 3)_priv->dpll_lock (typ
e: rwlock, depth: 0)taskq (type: rwlock, depth: 0) >struct_mutex (type: rw
lock, depth: 3)  >lock (type: rwlock, depth: 5) -- (already displayed)
   (type: rwlock, depth: 4) -- (already displayed)
  >vm_lock (type: rwlock, depth: 5)  _priv->pcu_lock (type: rwlock, de
pth: 4)   _priv->gt_pm.rps.power.mutex (type: rwlock, depth: 5)  _dom
ains->lock (type: rwlock, depth: 4)  _priv->mm.stolen_lock (type: rwlock, d
epth: 4)  >lock (type: rwlock, depth: 4)  >vm_lock (type: rwlock, de
pth: 4)  >mm.lock (type: rwlock, depth: 4)   >lock (type: rwlock, dep
th: 5) -- (already displayed)
   >vm_lock (type: rwlock, depth: 5) -- (already displayed)
  >mm.get_page.lock (type: rwlock, depth: 4) >vm_lock (type: rwlock, d
epth: 5) -- (already displayed)
 >mm.lock (type: rwlock, depth: 4) -- (already displayed)
_priv->av_mutex (type: rwlock, depth: 

RFC 8555 for acme-client(1)

2019-06-06 Thread Florian Obser
This switches acme-client(1) from draft-03 to the final standard.

Let's Encrypt claims - and this seems to be true in my testing - that
accounts created on the v01 api (the draf one) work on the v02 api
(rfc).

This drops support for v01, thinks got shuffled around too much to
support both. acme-client(1) complains fairly loudly when it's pointed
at an old server:

acme-client: https://acme-staging.api.letsencrypt.org/directory: bad CA 
paths

To test it switch the api url:
-   api url "https://acme-v01.api.letsencrypt.org/directory;
+   api url "https://acme-v02.api.letsencrypt.org/directory;

Tests, Comments, OKs?

There is certainly more stuff to do here, but it's already quite big.

diff --git etc/examples/acme-client.conf etc/examples/acme-client.conf
index 53bc65e17f3..94f1b3b5c72 100644
--- etc/examples/acme-client.conf
+++ etc/examples/acme-client.conf
@@ -2,12 +2,12 @@
 # $OpenBSD: acme-client.conf,v 1.1 2019/01/08 07:14:10 florian Exp $
 #
 authority letsencrypt {
-   api url "https://acme-v01.api.letsencrypt.org/directory;
+   api url "https://acme-v02.api.letsencrypt.org/directory;
account key "/etc/acme/letsencrypt-privkey.pem"
 }
 
 authority letsencrypt-staging {
-   api url "https://acme-staging.api.letsencrypt.org/directory;
+   api url "https://acme-staging-v02.api.letsencrypt.org/directory;
account key "/etc/acme/letsencrypt-staging-privkey.pem"
 }
 
diff --git usr.sbin/acme-client/acctproc.c usr.sbin/acme-client/acctproc.c
index 2b71fb2ec79..733cf8a0032 100644
--- usr.sbin/acme-client/acctproc.c
+++ usr.sbin/acme-client/acctproc.c
@@ -154,18 +154,16 @@ out:
 }
 
 static int
-op_sign_rsa(char **head, char **prot, EVP_PKEY *pkey, const char *nonce)
+op_sign_rsa(char **prot, EVP_PKEY *pkey, const char *nonce, const char *url)
 {
char*exp = NULL, *mod = NULL;
int rc = 0;
RSA *r;
 
-   *head = NULL;
*prot = NULL;
 
/*
 * First, extract relevant portions of our private key.
-* Then construct the public header.
 * Finally, format the header combined with the nonce.
 */
 
@@ -175,9 +173,7 @@ op_sign_rsa(char **head, char **prot, EVP_PKEY *pkey, const 
char *nonce)
warnx("bn2string");
else if ((exp = bn2string(r->e)) == NULL)
warnx("bn2string");
-   else if ((*head = json_fmt_header_rsa(exp, mod)) == NULL)
-   warnx("json_fmt_header_rsa");
-   else if ((*prot = json_fmt_protected_rsa(exp, mod, nonce)) == NULL)
+   else if ((*prot = json_fmt_protected_rsa(exp, mod, nonce, url)) == NULL)
warnx("json_fmt_protected_rsa");
else
rc = 1;
@@ -192,11 +188,12 @@ op_sign_rsa(char **head, char **prot, EVP_PKEY *pkey, 
const char *nonce)
  * This requires the sender ("fd") to provide the payload and a nonce.
  */
 static int
-op_sign(int fd, EVP_PKEY *pkey)
+op_sign(int fd, EVP_PKEY *pkey, enum acctop op)
 {
char*nonce = NULL, *pay = NULL, *pay64 = NULL;
-   char*prot = NULL, *prot64 = NULL, *head = NULL;
+   char*prot = NULL, *prot64 = NULL;
char*sign = NULL, *dig64 = NULL, *fin = NULL;
+   char*url = NULL, *kid = NULL;
unsigned char   *dig = NULL;
EVP_MD_CTX  *ctx = NULL;
int  cc, rc = 0;
@@ -208,6 +205,12 @@ op_sign(int fd, EVP_PKEY *pkey)
goto out;
else if ((nonce = readstr(fd, COMM_NONCE)) == NULL)
goto out;
+   else if ((url = readstr(fd, COMM_URL)) == NULL)
+   goto out;
+
+   if (op == ACCT_KID_SIGN)
+   if ((kid = readstr(fd, COMM_KID)) == NULL)
+   goto out;
 
/* Base64-encode the payload. */
 
@@ -216,14 +219,21 @@ op_sign(int fd, EVP_PKEY *pkey)
goto out;
}
 
-   switch (EVP_PKEY_type(pkey->type)) {
-   case EVP_PKEY_RSA:
-   if (!op_sign_rsa(, , pkey, nonce))
+   if (op == ACCT_KID_SIGN) {
+   if ((prot = json_fmt_protected_kid(kid, nonce, url)) == NULL) {
+   warnx("json_fmt_protected_kid");
goto out;
-   break;
-   default:
-   warnx("EVP_PKEY_type");
-   goto out;
+   }
+   } else {
+   switch (EVP_PKEY_type(pkey->type)) {
+   case EVP_PKEY_RSA:
+   if (!op_sign_rsa(, pkey, nonce, url))
+   goto out;
+   break;
+   default:
+   warnx("EVP_PKEY_type");
+   goto out;
+   }
}
 
/* The header combined with the nonce, base64. */
@@ -275,7 +285,7 @@ op_sign(int fd, EVP_PKEY *pkey)
 * when we next enter the read loop).
 */
 
-   if ((fin = json_fmt_signed(head, prot64, pay64, dig64)) == NULL) 

Re: [patch] push the KERNEL_LOCK deeper on read(2) and write(2)

2019-06-06 Thread Solène Rapenne

Le 2019-06-05 12:06, Mark Kettenis a écrit :

Date: Wed, 5 Jun 2019 10:16:25 +0200
From: Sebastien Marie 

Hi,

I would like to have feedback and testing on this diff. The initial 
work

was done by ian@.


Don't forget to run "make syscalls" in sys/kern when building your own
kernel with this diff!



no problem so far with this diff and the scheduler one.



Re: Pump my sched: fewer SCHED_LOCK() & kill p_priority

2019-06-06 Thread Stuart Henderson
I'm testing the "pump my sched" and read/write unlock diffs and ran into
the panic below. Seems more likely that it would be connected with the
sched diff rather than anything else. I'll build a WITNESS kernel and
see if I can get more details if it hapens again.

ddb{1}> show panic
kernel diagnostic assertion "__mp_lock_held(_lock, curcpu()) == 0" failed
: file "/sys/kern/kern_lock.c", line 63
ddb{1}> tr
db_enter() at db_enter+0x10
panic() at panic+0x128
__assert(81aebffd,81ac7a49,3f,81b0e315) at __assert+0x2
e
_kernel_lock() at _kernel_lock+0xf6
pageflttrap() at pageflttrap+0x75
kerntrap(800033ee0ca0) at kerntrap+0x91
alltraps_kern(6,380,fffb,2,80002201aff0,e0) at alltraps_kern+0x7b
setrunqueue(0,8000342b2ce8,e0) at setrunqueue+0xbc
setrunnable(8000342b2ce8,e0) at setrunnable+0xa5
ptsignal(8000342b2ce8,13,0) at ptsignal+0x3c4
sys_kill(800033c2fcc0,800033ee0ec0,800033ee0f20) at sys_kill+0x1c5
syscall(800033ee0f90) at syscall+0x389
Xsyscall(6,7a,0,7a,13,7f7e3e18) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7e3cb0, count: -13
ddb{1}> ps
   PID TID   PPIDUID  S   FLAGS  WAIT  COMMAND
 89165  411452   1971667  30x1000b2  poll  ping
  1971   11232  86869667  30x82  piperdcheck_ping
 66051  380473  92153667  30x1000b2  poll  ping
 92153  480496  86869667  30x82  piperdcheck_ping
  7608  110605   8332667  30x1000b2  poll  ping
  8332  284560  86869667  30x82  piperdcheck_ping
 40097  336365  35317667  30x1000b2  poll  ping
 35317  385677  86869667  30x82  piperdcheck_ping
 33714  102547  21689667  30x1000b2  poll  ping
 21689  210571  86869667  30x82  piperdcheck_ping
 72142  347023  62694667  30x1000b2  poll  ping
 62694  439660  86869667  30x82  piperdcheck_ping
 87347  447624  76730   1000  4   0x8000430xscreensaver
 37569  319522  76730   1000  20x800402piecewise
 11841  481452  18848667  30x1000b2  poll  ping
 18848  111230  86869667  30x82  piperdcheck_ping
 69292  189646  44902667  30x1000b2  poll  ping
 44902   21064  86869667  30x82  piperdcheck_ping
 67905  363250   1761507  30x92  kqreadpickup
 86709  146945  91951   1000  30x92  kqreadimap
 36596  227858  91951666  30x92  kqreadimap-login
 90723  180281  91951   1000  30x92  kqreadimap
 42867  129499  91951   1000  30x92  kqreadimap
 52963  203919  91951   1000  30x92  kqreadimap
 61599  167915  91951666  30x92  kqreadimap-login
 54450  154426  91951666  30x92  kqreadimap-login
 54061  103553  91951666  30x92  kqreadimap-login
 39422  362611  85872715  30x90  netconperl
 49809  260623  85872715  30x90  lockf perl
  3824   92473  85872715  30x90  lockf perl
 30990  398152  85872715  30x90  lockf perl
  3327  264795  85872715  30x90  lockf perl
 24406  217143  85872715  30x90  lockf perl
 52871  404585  85872715  30x90  lockf perl
 58220  118881  85872715  30x90  lockf perl
 32898  307331  85872715  30x90  lockf perl
 50203  396987  85872715  30x90  lockf perl
 93024   24785  1   1000  30x100080  selectssh
 22261  482889  52840   1000  30x100083  poll  ssh
 52840  404967  61470   1000  30x10008b  pause ksh
 35796  132597  61470   1000  30xb0  netio urxvt
 61470  485028  72107   1000  30xb2  selecturxvt
 72107  196192  1   1000  30x10008a  pause sh
 70238  396933  1   1000  30x100080  selectssh
 41840  302658  10362   1000  30x100083  poll  ssh
 89638  411799  91951   1000  30x92  kqreadimap
 49250  507579  81821   1000  30x100083  ttyin ksh
 37608  523403  91951666  30x92  kqreadimap-login
 75552   67606  33096   1000  30x100083  poll  mutt
 33096  492099  81821   1000  30x10008b  pause ksh
 81821  270355  1   1000  30x100080  kqreadtmux
 24522  341281  91861   1000  30x100083  kqreadtmux
 23218  441369  78842   1000  30x82  poll  firefox
 23218  430421  78842   1000  3   0x482  kqreadfirefox
 23218  460884  78842   1000  3   0x482  fsleepfirefox
 23218   17619  78842   1000  3   0x482  fsleepfirefox
 23218  138796  78842   1000  3   0x482  fsleepfirefox
 23218  341276  78842   1000  3   0x482  

Re: monotonic clock going backwards?

2019-06-06 Thread David Hill



On 6/6/19 8:49 AM, Mark Kettenis wrote:

From: David Hill 
Date: Thu, 6 Jun 2019 08:22:06 -0400

On 6/6/19 6:56 AM, Scott Cheloha wrote:

On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote:

Hello -

I noticed some negative roundtrip times when running traceroute, which uses
the monotonic clock to calculate the RTT.

If I run the following code, it eventually bombs.?? It bombs quick if I
launch Firefox.

timespeccmp failed
tp1 s:103780 n:63101042
tp2 s:103779 n:761117849

please share your dmesg and the output of

sysctl kern.clockrate kern.timecounter

Could you also share the output when you run the following code?
With and without firefox running?  Curious about how frequently
this happens.

#include 

#include 
#include 
#include 

int
main(void)
{
struct timespec t0, t1, timeout;

timeout.tv_sec = 0;
timeout.tv_nsec = 10;

for (;;) {
clock_gettime(CLOCK_MONOTONIC, );
if (nanosleep(, NULL) == -1)
err(1, "nanosleep");
clock_gettime(CLOCK_MONOTONIC, );
if (timespeccmp(, , >)) {
printf("%lld.%09ld precedes %lld.%09ld\n",
t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec);
}
}

return 0;
}


$ sysctl kern.clockrate kern.timecounter
kern.clockrate=tick = 1, tickadj = 40, hz = 100, profhz = 100,
stathz = 100
kern.timecounter.tick=1
kern.timecounter.timestepwarnings=0
kern.timecounter.hardware=tsc
kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000) acpitimer0(1000)

146259.148543164 precedes 146259.450584680
146259.488592813 precedes 146259.790556528
146259.808566236 precedes 146260.110528202
146261.148476141 precedes 146261.450398822
146262.888279505 precedes 146263.190240715
146263.887431808 precedes 146264.189435801
146274.487159451 precedes 146274.789148187
146278.186820816 precedes 146278.488803756
146279.546680181 precedes 146279.848674138
146284.326305557 precedes 146284.628224696
146285.426154861 precedes 146285.728066901
146287.885921929 precedes 146288.187826688
146289.985763711 precedes 146290.287731233
146290.985659041 precedes 146291.287640302
146293.985348615 precedes 146294.287327519
146295.465186742 precedes 146295.767218516
146295.885156426 precedes 146296.187140717
146298.544907956 precedes 146298.846882759
146298.644894308 precedes 146298.946889142

Does this also happen with

kern.timecounter.hardware=acpihpet

?


Ran the code with kern.timecounter.hardware=acpihpet0 for 10 minutes 
without any output.


With the code still running, I switched to tsc and starting seeing output:

151944.345840447 precedes 151944.647753895
151944.525774690 precedes 151944.827681380
151946.405553296 precedes 151946.707533012

The output stops as soon as I switch back to acpihpet0.

I will add that to my /etc/sysctl.conf.

Thank you.





OpenBSD 6.5-current (GENERIC.MP) #27: Tue Jun  4 15:36:14 EDT 2019
dh...@liver.mindcry.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8418242560 (8028MB)
avail mem = 8152993792 (7775MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x9a565000 (49 entries)
bios0: vendor American Megatrends Inc. version "X420UA.204" date 11/16/2018
bios0: ASUSTeK COMPUTER INC. VivoBook_ASUSLaptop X420UA
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT ECDT MCFG SSDT SSDT SSDT MSDM
SSDT HPET SSDT UEFI SSDT LPIT SSDT SSDT SSDT SSDT DBGP DBG2 DMAR NHLT
BGRT TPM2 WSMT
acpi0: wakeup devices PXSX(S4) RP06(S4) GLAN(S4) XHC_(S3) XDCI(S4) HDAS(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3393.27 MHz, 06-8e-0a
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S
SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI
1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 23MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3392.09 MHz, 06-8e-0a
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S

Re: monotonic clock going backwards?

2019-06-06 Thread Mark Kettenis
> From: David Hill 
> Date: Thu, 6 Jun 2019 08:22:06 -0400
> 
> On 6/6/19 6:56 AM, Scott Cheloha wrote:
> > On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote:
> >> Hello -
> >>
> >> I noticed some negative roundtrip times when running traceroute, which uses
> >> the monotonic clock to calculate the RTT.
> >>
> >> If I run the following code, it eventually bombs.?? It bombs quick if I
> >> launch Firefox.
> >>
> >> timespeccmp failed
> >> tp1 s:103780 n:63101042
> >> tp2 s:103779 n:761117849
> > please share your dmesg and the output of
> >
> > sysctl kern.clockrate kern.timecounter
> >
> > Could you also share the output when you run the following code?
> > With and without firefox running?  Curious about how frequently
> > this happens.
> >
> > #include 
> >
> > #include 
> > #include 
> > #include 
> >
> > int
> > main(void)
> > {
> > struct timespec t0, t1, timeout;
> >
> > timeout.tv_sec = 0;
> > timeout.tv_nsec = 10;
> >
> > for (;;) {
> > clock_gettime(CLOCK_MONOTONIC, );
> > if (nanosleep(, NULL) == -1)
> > err(1, "nanosleep");
> > clock_gettime(CLOCK_MONOTONIC, );
> > if (timespeccmp(, , >)) {
> > printf("%lld.%09ld precedes %lld.%09ld\n",
> > t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec);
> > }
> > }
> >
> > return 0;
> > }
> >
> 
> $ sysctl kern.clockrate kern.timecounter
> kern.clockrate=tick = 1, tickadj = 40, hz = 100, profhz = 100, 
> stathz = 100
> kern.timecounter.tick=1
> kern.timecounter.timestepwarnings=0
> kern.timecounter.hardware=tsc
> kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000) acpitimer0(1000)
> 
> 146259.148543164 precedes 146259.450584680
> 146259.488592813 precedes 146259.790556528
> 146259.808566236 precedes 146260.110528202
> 146261.148476141 precedes 146261.450398822
> 146262.888279505 precedes 146263.190240715
> 146263.887431808 precedes 146264.189435801
> 146274.487159451 precedes 146274.789148187
> 146278.186820816 precedes 146278.488803756
> 146279.546680181 precedes 146279.848674138
> 146284.326305557 precedes 146284.628224696
> 146285.426154861 precedes 146285.728066901
> 146287.885921929 precedes 146288.187826688
> 146289.985763711 precedes 146290.287731233
> 146290.985659041 precedes 146291.287640302
> 146293.985348615 precedes 146294.287327519
> 146295.465186742 precedes 146295.767218516
> 146295.885156426 precedes 146296.187140717
> 146298.544907956 precedes 146298.846882759
> 146298.644894308 precedes 146298.946889142

Does this also happen with

kern.timecounter.hardware=acpihpet

?

> OpenBSD 6.5-current (GENERIC.MP) #27: Tue Jun  4 15:36:14 EDT 2019
> dh...@liver.mindcry.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 8418242560 (8028MB)
> avail mem = 8152993792 (7775MB)
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x9a565000 (49 entries)
> bios0: vendor American Megatrends Inc. version "X420UA.204" date 11/16/2018
> bios0: ASUSTeK COMPUTER INC. VivoBook_ASUSLaptop X420UA
> acpi0 at bios0: rev 2
> acpi0: sleep states S0 S3 S4 S5
> acpi0: tables DSDT FACP APIC FPDT FIDT ECDT MCFG SSDT SSDT SSDT MSDM 
> SSDT HPET SSDT UEFI SSDT LPIT SSDT SSDT SSDT SSDT DBGP DBG2 DMAR NHLT 
> BGRT TPM2 WSMT
> acpi0: wakeup devices PXSX(S4) RP06(S4) GLAN(S4) XHC_(S3) XDCI(S4) HDAS(S4)
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3393.27 MHz, 06-8e-0a
> cpu0: 
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S
> SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI
> 1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
> cpu0: 256KB 64b/line 8-way L2 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 23MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3392.09 MHz, 06-8e-0a
> cpu1: 
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S
> SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI
> 

Re: monotonic clock going backwards?

2019-06-06 Thread David Hill



On 6/6/19 6:56 AM, Scott Cheloha wrote:

On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote:

Hello -

I noticed some negative roundtrip times when running traceroute, which uses
the monotonic clock to calculate the RTT.

If I run the following code, it eventually bombs.?? It bombs quick if I
launch Firefox.

timespeccmp failed
tp1 s:103780 n:63101042
tp2 s:103779 n:761117849

please share your dmesg and the output of

sysctl kern.clockrate kern.timecounter

Could you also share the output when you run the following code?
With and without firefox running?  Curious about how frequently
this happens.

#include 

#include 
#include 
#include 

int
main(void)
{
struct timespec t0, t1, timeout;

timeout.tv_sec = 0;
timeout.tv_nsec = 10;

for (;;) {
clock_gettime(CLOCK_MONOTONIC, );
if (nanosleep(, NULL) == -1)
err(1, "nanosleep");
clock_gettime(CLOCK_MONOTONIC, );
if (timespeccmp(, , >)) {
printf("%lld.%09ld precedes %lld.%09ld\n",
t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec);
}
}

return 0;
}



$ sysctl kern.clockrate kern.timecounter
kern.clockrate=tick = 1, tickadj = 40, hz = 100, profhz = 100, 
stathz = 100

kern.timecounter.tick=1
kern.timecounter.timestepwarnings=0
kern.timecounter.hardware=tsc
kern.timecounter.choice=i8254(0) acpihpet0(1000) tsc(2000) acpitimer0(1000)

146259.148543164 precedes 146259.450584680
146259.488592813 precedes 146259.790556528
146259.808566236 precedes 146260.110528202
146261.148476141 precedes 146261.450398822
146262.888279505 precedes 146263.190240715
146263.887431808 precedes 146264.189435801
146274.487159451 precedes 146274.789148187
146278.186820816 precedes 146278.488803756
146279.546680181 precedes 146279.848674138
146284.326305557 precedes 146284.628224696
146285.426154861 precedes 146285.728066901
146287.885921929 precedes 146288.187826688
146289.985763711 precedes 146290.287731233
146290.985659041 precedes 146291.287640302
146293.985348615 precedes 146294.287327519
146295.465186742 precedes 146295.767218516
146295.885156426 precedes 146296.187140717
146298.544907956 precedes 146298.846882759
146298.644894308 precedes 146298.946889142

OpenBSD 6.5-current (GENERIC.MP) #27: Tue Jun  4 15:36:14 EDT 2019
dh...@liver.mindcry.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8418242560 (8028MB)
avail mem = 8152993792 (7775MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0x9a565000 (49 entries)
bios0: vendor American Megatrends Inc. version "X420UA.204" date 11/16/2018
bios0: ASUSTeK COMPUTER INC. VivoBook_ASUSLaptop X420UA
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT ECDT MCFG SSDT SSDT SSDT MSDM 
SSDT HPET SSDT UEFI SSDT LPIT SSDT SSDT SSDT SSDT DBGP DBG2 DMAR NHLT 
BGRT TPM2 WSMT

acpi0: wakeup devices PXSX(S4) RP06(S4) GLAN(S4) XHC_(S3) XDCI(S4) HDAS(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3393.27 MHz, 06-8e-0a
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S

SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI
1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 23MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3392.09 MHz, 06-8e-0a
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S

SSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,SGX,BMI
1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 3392.09 MHz, 06-8e-0a
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,S


Re: [patch] rsync: fix free() on uninitialized pointer with -rx and same device

2019-06-06 Thread Christian Weisgerber
Björn Ketelaars:

> Diff below is based on the latest diff from naddy@. Changes:
> - reallocarray likes type_t, as such changes type of nxdev and i;
> - use reallocarray instead of malloc as xdev is initialised as NULL.

ok naddy@

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de



Re: monotonic clock going backwards?

2019-06-06 Thread Scott Cheloha
On Wed, Jun 05, 2019 at 08:33:56PM -0400, David Hill wrote:
> Hello -
> 
> I noticed some negative roundtrip times when running traceroute, which uses
> the monotonic clock to calculate the RTT.
> 
> If I run the following code, it eventually bombs.?? It bombs quick if I
> launch Firefox.
> 
> timespeccmp failed
> tp1 s:103780 n:63101042
> tp2 s:103779 n:761117849

please share your dmesg and the output of

sysctl kern.clockrate kern.timecounter

Could you also share the output when you run the following code?
With and without firefox running?  Curious about how frequently
this happens.

#include 

#include 
#include 
#include 

int
main(void)
{
struct timespec t0, t1, timeout;

timeout.tv_sec = 0;
timeout.tv_nsec = 10;

for (;;) {
clock_gettime(CLOCK_MONOTONIC, );
if (nanosleep(, NULL) == -1)
err(1, "nanosleep");
clock_gettime(CLOCK_MONOTONIC, );
if (timespeccmp(, , >)) {
printf("%lld.%09ld precedes %lld.%09ld\n",
t1.tv_sec, t1.tv_nsec, t0.tv_sec, t0.tv_nsec);
}
}

return 0;
}



Re: [patch] rsync: fix free() on uninitialized pointer with -rx and same device

2019-06-06 Thread Björn Ketelaars
On Thu 06/06/2019 00:37, Christian Weisgerber wrote:
> Christian Weisgerber:
> 
> > > To reproduce:
> > > 
> > >   mkdir -p /tmp/test /tmp/plop
> > >   openrsync -rx /tmp/test/ /tmp/plop/
> > > 
> > > Result:
> > > 
> > >   openrsync(3470) in free(): bogus pointer (double free?) 0x7f7dcdc8
> > >   Abort trap (core dumped)
> > > 
> > > The patch below fixes it and simplifies the logic:
> > 
> > I agree.  However, the (re)allocation of xdev also looks bogus.
> > 
> > How about this?
> > 
> > (Also, the realloc idiom is exactly the one the man page warns
> > against.  Do we care here?)
> 
> Well, let's not set a bad example.
> 

Guess I'm late to the party...I only read Hiltjo's mail, and the
replies, after being contacted offlist by naddy@.

Diff below is based on the latest diff from naddy@. Changes:
- reallocarray likes type_t, as such changes type of nxdev and i;
- use reallocarray instead of malloc as xdev is initialised as NULL.

diff --git flist.c flist.c
index e1f41b1a108..4c552ce3c5c 100644
--- flist.c
+++ flist.c
@@ -803,12 +803,12 @@ flist_gen_dirent(struct sess *sess, char *root, struct 
flist **fl, size_t *sz,
 size_t *max)
 {
char*cargv[2], *cp;
-   int  rc = 0, nxdev = 0, flag, i;
+   int  rc = 0, flag;
FTS *fts;
FTSENT  *ent;
struct flist*f;
-   size_t   flsz = 0, stripdir;
-   dev_t   *xdev;
+   size_t   i, flsz = 0, nxdev = 0, stripdir;
+   dev_t   *newxdev, *xdev = NULL;
struct stat  st;
 
cargv[0] = root;
@@ -931,11 +931,6 @@ flist_gen_dirent(struct sess *sess, char *root, struct 
flist **fl, size_t *sz,
!S_ISDIR(ent->fts_statp->st_mode))
continue;
 
-   if ((xdev = malloc(sizeof(dev_t))) == NULL) {
-   ERRX1("malloc");
-   goto out;
-   }
-
flag = 0;
for (i = 0; i < nxdev; i++)
if (xdev[i] == ent->fts_statp->st_dev) {
@@ -945,12 +940,12 @@ flist_gen_dirent(struct sess *sess, char *root, struct 
flist **fl, size_t *sz,
if (flag)
continue;
 
-   if (nxdev)
-   if ((xdev = realloc(xdev, sizeof(dev_t))) ==
-   NULL) {
-   ERRX1("realloc");
-   goto out;
-   }
+   if ((newxdev = reallocarray(xdev, nxdev + 1,
+   sizeof(dev_t))) == NULL) {
+   ERRX1("reallocarray");
+   goto out;
+   }
+   xdev = newxdev;
xdev[nxdev] = ent->fts_statp->st_dev;
nxdev++;
}
@@ -1008,8 +1003,7 @@ flist_gen_dirent(struct sess *sess, char *root, struct 
flist **fl, size_t *sz,
rc = 1;
 out:
fts_close(fts);
-   if (sess->opts->one_file_system)
-   free(xdev);
+   free(xdev);
return rc;
 }
 



no background scan while waiting for mgt frame

2019-06-06 Thread Stefan Sperling
Don't start a background scan while we're waiting for a response from
the AP to a management frame we have sent.

In particular, the logic which tries to detect dead APs might be waiting
for a probe response from the AP. Starting a background scan in this state
won't do any good. In case of iwn(4) it seems to result in an endless
scan command dance with the firwmare.

Associated:
Jun  6 10:37:03 jim /bsd: iwn0: missed beacon threshold set to 7 beacons, 
beacon interval is 100 TU

Got "beacon miss" interrupt from firmware; sending probe req to AP to check
if AP is still alive:
Jun  6 10:37:04 jim /bsd: iwn0: sending probe_req to 00:25:56:73:b7:a2 on 
channel 1 mode 11n

Meanwhile, low signal of received frames kicks off a background scan:
Jun  6 10:37:05 jim /bsd: iwn0: begin background scan

Response from AP never arrived; interface watchdog moves to SCAN state
while background scan isn't done yet:
Jun  6 10:37:09 jim /bsd: iwn0: RUN -> SCAN

Driver issues SCAN_ABORT and SCAN commands here.
Jun  6 10:37:09 jim /bsd: iwn0: end active scan
Jun  6 10:37:09 jim /bsd: iwn0: SCAN -> SCAN
Jun  6 10:37:09 jim /bsd: iwn0: end active scan
Driver issues SCAN command here.
Jun  6 10:37:09 jim /bsd: iwn0: SCAN -> SCAN
Jun  6 10:37:09 jim /bsd: iwn0: end passive scan
Driver issues SCAN command here.
Jun  6 10:37:09 jim /bsd: iwn0: SCAN -> SCAN
Jun  6 10:37:09 jim /bsd: iwn0: end passive scan
Driver issues SCAN command here.
Jun  6 10:37:09 jim /bsd: iwn0: SCAN -> SCAN
Jun  6 10:37:09 jim /bsd: iwn0: end passive scan
Driver issues SCAN command here.

These SCAN -> SCAN transitions go on forever until the interface is put down.
For some reason we never get scan results from firmware.

So this diff is merely masking an actual driver bug; but such bugs are
hard to fix without really understanding how firmware's SCAN, SCAN_ABORT
commands and "scan done" interrupts interact.
I suppose asking Intel for docs would be an exercise in futility...

Fortunately, it is easy to avoid this situation in the first place.
Note that the inverse case (ignore "missed beacon" events while a background
scan is in progress) must be fixed separately in the driver, not the stack.

OK?

diff c3669ebdb8dc5ae83ec861abde2c7a1cf357a580 /usr/src
blob - f0e9b2ba482a96a5d7deb442d094f6ecc63afec0
file + sys/net80211/ieee80211.c
--- sys/net80211/ieee80211.c
+++ sys/net80211/ieee80211.c
@@ -75,7 +75,7 @@ ieee80211_begin_bgscan(struct ifnet *ifp)
struct ieee80211com *ic = (void *)ifp;
 
if ((ic->ic_flags & IEEE80211_F_BGSCAN) ||
-   ic->ic_state != IEEE80211_S_RUN)
+   ic->ic_state != IEEE80211_S_RUN || ic->ic_mgt_timer != 0)
return;
 
if (ic->ic_bgscan_start != NULL && ic->ic_bgscan_start(ic) == 0) {



Re: [patch] use acme-client to sign certificated with ecdsa keys

2019-06-06 Thread Renaud Allard



On 6/6/19 10:10 AM, Florian Obser wrote:

On Wed, Jun 05, 2019 at 05:37:51PM +0200, Gilles Chehade wrote:

On Wed, Jun 05, 2019 at 08:39:51AM +0200, Renaud Allard wrote:



On 6/5/19 8:20 AM, Gilles Chehade wrote:

On Tue, Jun 04, 2019 at 03:54:11PM +0200, Renaud Allard wrote:



On 6/3/19 11:53 AM, Renaud Allard wrote:


On 5/29/19 9:58 AM, Florian Obser wrote:

why not let acme-client generate the key?




Here is a more complete diff where you can use the -E switch to
generate a ECDSA key instead of the RSA one.


I refined a little bit the patch to not put ecdsa functions into rsa.c. So I
renamed rsa.c to key.c and removed the rsa references to functions which
apply to both rsa and ecdsa.



reads, builds and works fine for me

a couple comments inlined



I removed the parenthesis and used another wording, removed the RSA from a
"Load RSA key" as it might not be RSA and added E to the SYNOPSYS.



ok gilles@


We don't do bool in base as far as I know, just use an int.



No problem, that's a trivial change.


I currently don't have time to review this. I'm busy switching
acme-client to the rfc 8555 / letsencrypt v2 api. Doesn't look like
this conflicts too badly with my work, but I'd appreciate it if we
could hold this off for a bit and rebase it ontop of the v2 work.


OK, let's wait till your v2 is done. People have the patch already if 
they want to try out ecdsa in the meantime.




I'd also like to get rid of most of the flags and it feels like RSA
vs. ECDSA should be handled in the config file?



I think the flags A,D,v and E could indeed belong to the config file.




smime.p7s
Description: S/MIME Cryptographic Signature


Re: [patch] use acme-client to sign certificated with ecdsa keys

2019-06-06 Thread Florian Obser
On Wed, Jun 05, 2019 at 05:37:51PM +0200, Gilles Chehade wrote:
> On Wed, Jun 05, 2019 at 08:39:51AM +0200, Renaud Allard wrote:
> > 
> > 
> > On 6/5/19 8:20 AM, Gilles Chehade wrote:
> > > On Tue, Jun 04, 2019 at 03:54:11PM +0200, Renaud Allard wrote:
> > > > 
> > > > 
> > > > On 6/3/19 11:53 AM, Renaud Allard wrote:
> > > > > > > 
> > > > > > > On 5/29/19 9:58 AM, Florian Obser wrote:
> > > > > > > > why not let acme-client generate the key?
> > > > > > > 
> > > > > > 
> > > > > > Here is a more complete diff where you can use the -E switch to
> > > > > > generate a ECDSA key instead of the RSA one.
> > > > 
> > > > I refined a little bit the patch to not put ecdsa functions into rsa.c. 
> > > > So I
> > > > renamed rsa.c to key.c and removed the rsa references to functions which
> > > > apply to both rsa and ecdsa.
> > > > 
> > > 
> > > reads, builds and works fine for me
> > > 
> > > a couple comments inlined
> > > 
> > 
> > I removed the parenthesis and used another wording, removed the RSA from a
> > "Load RSA key" as it might not be RSA and added E to the SYNOPSYS.
> > 
> 
> ok gilles@

We don't do bool in base as far as I know, just use an int.

I currently don't have time to review this. I'm busy switching
acme-client to the rfc 8555 / letsencrypt v2 api. Doesn't look like
this conflicts too badly with my work, but I'd appreciate it if we
could hold this off for a bit and rebase it ontop of the v2 work.

I'd also like to get rid of most of the flags and it feels like RSA
vs. ECDSA should be handled in the config file?

> 
> > Index: Makefile
> > ===
> > RCS file: /cvs/src/usr.sbin/acme-client/Makefile,v
> > retrieving revision 1.8
> > diff -u -p -r1.8 Makefile
> > --- Makefile3 Jul 2017 22:21:47 -   1.8
> > +++ Makefile5 Jun 2019 06:37:00 -
> > @@ -2,7 +2,7 @@
> >  PROG=  acme-client
> >  SRCS=  acctproc.c base64.c certproc.c chngproc.c dbg.c 
> > dnsproc.c
> >  SRCS+= fileproc.c http.c jsmn.c json.c keyproc.c main.c 
> > netproc.c
> > -SRCS+= parse.y revokeproc.c rsa.c util.c
> > +SRCS+= parse.y revokeproc.c key.c util.c
> >  
> >  MAN=   acme-client.1 acme-client.conf.5
> >  
> > Index: acctproc.c
> > ===
> > RCS file: /cvs/src/usr.sbin/acme-client/acctproc.c,v
> > retrieving revision 1.12
> > diff -u -p -r1.12 acctproc.c
> > --- acctproc.c  28 Jul 2018 15:25:23 -  1.12
> > +++ acctproc.c  5 Jun 2019 06:37:00 -
> > @@ -82,8 +82,8 @@ op_thumb_rsa(EVP_PKEY *pkey)
> > warnx("bn2string");
> > else if ((exp = bn2string(r->e)) == NULL)
> > warnx("bn2string");
> > -   else if ((json = json_fmt_thumb_rsa(exp, mod)) == NULL)
> > -   warnx("json_fmt_thumb_rsa");
> > +   else if ((json = json_fmt_thumb_key(exp, mod)) == NULL)
> > +   warnx("json_fmt_thumb_key");
> >  
> > free(exp);
> > free(mod);
> > @@ -175,10 +175,10 @@ op_sign_rsa(char **head, char **prot, EV
> > warnx("bn2string");
> > else if ((exp = bn2string(r->e)) == NULL)
> > warnx("bn2string");
> > -   else if ((*head = json_fmt_header_rsa(exp, mod)) == NULL)
> > -   warnx("json_fmt_header_rsa");
> > -   else if ((*prot = json_fmt_protected_rsa(exp, mod, nonce)) == NULL)
> > -   warnx("json_fmt_protected_rsa");
> > +   else if ((*head = json_fmt_header_key(exp, mod)) == NULL)
> > +   warnx("json_fmt_header_key");
> > +   else if ((*prot = json_fmt_protected_key(exp, mod, nonce)) == NULL)
> > +   warnx("json_fmt_protected_key");
> > else
> > rc = 1;
> >  
> > @@ -338,7 +338,7 @@ acctproc(int netsock, const char *acctke
> > goto out;
> > dodbg("%s: generated RSA account key", acctkey);
> > } else {
> > -   if ((pkey = rsa_key_load(f, acctkey)) == NULL)
> > +   if ((pkey = key_load(f, acctkey)) == NULL)
> > goto out;
> > doddbg("%s: loaded RSA account key", acctkey);
> > }
> > Index: acme-client.1
> > ===
> > RCS file: /cvs/src/usr.sbin/acme-client/acme-client.1,v
> > retrieving revision 1.29
> > diff -u -p -r1.29 acme-client.1
> > --- acme-client.1   3 Feb 2019 20:39:35 -   1.29
> > +++ acme-client.1   5 Jun 2019 06:37:00 -
> > @@ -22,7 +22,7 @@
> >  .Nd ACME client
> >  .Sh SYNOPSIS
> >  .Nm acme-client
> > -.Op Fl ADFnrv
> > +.Op Fl ADEFnrv
> >  .Op Fl f Ar configfile
> >  .Ar domain
> >  .Sh DESCRIPTION
> > @@ -79,7 +79,9 @@ The options are as follows:
> >  .It Fl A
> >  Create a new RSA account key if one does not already exist.
> >  .It Fl D
> > -Create a new RSA domain key if one does not already exist.
> > +Create a new domain key if one does not already exist. Defaults to RSA.
> > +.It Fl E
> > +Switch the 

Re: [patch] rsync: fix free() on uninitialized pointer with -rx and same device

2019-06-06 Thread Florian Obser
OK

On Thu, Jun 06, 2019 at 12:37:13AM +0200, Christian Weisgerber wrote:
> Christian Weisgerber:
> 
> > > To reproduce:
> > > 
> > >   mkdir -p /tmp/test /tmp/plop
> > >   openrsync -rx /tmp/test/ /tmp/plop/
> > > 
> > > Result:
> > > 
> > >   openrsync(3470) in free(): bogus pointer (double free?) 0x7f7dcdc8
> > >   Abort trap (core dumped)
> > > 
> > > The patch below fixes it and simplifies the logic:
> > 
> > I agree.  However, the (re)allocation of xdev also looks bogus.
> > 
> > How about this?
> > 
> > (Also, the realloc idiom is exactly the one the man page warns
> > against.  Do we care here?)
> 
> Well, let's not set a bad example.
> 
> Index: flist.c
> ===
> RCS file: /cvs/src/usr.bin/rsync/flist.c,v
> retrieving revision 1.27
> diff -u -p -r1.27 flist.c
> --- flist.c   2 Jun 2019 14:29:58 -   1.27
> +++ flist.c   5 Jun 2019 22:36:28 -
> @@ -808,7 +808,7 @@ flist_gen_dirent(struct sess *sess, char
>   FTSENT  *ent;
>   struct flist*f;
>   size_t   flsz = 0, stripdir;
> - dev_t   *xdev;
> + dev_t   *newxdev, *xdev = NULL;
>   struct stat  st;
>  
>   cargv[0] = root;
> @@ -931,7 +931,8 @@ flist_gen_dirent(struct sess *sess, char
>   !S_ISDIR(ent->fts_statp->st_mode))
>   continue;
>  
> - if ((xdev = malloc(sizeof(dev_t))) == NULL) {
> + if (xdev == NULL &&
> + (xdev = malloc(sizeof(dev_t))) == NULL) {
>   ERRX1("malloc");
>   goto out;
>   }
> @@ -945,12 +946,14 @@ flist_gen_dirent(struct sess *sess, char
>   if (flag)
>   continue;
>  
> - if (nxdev)
> - if ((xdev = realloc(xdev, sizeof(dev_t))) ==
> - NULL) {
> + if (nxdev) {
> + if ((newxdev = reallocarray(xdev, nxdev + 1,
> + sizeof(dev_t))) == NULL) {
>   ERRX1("realloc");
>   goto out;
>   }
> + xdev = newxdev;
> + }
>   xdev[nxdev] = ent->fts_statp->st_dev;
>   nxdev++;
>   }
> @@ -1008,8 +1011,7 @@ flist_gen_dirent(struct sess *sess, char
>   rc = 1;
>  out:
>   fts_close(fts);
> - if (sess->opts->one_file_system)
> - free(xdev);
> + free(xdev);
>   return rc;
>  }
>  
> -- 
> Christian "naddy" Weisgerber  na...@mips.inka.de
> 

-- 
I'm not entirely sure you are real.



Re: [patch] rsync: fix free() on uninitialized pointer with -rx and same device

2019-06-06 Thread Hiltjo Posthuma
On Thu, Jun 06, 2019 at 12:37:13AM +0200, Christian Weisgerber wrote:
> Christian Weisgerber:
> 
> > > To reproduce:
> > > 
> > >   mkdir -p /tmp/test /tmp/plop
> > >   openrsync -rx /tmp/test/ /tmp/plop/
> > > 
> > > Result:
> > > 
> > >   openrsync(3470) in free(): bogus pointer (double free?) 0x7f7dcdc8
> > >   Abort trap (core dumped)
> > > 
> > > The patch below fixes it and simplifies the logic:
> > 
> > I agree.  However, the (re)allocation of xdev also looks bogus.
> > 

Hi,

I noticed the realloc issue also and wanted to send a separate patch later for
it. Nice it is addressed also :)

> > How about this?
> > 
> > (Also, the realloc idiom is exactly the one the man page warns
> > against.  Do we care here?)
> 
> Well, let's not set a bad example.
> 
> Index: flist.c
> ===
> RCS file: /cvs/src/usr.bin/rsync/flist.c,v
> retrieving revision 1.27
> diff -u -p -r1.27 flist.c
> --- flist.c   2 Jun 2019 14:29:58 -   1.27
> +++ flist.c   5 Jun 2019 22:36:28 -
> @@ -808,7 +808,7 @@ flist_gen_dirent(struct sess *sess, char
>   FTSENT  *ent;
>   struct flist*f;
>   size_t   flsz = 0, stripdir;
> - dev_t   *xdev;
> + dev_t   *newxdev, *xdev = NULL;
>   struct stat  st;
>  
>   cargv[0] = root;
> @@ -931,7 +931,8 @@ flist_gen_dirent(struct sess *sess, char
>   !S_ISDIR(ent->fts_statp->st_mode))
>   continue;
>  
> - if ((xdev = malloc(sizeof(dev_t))) == NULL) {
> + if (xdev == NULL &&
> + (xdev = malloc(sizeof(dev_t))) == NULL) {
>   ERRX1("malloc");
>   goto out;
>   }
> @@ -945,12 +946,14 @@ flist_gen_dirent(struct sess *sess, char
>   if (flag)
>   continue;
>  
> - if (nxdev)
> - if ((xdev = realloc(xdev, sizeof(dev_t))) ==
> - NULL) {
> + if (nxdev) {
> + if ((newxdev = reallocarray(xdev, nxdev + 1,
> + sizeof(dev_t))) == NULL) {

Also could the first malloc and realloc maybe get simplified to
reallocarray(NULL, ...) ?

>   ERRX1("realloc");
>   goto out;
>   }
> + xdev = newxdev;
> + }
>   xdev[nxdev] = ent->fts_statp->st_dev;
>   nxdev++;
>   }
> @@ -1008,8 +1011,7 @@ flist_gen_dirent(struct sess *sess, char
>   rc = 1;
>  out:
>   fts_close(fts);
> - if (sess->opts->one_file_system)
> - free(xdev);
> + free(xdev);
>   return rc;
>  }
>  
> -- 
> Christian "naddy" Weisgerber  na...@mips.inka.de
> 

-- 
Kind regards,
Hiltjo