On Mon, 20 Mar 2017, Filippo Valsorda wrote: > Every time I start a certain daemon (Camlistore, written in golang), > during a precise step involving TCP/HTTPS traffic, AES decryption, and > loading data into the heap (loading the encryption metadata of a remote > storage), I experience a long userspace hang. > > Sometimes the hang lasts seconds, sometimes minutes. During the hang > everything is unresponsive except the kernel: network traffic drops, > serial console is stuck, ssh is stuck and eventually times out, > sometimes other applications running on the machine notice that "time > jumped forwards", ICMP ping works, trying to ssh in manages to open a > connection but no banner is ever received. Essentially, userspace > doesn't get any CPU time. Most of the times Camlistore fails to start > with a TLS timeout, sometimes it manages to carry on. ... > ddb breaking from the serial console works, producing variations of the > following two traces. ... > --- interrupt --- > Xspllower() at Xspllower+0xc > mtx_leave() at mtx_leave+0x34 > uvmpd_scan_inactive() at uvmpd_scan_inactive+0x3e5 > uvmpd_scan() at uvmpd_scan+0x2a0 > uvm_pageout() at uvm_pageout+0x72 > end trace frame: 0x0, count: -9 ... > dmesg: > OpenBSD 6.1-beta (GENERIC.MP) #37: Fri Mar 17 07:46:47 MDT 2017 > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP > real mem = 2094940160 (1997MB) > avail mem = 2026835968 (1932MB)
In the interest of helping you get work done, it sounds like you're running a task that has a (much?) larger resident set requirement than this box has physical memory available to userspace, resulting in constant paging that drags it into the bowels of the UVM subsystem. We could handle that better... "You're going to need a bigger boat." Philip Guenther
