On 24/04/15 08:08, Martin Lucina wrote:
Heh, interesting. Something is leaking and causing a slowdown. The
good news is that we can extrapolate that the actual performance
will be >1.4k/s once we find that leak. You can probably get a hint
of that value by running a 2s blast.
I'll try to repro and find the problem (but most likely not until
next week).
I've known about this (nginx getting slower over time) but not had time to
investigate thoroughly.
I think I fixed it. Pull rumprun and rebuild (cd platform/xen && make
clean && make) and then remember to also rebuild the nginx binary.
Can we get a re-run of the benchmarks? I get a >30% performance boost
for the first 20s siege run and don't observe slowdown over multiple
long 200s runs. However, since I'm running on baremetal/qemu instead of
Xen, the numbers aren't at least directly comparable.
So what was the problem? It was somewhat of a configuration problem.
nginx was writing access.log to /tmp. By default, /tmp is rumpfs.
rumpfs is the simplest, most barebones viable file system driver. It
does not even allocate files in blocks, just as a continuous slice of
memory:
http://nxr.netbsd.org/xref/src/sys/rump/librump/rumpvfs/rumpfs.c#1418
Yea, I guess I don't have to go into detail on why that causes
exponential slowdown and eventually causes meltdown.
So, I made rumprun_boot() mount a 1MB tmpfs on /tmp by default. The
tmpfs driver is multiple times the size of rumpfs, but the rumprun
unikernel isn't really a _minimal_ rump kernel environment, so we can
splurge a bit. That said, a 1MB access log fills up pretty fast, but
for any serious access.log-like business we should offer file-based
storage anyway.
I think I'll make the rootfs (rumpfs) permissions something else than
0777 by default. That will at least make it impossible for programs not
running as "root" to accidentally write to inopportune locations.
