If you had a simple test program or microbenchmark which put some numbers behind your concerns, I think you'll get a bit more traction. Have you tried to measure the effect of page faults on reading data from embedded files (or, I guess, any sort of data that has fallen out of the pagecache)? I'm interested in calibrating my gut, which is currently surprised that you're talking about page fault latency in the context of a garbage-collected language like Go, but I'm happy to be proven wrong :)
(And from a tuning side, if you're only concerned about tying up Ps I suspect you could increase GONUMPROCS to get threads "running", but of course that might cause trouble if you have something that is CPU-bounded in the server.) On Tuesday, February 16, 2021 at 9:48:30 AM UTC-7 ca...@mercari.com wrote: > Was going through the implementation of embed.FS and started wondering > about the lack of WriteTo - specifically about the implications in case > the embedded data is not present in memory e.g. because it got evicted due > to memory pressure - or simply due to outright not fitting in memory. Won’t > this tie Ps up while the data is faulted in, with no way for the scheduler > to park the offending Gs? > The design doc discusses some arguments against adding WriteTo > <https://go.googlesource.com/proposal/+/master/design/draft-embed.md#direct-access-to-embedded-data> > > but does not discuss the issue described above. Am I missing something? > Obviously, WriteTo wouldn't solve this problem above for *all* cases > (e.g. all cases in which the WriteTo implementation is unable to bypass > the userspace copy) although it should help significantly when it does > (e.g. in case of embedded static files to be served over plain HTTP). Or > maybe reads from embed.FS instances could try to opportunistically > madvise(MADV_WILLNEED) and yield if mincore returns that the pages are > not (yet) resident in memory? Or the scheduler/sysmon should be taught how > to handle that case (e.g. with userfaultfd on Linux)? > > Carlo > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b49a03e3-ead3-4c11-9044-85affe8e1017n%40googlegroups.com.
