Github user theburge commented on the issue:
    @jaydoane Thanks for running and documenting those experiments, I think 
those results look very promising. Reducing time taken and user-space memory 
usage can only be good. :)
    FWIW, I'm not convinced major faults is the metric of interest for 
identifying file reads on newly opened file in an empty cache scenario. (My 
thinking is that it shows faults for explicitly mapped ranges that are only 
available on-disk; whereas page ins show pages being pulled in for whatever 
reason via the VM itself. For laughs and demonstration purposes, run `sar -B 1` 
and try some experiments, like dropping cache and loading a `man` page. You'll 
typically see page in rate exceed major fault rate, once you do the 
conversation from pages/s to kB/s. In truth, it's probably most comprehensive 
to look at both fault types and page in/page out rates, but that gets trickier.)
    The big surprise here for me is the difference in physical disk operations, 
since my understanding would be that the VM would fill complete OS pages to 
satisfy reads, so I can't see why reading N bytes on page boundaries would be 
much different (from an OS perspective) to reading 4kB. I'd love an explanation 
for that (although the user-space benefits are clear to see and very positive), 
just to be sure it's not some other effect.
    My final suggestion would be to evaluate a more serious MT-esque scenario: 
how does the performance improvement of the new algorithm vary with with lots 
(10s to 100s) of files opening concurrently? (And maybe with higher background 
CPU/disk utilisation, if you can set-up the experiment.) Remember that that's 
the scenario where we see this as most relevant, and timeouts there are a big 
issue (file opening times out, compaction keels over, file handle remains, 
etc.), so if there's little or no wall clock time improvement in a highly 
concurrent setting, that'd be a shame and undermine some of the practical 
benefit. Obviously, I'd expect only positive things compared to the status quo 
based on the evaluation above, but performance expectations in varying 
scenarios don't always comply with common sense... :) 

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

Reply via email to