On Mon, Sep 19, 2016 at 11:55 AM, Avi Kivity <a...@scylladb.com> wrote:

> On 09/19/2016 11:49 AM, Nadav Har'El wrote:
> On Mon, Sep 19, 2016 at 10:52 AM, Gleb Natapov <g...@scylladb.com> wrote:
>> > 2. I'm afraid the scheduler thing might only be the tip of the iceberg
>> of
>> > problems caused by this lazy TLB thing. Could we have, for example (and
>> > this is just a hypothetical example) one thread doing a write() to disk
>> of
>> > some data from an mmap'ed area, and this data is supposed to be read by
>> a
>> > ZFS thread which runs on a different CPU - and because it is labeled a
>> > "system thread", it won't do a TLB flush before reading the mmap'ed
>> area?
>> write() copies data into ZFS ARC.
>> > Why are we confident that "system threads" never need to read user's
>> > mmap'ed data?
>> >
>> If they do this is a bug as you discovered. They may access mmaped
>> memory, but they should do so through their own mappings.
> So basically, "system threads" (and the scheduler) are not allowed to read
> any user memory, because any user memory may be mmap'ed.
> Whatever user memory is needed in a system thread, must first be *copied*
> in the original user thread...
> Or, the mapping must be pinned (and paged in) for the duration of the
> access.

So if I understand correctly, both your and Gleb's sentiment is there is no
need not to make the scheduler more forgiving for page table changes, but
rather require that sched::threads objects live in a mapping which could
never change - which more-or-less means in the OSv case, that sched::thread
objects should be allocated only on the heap.

So a patch which will enforce sched::thread to be allocated only on the
heap is the right way to go here? (I sent such a patch already, but I'll
try to send a slightly less ugly one. In any case it will be a long patch).

By the way, it is obvious that a sched::thread cannot live in a paged-out
mapping, as the scheduler cannot sleep to wait for it. But the issue here
is more subtle: stack memory is guaranteed to be always paged in (for
reasons explained here https://github.com/cloudius-systems/osv/issues/143)
- and nontheless, it is still not fine to put a sched::thread there because
of the TLB flushing issue.

You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to