I'm going to wait until tomorrow before posting it.  I have a bunch of
    cleanup to do.

    Bruce, your sys.dif was *invaluable*!  It would have taken me a lot 
    longer to figure out the interrupt disablement requirement around
    the icu_lock, and the statclock and hardclock forwarding issues (which I
    got working by the way!).

    It turns out that putting the pending masks in the per-cpu area was
    the right solution!  It made statclock and hardclock forwarding easy
    (I can see why you didn't even try in your patch set, doing it with
    a global mask would be nearly impossible).  In fact, it made everything
    unbelievably easy.

    Apart from all the assembly coding, there were two serious issues.
    fork_exit() assumes that interrupts are hard-disabled on entry.  I
    readjusted the code such that the trampoline assembly initialized
    the td_critnest count so it could STI prior to calling fork_exit().

    The second issue is that cpu_switch() does not save/restore the 
    PSL_I (interrupt disablement mask).  I added a PSL word to the PCB
    structure to take care of the problem.  Without this if you have
    a thread switch away while holding interrupts hard-disabled, the
    next thread will inherit the hard disablement.  I saw the sti's
    you added in various places but I figured the best solution was
    to simply save/restore the state.  The original code didn't have
    this problem because interrupts were always hard-disabled while
    holding the sched_lock and, of course, cpu_switch() is always
    called while holding sched_lock.  (Similarly, icu_lock needed 
    hard-disablements wrapped around it for the same reason, because
    a level interrupt still needs to be able to get icu_lock in order to
    defer itself).

    In anycase, I have successfully built the world in a -current 
    SMP + apic_vector system.  Tomorrow I will cleanup on the UP icu_vector
    code to match the apic_vector code and post the results.  I also
    have a sysctl that allows developers to toggle the mode for testing
    purposes (old way or new way).

    Finally, I took your suggestion in regards to not trying to combine
    the masks together.  I have a 'some sort of interrupt is pending'
    variable then I have fpending (for fast interrupts), ipending (for
    normal interrupt threads), and spending (which I use for the stat and
    hardclock forwarding).  They're all per-cpu entities, of course.
    unpend() prioritizes them.

    In anycase, I'll post it all tomorrow after I've got icu_vector cleaned
    up.  One of the best things about this patch set is that it is really
    flexible.  We should be able to really make interrupts fly.  In fact,
    it should even be possible for one cpu to steal another cpu's pending
    interrupt(s) fairly trivially, without having to resort to IPIs.


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Reply via email to