the bitrig haesbaert_ithread branch has been created by haesbaert. it is 1 commit behind master, and 35 commits ahead.
commit e3f609c190016d6f2b83f12682847691fe639b80 diff: https://github.com/bitrig/bitrig/commit/e3f609c author: Christiano Haesbaert <[email protected]> date: Fri Jan 16 12:37:13 2015 +0100 Newly formed process must be in a critical section. M sys/kern/kern_fork.c M sys/sys/proc.h commit ebfb3e3a06da120a9dfa967ee10e4438a5f10fba diff: https://github.com/bitrig/bitrig/commit/ebfb3e3 author: Christiano Haesbaert <[email protected]> date: Thu Jan 15 14:13:48 2015 +0100 Add missing cld calls in vector.S to conform with the latest sync changes. M sys/arch/amd64/amd64/vector.S commit 43fae6fdfe67b89c29f03787db32467aa1d4d5c2 diff: https://github.com/bitrig/bitrig/commit/43fae6f author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 23:08:30 2015 +0100 Kill ih_pin in amd64, we have that in intrsource.is_pin. M sys/arch/amd64/amd64/intr.c M sys/arch/amd64/include/intr.h M sys/kern/kern_softintr.c commit 24cd9ef7a4457647ffce53514a4d60decca7cb44 diff: https://github.com/bitrig/bitrig/commit/24cd9ef author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 18:52:39 2015 +0100 ARM atomic.h should include stdatomic if it inlines C11 atomic stuff M sys/arch/arm/include/atomic.h commit fd67ef5c78b4b4a2940664d2732e23866150e1b7 diff: https://github.com/bitrig/bitrig/commit/fd67ef5 author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 20:10:52 2015 +0100 Missing softintr.h include in kern_sig.c M sys/kern/kern_sig.c commit 0ccee7fca0155770e52d508847eaba4c9c7b283c diff: https://github.com/bitrig/bitrig/commit/0ccee7f author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 20:46:16 2015 +0100 More missing includes found while porting smpns to ARM. M sys/dev/ic/ahcivar.h M sys/dev/sdmmc/sdmmc_io.c M sys/kern/kern_malloc.c M sys/kern/subr_disk.c M sys/kern/subr_evcount.c commit 770188aa801d2913a7260fd55bf48dc7ff6cecf4 diff: https://github.com/bitrig/bitrig/commit/770188a author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 17:46:33 2015 +0100 Some define alignment and whitespace cleanup. M sys/arch/arm/cortex/ampintc.c commit e599176b2969991878a141a5538098dbef820092 diff: https://github.com/bitrig/bitrig/commit/e599176 author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 17:39:42 2015 +0100 Rename intrsource members from iq_* to is_* and kill two unused ones. M sys/arch/arm/cortex/ampintc.c M sys/arch/armv7/include/intr.h M sys/arch/armv7/omap/intc.c M sys/arch/armv7/sunxi/a1xintc.c commit 89dc405d6e949c91ac0f943ccfdbdbd661d4ffd4 diff: https://github.com/bitrig/bitrig/commit/89dc405 author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 17:32:10 2015 +0100 Rename intrq to intrsource and move to a common arm header. M sys/arch/arm/cortex/ampintc.c M sys/arch/armv7/include/intr.h M sys/arch/armv7/omap/intc.c M sys/arch/armv7/sunxi/a1xintc.c commit 99f09a3488e51b56c34c5aa368f96bf19d64106b diff: https://github.com/bitrig/bitrig/commit/99f09a3 author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 18:43:21 2015 +0100 If you're using crit_enter you need proc.h M sys/dev/ic/com.c M sys/dev/usb/ehci.c commit 97d8f6d78c915576178823857797d805bef5ea17 diff: https://github.com/bitrig/bitrig/commit/97d8f6d author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 16:49:03 2015 +0100 Convert hand rolled intr handler list to TAILQ in amd64. Moves it closer to arm in making intrsource MI. M sys/arch/amd64/amd64/intr.c M sys/arch/amd64/include/intr.h M sys/arch/amd64/isa/isa_machdep.c M sys/kern/kern_ithread.c M sys/kern/kern_softintr.c commit 3fd07c3377a75250e4b0e90bd1473e3410bdc79c diff: https://github.com/bitrig/bitrig/commit/3fd07c3 author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 16:44:02 2015 +0100 Fix use after free in intr_disestablish(). M sys/arch/amd64/amd64/intr.c commit d2bf96409c4f5eadb7b6ec7eac14d155260774d5 diff: https://github.com/bitrig/bitrig/commit/d2bf964 author: Christiano Haesbaert <[email protected]> date: Tue Jan 13 15:06:32 2015 +0100 Cleanup genassym.cf from amd64 M sys/arch/amd64/amd64/genassym.cf commit ce7d8d4ea9bac5117d65fde0cf12c961f781b763 diff: https://github.com/bitrig/bitrig/commit/ce7d8d4 author: Christiano Haesbaert <[email protected]> date: Mon Jan 12 16:37:35 2015 +0100 Make sure ipi and clocks are always in a critical section. Don't assert ipi CRIT_DEPTH because of ddb and so on. M sys/arch/amd64/amd64/ipi.c M sys/arch/amd64/amd64/lapic.c M sys/arch/amd64/amd64/spl.S M sys/kern/kern_clock.c commit 240be903cba9174aaa0916bb0927821a6b8e0ee6 diff: https://github.com/bitrig/bitrig/commit/240be90 author: Christiano Haesbaert <[email protected]> date: Mon Jan 12 15:54:27 2015 +0100 Add P_ITHREAD to the string formatting bits. M sys/sys/proc.h commit 2ef296b85e6f01924c275848ae5125b67f83bca1 diff: https://github.com/bitrig/bitrig/commit/2ef296b author: Christiano Haesbaert <[email protected]> date: Wed Dec 10 02:14:52 2014 +0100 Give device names to ithreads p_comm and evcounters to softintr. Concatenate all device names in the ithread's process name, this is useful for debugging and/or profiling. Also put event counters on software interrupts, I can't recall how many times I wanted to know this. A ps auxkl might show the following now: root 6034 0.0 0.0 0 0 ?? DK 2:01AM 0:04.97 (softclock) 0 0 0 -18 0 intr root 24717 0.0 0.0 0 0 ?? DK 2:01AM 0:01.06 (softnet) 0 0 0 -18 0 intr root 25481 0.0 0.0 0 0 ?? DK 2:01AM 0:01.08 (ehci0,mpi0) 0 0 0 -18 0 intr root 23688 0.0 0.0 0 0 ?? DK 2:01AM 0:01.02 (em0) 0 0 0 -18 0 intr .... M sys/dev/ic/com.c M sys/dev/usb/usb.c M sys/kern/kern_clock.c M sys/kern/kern_ithread.c M sys/kern/kern_softintr.c M sys/net/netisr.c M sys/net/pipex.c M sys/sys/softintr.h commit 5c16a78757072ec2a246d447656464522d03a0da diff: https://github.com/bitrig/bitrig/commit/5c16a78 author: Christiano Haesbaert <[email protected]> date: Mon Dec 8 23:36:55 2014 +0100 Cleanup softintr, remove establish_mpsafe now that IPL_FLAGS is here. Also kill unused softintr.c from amd64. ok pwildt@ D sys/arch/amd64/amd64/softintr.c M sys/kern/kern_softintr.c M sys/sys/softintr.h commit 2f6417e796959c2e0e293f91fbcb7f99134ab486 diff: https://github.com/bitrig/bitrig/commit/2f6417e author: Patrick Wildt <[email protected]> date: Sat Aug 23 22:52:57 2014 +0200 Revert to nearly the old softintr_ API, but keep ithreads for it. ok haesbaert@ M sys/arch/amd64/include/intr.h M sys/conf/files M sys/dev/ic/com.c M sys/dev/usb/usb.c M sys/kern/kern_clock.c M sys/kern/kern_ithread.c M sys/kern/kern_sig.c A sys/kern/kern_softintr.c M sys/net/netisr.c M sys/net/netisr.h M sys/net/pipex.c M sys/sys/ithread.h A sys/sys/softintr.h commit c8f72708b541d20bcbe4c225706aa7f740c1491c diff: https://github.com/bitrig/bitrig/commit/c8f7270 author: Christiano Haesbaert <[email protected]> date: Tue Dec 2 19:21:36 2014 +0100 Simplify ithread_sleep() and ithread_run(). setrunnable() is less efficient since it does unecessary computations for a kthread. Even so, lets use it as it diminishes the amount of replicated code. Conflicts: sys/kern/kern_ithread.c sys/kern/sched_bsd.c M sys/kern/kern_ithread.c M sys/sys/proc.h commit 1471edd17b4436a6662e6d7dbf20b23c2446114c diff: https://github.com/bitrig/bitrig/commit/1471edd author: Christiano Haesbaert <[email protected]> date: Thu Oct 30 21:04:14 2014 +0100 crit_rundeferred() -> crit_unpend(), call MD intr_unpend(). Make crit_unpend() the function to be called everytime we leave a critical section, make it call intr_unpend() which is MD. Trying to keep the code in intr_unpend() MI now is unrealistic. Conflicts: sys/kern/kern_crit.c M sys/arch/amd64/amd64/intr.c M sys/arch/amd64/amd64/spl.S M sys/arch/amd64/include/intr.h M sys/kern/kern_crit.c M sys/sys/proc.h commit baedf64cfedbfac8a0ece361a4c606875b02f21f diff: https://github.com/bitrig/bitrig/commit/baedf64 author: Christiano Haesbaert <[email protected]> date: Mon Sep 29 22:46:05 2014 +0200 Slightly more efficient crit_rundeferred(). Instead of disabling/enabling interrupts for every ipending intrsource, do it once and clear it. Another option is using atomic_exchange_explicit() but the bus locking is unecessary since this is a cpu local word. M sys/kern/kern_crit.c commit 47bce7f6768c6983a741122e79f9c9f186d0842e diff: https://github.com/bitrig/bitrig/commit/47bce7f author: Christiano Haesbaert <[email protected]> date: Sun Sep 28 12:23:23 2014 +0200 Be a bit more strict with KERNEL_LOCK ordering DIAGNOSTIC. Also make the loss of atomicity in pool_get() drop and reaquire the critical sections. This makes the kernel lock crit depth messages go away on boot. M sys/kern/kern_malloc.c M sys/kern/subr_pool.c M sys/sys/systm.h commit e8f5b8078eb0f0a3a463f3b933ca2d472cd6cc0e diff: https://github.com/bitrig/bitrig/commit/e8f5b80 author: Christiano Haesbaert <[email protected]> date: Sat Sep 27 13:01:27 2014 +0200 Kill CRITCOUNTERS left overs M sys/kern/kern_crit.c commit b9652d2b3bf0940ab12f059e4fd991c10306c812 diff: https://github.com/bitrig/bitrig/commit/b9652d2 author: Christiano Haesbaert <[email protected]> date: Sat Sep 27 12:52:48 2014 +0200 Use the intr_ as prefix for interrupt API, intr_[disable|enable|...] Discussed with Patrick, we both agree this makes more sense than using a suffix. Also use intr_get_state() and intr_set_state() instead of state() and restore(). M sys/arch/amd64/amd64/amd64_mem.c M sys/arch/amd64/amd64/hibernate_machdep.c M sys/arch/amd64/amd64/i8259.c M sys/arch/amd64/amd64/ioapic.c M sys/arch/amd64/amd64/ipifuncs.c M sys/arch/amd64/amd64/lapic.c M sys/arch/amd64/amd64/lock_machdep.c M sys/arch/amd64/amd64/machdep.c M sys/arch/amd64/include/cpufunc.h M sys/arch/amd64/isa/clock.c M sys/dev/acpi/acpi.c M sys/dev/isa/gus.c M sys/kern/kern_crit.c M sys/kern/kern_sched.c commit dc8eb4293adc7e0f8479cce2161f126891fdd62d diff: https://github.com/bitrig/bitrig/commit/dc8eb42 author: Christiano Haesbaert <[email protected]> date: Tue Sep 16 20:21:46 2014 +0200 Add ithread.h to distrib sets. M distrib/sets/lists/base/md.amd64 M distrib/sets/lists/comp/mi commit e568733bfac6aef30616f190df1cbd4c75a6718f diff: https://github.com/bitrig/bitrig/commit/e568733 author: Christiano Haesbaert <[email protected]> date: Fri Aug 29 13:38:18 2014 +0200 Use more sensical labels on vector.S M sys/arch/amd64/amd64/vector.S commit d58e6b878998554e25d20128f6a976818ea14f58 diff: https://github.com/bitrig/bitrig/commit/d58e6b8 author: Christiano Haesbaert <[email protected]> date: Wed Aug 27 18:21:02 2014 +0200 Decouple the idea of masking/unmasking an intrsource from pic{}. When we take an interrupt, we mask the source and schedule the thread, when the thread eventually runs and finishes processing, it unmasks the source. So there must be a way of unmasking the source in a MI fashion, each architecture should have a intrsource_unmask() function. In amd64, we just map it to the corresponding pic{} callback. M sys/arch/amd64/include/intr.h M sys/kern/kern_ithread.c commit e2ae4a43e05445c64aa786992f9c1919fc06fc74 diff: https://github.com/bitrig/bitrig/commit/e2ae4a4 author: Christiano Haesbaert <[email protected]> date: Wed Aug 27 18:06:11 2014 +0200 Fix interrupt account with intr_shared_edge != 0. M sys/kern/kern_ithread.c commit 92edddbb230d5fc0211996578203abc7ed4ca00b diff: https://github.com/bitrig/bitrig/commit/92edddb author: Christiano Haesbaert <[email protected]> date: Wed Aug 27 15:49:12 2014 +0200 Whitespace & wording M sys/kern/kern_ithread.c commit 6b0b205d8ce040f50d1a5398f1999ca29a218b2e diff: https://github.com/bitrig/bitrig/commit/6b0b205 author: Christiano Haesbaert <[email protected]> date: Wed Aug 27 14:50:15 2014 +0200 Introduce intr_state_t and MI "API" for blocking interrupts. The fact that disabling/enabling real interrupts is totally MD makes writing portable code harder, here I propose the following API: enable_intr(void) Enables "all" hw interrupts. disable_intr(void) Disables "all" hw interrupts. intr_state_t state_intr(void) Reads hw interrupts state. restore_intr(intr_state_t) Restore hw interrupts state. I think this should map easily to arm or any other future platform, intr_state_t should be defined to whatever is convenient to the arch. Switch crit_rundeferred() to use it, when we port stuff to arm, we just need to mimick the API. M sys/arch/amd64/include/cpufunc.h M sys/kern/kern_crit.c commit 17ead20cfcf47becdc7a526bd08aef339e84a131 diff: https://github.com/bitrig/bitrig/commit/17ead20 author: Christiano Haesbaert <[email protected]> date: Tue Aug 26 16:13:16 2014 +0200 Kill is_minlevel & is_maxlevel from intrsource{}. This is a step into turning intrsource{} MI and getting rid of the IPL leftovers. Conflicts: sys/arch/amd64/amd64/intr.c M sys/arch/amd64/amd64/acpi_machdep.c M sys/arch/amd64/amd64/genassym.cf M sys/arch/amd64/amd64/intr.c M sys/arch/amd64/amd64/machdep.c M sys/arch/amd64/include/intr.h M sys/arch/amd64/include/segments.h M sys/kern/kern_ithread.c commit 02bfc5e2504d2582aa1a1e5c714931603bd27ab2 diff: https://github.com/bitrig/bitrig/commit/02bfc5e author: Christiano Haesbaert <[email protected]> date: Fri Jun 6 11:42:01 2014 +0200 Stop using the sleepqueue for interrupt threads. There is no need for it, it cuts some hacks and it's less instructions. Since we always wakeup from ithread_run(), we always know the proc to awake, the solely point of the sleepqueue is to be a storage to be found via ident hashing. Conflicts: sys/kern/kern_ithread.c M sys/kern/kern_ithread.c commit e3e021854f07d8e2cdc54ee17dca2332a5e8fe28 diff: https://github.com/bitrig/bitrig/commit/e3e0218 author: Christiano Haesbaert <[email protected]> date: Sun May 11 17:22:11 2014 +0200 No need for running interrupt handlers in critical sections anymore. Preemption correctly relies on the priority to do preemption, having a ithread->ithread preemption due to ithread sleeping at a lower priority should be ok. M sys/kern/kern_ithread.c commit 59fd8906da0a813135a2f35c1082106e52eacfbf diff: https://github.com/bitrig/bitrig/commit/59fd890 author: Christiano Haesbaert <[email protected]> date: Fri Jan 16 12:00:55 2015 +0100 Conver spl{high,ipi,clock,statclock} -> crit_enter(). This is a bunch of manual convertions, nothing special, the hard parts are in another commit. M sys/arch/amd64/amd64/cpu.c M sys/arch/amd64/amd64/db_interface.c M sys/arch/amd64/amd64/fpu.c M sys/arch/amd64/amd64/lapic.c M sys/arch/amd64/amd64/machdep.c M sys/arch/amd64/amd64/via.c M sys/arch/amd64/isa/clock.c M sys/arch/arm/arm/db_interface.c M sys/arch/arm/arm/pmap.c M sys/dev/acpi/acpi.c M sys/dev/cardbus/cardbus.c M sys/dev/ic/com.c M sys/dev/ic/elink3.c M sys/dev/ic/i82365.c M sys/dev/ic/vga.c M sys/dev/ic/vga_subr.c M sys/dev/isa/if_ef_isapnp.c M sys/dev/isa/pcppi.c M sys/dev/onewire/onewire_bitbang.c M sys/dev/pci/drm/drm_atomic.h M sys/dev/pci/drm/radeon/radeon_kms.c M sys/dev/pci/mbg.c M sys/dev/pci/pccbb.c M sys/dev/pci/pci.c M sys/dev/pci/pci_map.c M sys/dev/pci/pciide.c M sys/dev/pcmcia/if_xe.c M sys/dev/rasops/rasops.c M sys/dev/sdmmc/sdmmc_io.c M sys/dev/usb/ehci.c M sys/dev/usb/uhci.c M sys/dev/wsfont/wsfont.c M sys/kern/kern_clock.c M sys/kern/kern_event.c M sys/kern/kern_exec.c M sys/kern/kern_sensors.c M sys/kern/kern_sig.c M sys/kern/kern_time.c M sys/kern/subr_disk.c M sys/kern/subr_evcount.c M sys/kern/subr_log.c M sys/kern/subr_prf.c M sys/kern/subr_prof.c M sys/kern/sys_generic.c M sys/net/if_tun.c commit b13b0b4d8a06294ecb4e73dc83cf55391c62c574 diff: https://github.com/bitrig/bitrig/commit/b13b0b4 author: Christiano Haesbaert <[email protected]> date: Fri Jan 16 12:10:13 2015 +0100 New interrupt model, move away from IPLs. This diff changes the interrupt model to something very similar to what other modern unixes like Solaris, FreeBSD, DragonflyBSD and linux. It also introduces critical sections. Interrupts except clock and ipi are handled by interrupt threads, when an interrupt fires, the only job for the small interrupt stub is to schedule the corresponding interrupt thread, when the ithread gets scheduled, it services the interrupt. The kernel must be made preemptive, so that interrupt threads may preempt the current running code. In this model, you normally never block interrupts, you rely solely on locks to protect the code, if the ithread preempts the running code, and it tries to acquire a contested lock, it blocks (sleeps) and gets resumed only when that lock is released. This allows us to have lower interrupt latency, as instead of blocking interrupts for a long section, you can fine grain that section in a specific lock. Instead of raising to IPL_BIO and preventing softclock from running for example, we can make softclock preempt the old IPL_BIO and only block if there is an actual lock contention. This was first implemented in Solaris 2 (Kleiman & Eykholt circa 93), they demonstrated that this model, after properly locked, changed the worst case latency from 1s to <1ms on a single-core 40mhz machine. It also allows for us to properly fight livelocks in the future, as the scheduler will make sure "everything runs at some point". The kernel can now be synchronous with regard to locking, as you can use the same lock to interlock against interrupts or other normal kthreads. You can only block on locks if you have a process context, so you still need a way to block interrupts from normal (read: not ithread interrupts), like clock and ipi. For that we introduce critical sections, which blocks everything, in practice they're only used to protect scheduler data and on mutexes, so they are very short. In the future critical sections will also be the only thing that prevents kernel preemption. In order to prevent deadlocks, you must never be preempted while holding a spinlock, so a critical section is implied there, this is also akin to what every system does. In this present state, kernel preemption has not been implemented, all threads <IPL_CLOCK were moved to ithreads, there is no observable loss of performance, it's been stable since the last half a year. All spl calls <IPL_CLOCK were made no-ops, while >= IPL_CLOCK were replaced by critical sections. Most of the old assembly code has been rewritten in C, just because I refuse to maintain unecessary asm blocks. The next steps are: o Turn kernel lock into a rrwlock. o Enable kernel preemption, at this point all the interrupt interlocking will be done through kernel lock. o Decide on which subsystem to release first, having a wide subsytem lock. M sys/arch/amd64/amd64/cpu.c M sys/arch/amd64/amd64/fpu.c M sys/arch/amd64/amd64/genassym.cf M sys/arch/amd64/amd64/intr.c M sys/arch/amd64/amd64/ipi.c M sys/arch/amd64/amd64/lapic.c M sys/arch/amd64/amd64/locore.S M sys/arch/amd64/amd64/machdep.c M sys/arch/amd64/amd64/spl.S M sys/arch/amd64/amd64/trap.c M sys/arch/amd64/amd64/vector.S M sys/arch/amd64/conf/files.amd64 M sys/arch/amd64/include/cpu.h M sys/arch/amd64/include/frame.h M sys/arch/amd64/include/frameasm.h M sys/arch/amd64/include/intr.h M sys/arch/amd64/include/intrdefs.h M sys/conf/files M sys/dev/ic/com.c M sys/dev/ic/comvar.h M sys/dev/usb/ehci.c M sys/dev/usb/ohci.c M sys/dev/usb/uhci.c M sys/dev/usb/umidi.c M sys/dev/usb/usb.c M sys/dev/usb/usbdivar.h M sys/dev/usb/usbf_subr.c M sys/dev/usb/usbfvar.h M sys/dev/usb/xhci.c M sys/kern/init_main.c M sys/kern/kern_clock.c A sys/kern/kern_crit.c M sys/kern/kern_event.c M sys/kern/kern_fork.c A sys/kern/kern_ithread.c M sys/kern/kern_mutex.c M sys/kern/kern_proc.c M sys/kern/kern_resource.c M sys/kern/kern_sched.c M sys/kern/kern_sig.c M sys/kern/kern_synch.c M sys/kern/kern_time.c M sys/kern/kern_timeout.c M sys/kern/sched_bsd.c M sys/kern/subr_hibernate.c M sys/kern/subr_xxx.c M sys/kern/sys_generic.c M sys/kern/sys_process.c M sys/kern/vfs_subr.c M sys/kern/vfs_sync.c M sys/lib/libkern/libkern.h M sys/net/netisr.c M sys/net/netisr.h M sys/net/pipex.c A sys/sys/ithread.h M sys/sys/proc.h M sys/sys/sched.h M sys/sys/systm.h M sys/sys/timeout.h M sys/uvm/uvm_map.c
