Hi - On 2017-03-14 at 15:40 Barret Rhoden <[email protected]> wrote: > So concretely, I'm currently planning:
[ a bunch of things ] > Anyway, that's my current plan. I imagine it'll change as I dig in > more. The minimum that we need to do is gcc's gthr.h interface, so I > might hold off on the pthread stuff until we get a better feel for > specific library code that is trying to use the pthread's API. That is pretty much done for now, and pretty close to the original plan. As expected, I also broke up part of pthread code and made it 2LS-independent, but there's still a bit more to do. I also got involved in glibc's locking and threading. The short version is that our port of glibc has two different types of locks. The libc_lock and _IO_lock are mutexes. The malloc lock is a spinlock. Vcore context (including interprocess signal handlers) can use malloc/free, but not anything that could block. Ultimately, this fixes a bunch of toolchain bugs with C++ code as well as intermittent problems/panics with blocking while holding locks. We also have implementations of a bunch of sync primitives that we either ignored (e.g. timed CVs, recursive mutexes) or faked (pthread RW locks were just mutexes). Hopefully I didn't break anyone's code. We'll see. You'll need to rebuild the world. A couple other notes, mostly for Gan: - benchutil/alarm.h is now parlib/alarm.h. - vmm task thread functions now return void *, like a pthread would. - if vcore context attempts to grab a mutex (which *could* block), it will panic. - panics and asserts spit out a backtrace on the monitor/console. - vcore context code *can* call fprintf, but only for stdout or stderr, where we'll macro-magic the call to a write() to FD 1 or 2. you'll need to #include <parlib/stdio.h> for this. Check out the commits for more info. Barret The following changes since commit 41ab54b78d870d46cdbc6100e9a96dcf64b82fae: Mark 'user' as PHONY in the top-level Makefile (2017-04-25 15:27:41 -0400) are available in the git repository at: [email protected]:brho/akaros.git uth for you to fetch changes up to 1f6e281e8bb5343138f7b21b4d63a2b092cda283: pthreads: Remove sched_policy and sched_priority (2017-04-28 16:00:53 -0400) ---------------------------------------------------------------- View this online at: https://github.com/brho/akaros/compare/41ab54b78d87...1f6e281e8bb5 ---------------------------------------------------------------- Barret Rhoden (57): parlib: Use 'const' in the set_dtls() interface parlib: Add trylock to uthread mutexes parlib: Add recursive uthread mutexes parlib: Add a helper for multi-threaded-ness ndblib: Remove dependency on pthreads Remove kernel-specific parts of ros/common.h (XCC) parlib: Move init_once_racy() to parlib (XCC) Convert run_once() to parlib_run_once() (XCC) parlib: Use cpu_relax_vc() in parlib_run_once() (XCC) parlib: Add synchronization objects parlib: Always use the default mutex/cv interface parlib: Change opaque mutex/cv types to be structs parlib: Add static initializers for mutexes / CVs parlib: Add a test for recursive mutexes parlib: Implement uthread mutexes with semaphores parlib: sem/mtx/cv: Add init() and destroy() parlib: Add uth_cond_var_wait_recurse() benchutil: Clarify unset_alarm()'s semantics parlib: Move alarm/timer functions from benchutil vmm: Remove references to pthreads parlib: Add an alarm helper parlib: Add 'timed' functions for sems/mtxs/cvs parlib: Implement join/detach() for all uthreads vmm: Make all task functions return void * parlib: Add uthread_create() parlib: Add uthread_sched_yield() Implement sched_yield() with uthread_sched_yield() (XCC) pthread: Implement POSIX semaphores with uth sems futex: Make futexes independent of pthreads parlib: UTH_EXT_BLK_JUSTICE -> _MISC parlib: Add uthread_self() Add the 'akaros' threading model to gcc (XCC) Use patches for existing toolchain files (XCC) parlib: Allow cpu_relax_vc() calls from uthreads parlib: Add cpu_relax_any() Add a shared_ptr sanity check parlib: Add reader-writer sleeping locks parlib: Trim some header includes Remove old tests and mhello parlib: Make sync objects static (XCC) Use weak __thread declarations in parlib-compat (XCC) Fix a couple warnings in glibc (XCC) parlib: Catch illegal block attempts parlib: Catch broken uses of notif_disabled_depth parlib: Set line buffering for stdout parlib: Improve panic() and assert() Have all asserts call parlib's assert (XCC) parlib: Make parlib printfs safe Overhaul glibc locking (XCC) parlib: Add a rwlock test parlib: Make uth mutex/CV initializers C++ safe pthread: Reimplement mutexes, CVs, and RW locks (XCC) parlib: Tease out uth_sync_t from has_blocked() parlib: Add __uth_sync_swap and __uth_sync_is_empty parlib: Add a thread_bulk_runnable() 2LS op pthread: Make pthread barriers 2LS-independent pthreads: Remove sched_policy and sched_priority kern/arch/x86/trap.c | 7 +- kern/include/common.h | 56 + kern/include/ros/common.h | 42 - tests/alarm.c | 2 +- tests/childfdmap.c | 2 +- tests/cpp_streams.cc | 9 + tests/evq_block.c | 2 +- tests/getifaddrs.c | 2 +- tests/hello.c | 2 +- tests/listen1.c | 2 +- tests/lock_test.c | 1 + tests/mcp_halt.c | 66 - tests/mhello.c | 179 - tests/microb_test.c | 1 + tests/old/appender.c | 40 - tests/old/arsc_mt.c | 52 - tests/old/arsc_test.c | 44 - tests/old/badsegment.c | 14 - tests/old/breakpoint.c | 10 - tests/old/buggyhello.c | 11 - tests/old/condvar_test.c | 154 - tests/old/divzero.c | 12 - tests/old/dtls_test.c | 62 - tests/old/eth_audio.c | 137 - tests/old/evilhello.c | 13 - tests/old/faultread.c | 10 - tests/old/faultreadkernel.c | 10 - tests/old/faultwrite.c | 8 - tests/old/faultwritekernel.c | 9 - tests/old/fork.c | 21 - tests/old/fp_test.c | 15 - tests/old/fpperf.cc | 108 - tests/old/hart_test.c | 6 - tests/old/idle.c | 23 - tests/old/manycore_test.c | 32 - tests/old/mproctests.c | 161 - tests/old/msr_cycling_vcores.c | 58 - tests/old/msr_dumb_while.c | 33 - tests/old/msr_get_cores.c | 114 - tests/old/msr_get_singlecore.c | 110 - tests/old/msr_nice_while.c | 39 - tests/old/msr_single_while.c | 10 - tests/old/null.c | 6 - tests/old/proctests.c | 32 - tests/old/raise.c | 77 - tests/old/slab.c | 38 - tests/old/softint.c | 16 - tests/old/spawn.c | 58 - tests/old/syscall.c | 132 - tests/old/syscall_speed.c | 18 - tests/old/tcp_test.c | 108 - tests/old/test_mmap_ipc.c | 43 - tests/old/test_ucq.c | 62 - tests/old/testbss.c | 29 - tests/old/tlstest.c | 15 - tests/old/tsc_spitter.c | 9 - tests/old/udp_test.c | 103 - tests/old/vvadd.c | 26 - tests/ping.c | 3 +- tests/pthread_barrier_test.c | 2 +- tests/pthread_test.c | 13 + tests/timerfd.c | 2 +- tests/vmm/vmrunkernel.c | 3 +- tests/xmm.c | 2 +- tools/apps/ipconfig/ipv6.c | 3 +- tools/apps/ipconfig/main.c | 2 +- tools/compilers/gcc-glibc/Makefile | 22 +- .../gcc-glibc/gcc-4.9.2-akaros/config.sub | 1793 - .../gcc-glibc/gcc-4.9.2-akaros/gcc/config.gcc | 4273 - .../gcc-glibc/gcc-4.9.2-akaros/libgcc/config.host | 1286 - .../gcc-4.9.2-akaros/libgcc/config/gthr-akaros.h | 228 + .../gcc-glibc/gcc-4.9.2-akaros/libgcc/crtstuff.c | 765 - .../gcc-4.9.2-akaros/libgomp/configure.tgt | 150 - .../gcc-4.9.2-akaros/libstdc++-v3/configure | 82676 ------------------- .../gcc-4.9.2-akaros/libstdc++-v3/configure.host | 370 - .../gcc-4.9.2-akaros/libstdc++-v3/crossconfig.m4 | 288 - tools/compilers/gcc-glibc/gcc-4.9.2-existing.patch | 435 + .../gcc-glibc/glibc-2.19-akaros/Makeconfig | 1093 - .../compilers/gcc-glibc/glibc-2.19-akaros/abi-tags | 34 - .../gcc-glibc/glibc-2.19-akaros/configure | 8863 -- .../gcc-glibc/glibc-2.19-akaros/configure.ac | 2192 - .../gcc-glibc/glibc-2.19-akaros/elf/Makefile | 1150 - .../gcc-glibc/glibc-2.19-akaros/include/alloca.h | 81 - .../gcc-glibc/glibc-2.19-akaros/include/errno.h | 52 - .../glibc-2.19-akaros/include/libc-symbols.h | 837 - .../glibc-2.19-akaros/nptl/shlib-versions | 8 - .../gcc-glibc/glibc-2.19-akaros/rt/Makefile | 75 - .../gcc-glibc/glibc-2.19-akaros/scripts/config.sub | 1793 - .../gcc-glibc/glibc-2.19-akaros/shlib-versions | 112 - .../glibc-2.19-akaros/stdio-common/vfprintf.c | 2384 - .../gcc-glibc/glibc-2.19-akaros/sunrpc/netname.c | 179 - .../glibc-2.19-akaros/sysdeps/akaros/Versions | 23 + .../sysdeps/akaros/bits/libc-lock.h | 410 +- .../sysdeps/akaros/bits/stdio-lock.h | 71 +- .../sysdeps/akaros/malloc-machine.h | 17 +- .../glibc-2.19-akaros/sysdeps/akaros/mmap.c | 2 +- .../sysdeps/akaros/parlib-compat.c | 193 +- .../glibc-2.19-akaros/sysdeps/akaros/sbrk.c | 14 +- .../glibc-2.19-akaros/sysdeps/akaros/sched_yield.c | 11 +- .../glibc-2.19-akaros/sysdeps/akaros/socket.c | 12 +- .../glibc-2.19-akaros/sysdeps/i386/configure | 244 - .../glibc-2.19-akaros/sysdeps/i386/configure.ac | 93 - .../glibc-2.19-akaros/sysdeps/i386/tlsdesc.sym | 17 - .../glibc-2.19-akaros/sysdeps/x86_64/tlsdesc.sym | 17 - .../compilers/gcc-glibc/glibc-2.19-existing.patch | 497 + tools/dev-util/perf/akaros.c | 7 +- user/iplib/epoll.c | 10 +- user/iplib/select.c | 10 +- user/ndblib/fcallfmt.c | 1 - user/ndblib/include/ndblib/ndb.h | 1 + user/{benchutil => parlib}/alarm.c | 14 +- user/parlib/asynccall.c | 2 + user/parlib/ceq.c | 2 +- user/parlib/debug.c | 20 +- user/parlib/dtls.c | 12 +- user/parlib/event.c | 5 + .../benchutil => parlib/include/parlib}/alarm.h | 13 +- user/parlib/include/parlib/assert.h | 5 - user/parlib/include/parlib/common.h | 2 - user/parlib/include/parlib/dtls.h | 2 +- user/parlib/include/parlib/parlib.h | 57 +- .../benchutil => parlib/include/parlib}/pvcalarm.h | 2 +- user/parlib/include/parlib/ros_debug.h | 20 +- user/parlib/include/parlib/slab.h | 1 - user/parlib/include/parlib/stdio.h | 55 + user/parlib/include/parlib/uthread.h | 226 +- user/parlib/include/parlib/vcore.h | 11 +- .../include/parlib}/vcore_tick.h | 0 user/parlib/mutex.c | 832 +- user/parlib/panic.c | 62 +- user/parlib/parlib.c | 1 + user/{benchutil => parlib}/pvcalarm.c | 15 +- user/parlib/signal.c | 1 + user/parlib/slab.c | 10 +- user/parlib/syscall.c | 7 +- user/parlib/thread0_sched.c | 99 +- user/parlib/ucq.c | 3 +- user/parlib/uthread.c | 396 +- user/parlib/vcore.c | 27 +- user/{benchutil => parlib}/vcore_tick.c | 4 +- user/parlib/x86/vcore.c | 2 +- user/pthread/futex.c | 23 +- user/pthread/profalarm.c | 2 +- user/pthread/pthread.c | 639 +- user/pthread/pthread.h | 108 +- user/pthread/semaphore.c | 172 +- user/pthread/semaphore.h | 42 +- user/utest/alarm.c | 2 +- user/utest/cv.c | 267 +- user/utest/pvcalarm.c | 3 +- user/vmm/apic.c | 1 - user/vmm/decode.c | 1 - user/vmm/include/vmm/sched.h | 8 +- user/vmm/include/vmm/virtio.h | 2 +- user/vmm/include/vmm/virtio_blk.h | 2 +- user/vmm/include/vmm/virtio_lguest_console.h | 4 +- user/vmm/include/vmm/virtio_net.h | 4 +- user/vmm/include/vmm/vmm.h | 1 + user/vmm/io.c | 1 - user/vmm/ioapic.c | 1 - user/vmm/memory.c | 1 + user/vmm/nat.c | 10 +- user/vmm/sched.c | 74 +- user/vmm/virtio_blk.c | 3 +- user/vmm/virtio_lguest_console.c | 7 +- user/vmm/virtio_net.c | 6 +- user/vmm/vmx.c | 1 - user/vmm/vmxmsr.c | 1 - 168 files changed, 3938 insertions(+), 114636 deletions(-) delete mode 100644 tests/mcp_halt.c delete mode 100644 tests/mhello.c delete mode 100644 tests/old/appender.c delete mode 100644 tests/old/arsc_mt.c delete mode 100644 tests/old/arsc_test.c delete mode 100644 tests/old/badsegment.c delete mode 100644 tests/old/breakpoint.c delete mode 100644 tests/old/buggyhello.c delete mode 100644 tests/old/condvar_test.c delete mode 100644 tests/old/divzero.c delete mode 100644 tests/old/dtls_test.c delete mode 100644 tests/old/eth_audio.c delete mode 100644 tests/old/evilhello.c delete mode 100644 tests/old/faultread.c delete mode 100644 tests/old/faultreadkernel.c delete mode 100644 tests/old/faultwrite.c delete mode 100644 tests/old/faultwritekernel.c delete mode 100644 tests/old/fork.c delete mode 100644 tests/old/fp_test.c delete mode 100644 tests/old/fpperf.cc delete mode 100644 tests/old/hart_test.c delete mode 100644 tests/old/idle.c delete mode 100644 tests/old/manycore_test.c delete mode 100644 tests/old/mproctests.c delete mode 100644 tests/old/msr_cycling_vcores.c delete mode 100644 tests/old/msr_dumb_while.c delete mode 100644 tests/old/msr_get_cores.c delete mode 100644 tests/old/msr_get_singlecore.c delete mode 100644 tests/old/msr_nice_while.c delete mode 100644 tests/old/msr_single_while.c delete mode 100644 tests/old/null.c delete mode 100644 tests/old/proctests.c delete mode 100644 tests/old/raise.c delete mode 100644 tests/old/slab.c delete mode 100644 tests/old/softint.c delete mode 100644 tests/old/spawn.c delete mode 100644 tests/old/syscall.c delete mode 100644 tests/old/syscall_speed.c delete mode 100644 tests/old/tcp_test.c delete mode 100644 tests/old/test_mmap_ipc.c delete mode 100644 tests/old/test_ucq.c delete mode 100644 tests/old/testbss.c delete mode 100644 tests/old/tlstest.c delete mode 100644 tests/old/tsc_spitter.c delete mode 100644 tests/old/udp_test.c delete mode 100644 tests/old/vvadd.c delete mode 100755 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/config.sub delete mode 100644 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/gcc/config.gcc delete mode 100644 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/libgcc/config.host create mode 100644 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/libgcc/config/gthr-akaros.h delete mode 100644 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/libgcc/crtstuff.c delete mode 100644 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/libgomp/configure.tgt delete mode 100755 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/libstdc++-v3/configure delete mode 100644 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/libstdc++-v3/configure.host delete mode 100644 tools/compilers/gcc-glibc/gcc-4.9.2-akaros/libstdc++-v3/crossconfig.m4 create mode 100644 tools/compilers/gcc-glibc/gcc-4.9.2-existing.patch delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/Makeconfig delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/abi-tags delete mode 100755 tools/compilers/gcc-glibc/glibc-2.19-akaros/configure delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/configure.ac delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/elf/Makefile delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/include/alloca.h delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/include/errno.h delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/include/libc-symbols.h delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/nptl/shlib-versions delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/rt/Makefile delete mode 100755 tools/compilers/gcc-glibc/glibc-2.19-akaros/scripts/config.sub delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/shlib-versions delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/stdio-common/vfprintf.c delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/sunrpc/netname.c delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/sysdeps/i386/configure delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/sysdeps/i386/configure.ac delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/sysdeps/i386/tlsdesc.sym delete mode 100644 tools/compilers/gcc-glibc/glibc-2.19-akaros/sysdeps/x86_64/tlsdesc.sym create mode 100644 tools/compilers/gcc-glibc/glibc-2.19-existing.patch rename user/{benchutil => parlib}/alarm.c (98%) rename user/{benchutil/include/benchutil => parlib/include/parlib}/alarm.h (91%) rename user/{benchutil/include/benchutil => parlib/include/parlib}/pvcalarm.h (99%) rename user/{benchutil/include/benchutil => parlib/include/parlib}/vcore_tick.h (100%) rename user/{benchutil => parlib}/pvcalarm.c (97%) rename user/{benchutil => parlib}/vcore_tick.c (98%) -- You received this message because you are subscribed to the Google Groups "Akaros" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/d/optout.
