On Thu, May 08, 2014 at 07:58:11PM +0100, Dr. David Alan Gilbert wrote: > * Stefan Hajnoczi (stefa...@gmail.com) wrote: > > On Thu, May 8, 2014 at 3:44 PM, Dr. David Alan Gilbert > > <dgilb...@redhat.com> wrote: > > > * Stefan Hajnoczi (stefa...@redhat.com) wrote: > > > > > > <snip> > > > > > >> How to synchronize with an IOThread > > >> ----------------------------------- > > >> AioContext is not thread-safe so some rules must be followed when using > > >> file > > >> descriptors, event notifiers, timers, or BHs across threads: > > >> > > >> 1. AioContext functions can be called safely from file descriptor, event > > >> notifier, timer, or BH callbacks invoked by the AioContext. No locking > > >> is > > >> necessary. > > >> > > >> 2. Other threads wishing to access the AioContext must use > > >> aio_context_acquire()/aio_context_release() for mutual exclusion. Once > > >> the > > >> context is acquired no other thread can access it or run event loop > > >> iterations > > >> in this AioContext. > > >> > > >> aio_context_acquire()/aio_context_release() calls may be nested. This > > >> means you can call them if you're not sure whether #1 applies. > > >> > > >> Side note: the best way to schedule a function call across threads is to > > >> create > > >> a BH in the target AioContext beforehand and then call > > >> qemu_bh_schedule(). No > > >> acquire/release or locking is needed for the qemu_bh_schedule() call. > > >> But be > > >> sure to acquire the AioContext for aio_bh_new() if necessary. > > > > > > How do these IOThreads pause during migration? > > > Are they paused by the 'qemu_mutex_lock_iothread' that the migration > > > thread calls? > > > > Currently the only IOThread user is virtio-blk data-plane. It has a > > VM state change listener registered that will stop using the IOThread > > during migration. > > > > In the future we'll have to do more than that: > > It is possible to suspend all IOThreads simply by looping over > > IOThread objects and calling aio_context_acquire() on their > > AioContext. You can release the AioContexts when you are done. This > > would be suitable for a "stop the world" operation for migration > > hand-over. > > That worries me for two reasons: > 1) I'm assuming there is some subtlety so that it doesn't deadlock when > another thread is trying to get a couple of contexts.
Only the main loop acquires contexts, that's why there is no lock ordering problem. > 2) The migration code that has to pause everything is reasonably time > critical (OK not super critical - but it worries if it gains more than > a few > ms). Doing something to each thread in series where that thread might > have to finish up a transaction sounds like it could add together to be > quite > large. It's no different from today where we need to bdrv_drain_all(); bdrv_flush_all(). That's a synchronous operation that can take a while. > > For smaller one-off operations like block-migration.c it may also make > > sense to acquire/release the AioContext. But that's not necessary > > today since dataplane is disabled during migration. > > I guess it's probably right to hide this behind some interface on the Aio > stuff > that migration can call and it can worry about speed, and locking order etc. > > I also would we end up wanting some IOThreads to continue - e.g. could we be > using > them for transport of the migration stream or are they strictly for the guests > use? IOThreads are just threads running AioContext event loops. They are generic and could be used for stuff I/O intensive stuff like migration or the VNC server. Stefan
