On Wed, Apr 17, 2024 at 12:42:42PM +0100, Daniel P. Berrangé wrote: > On Tue, Apr 16, 2024 at 10:22:32PM +0100, Richard W.M. Jones wrote: > > This has not worked for a long time because of a fundamental bug in > > how Ruby's garbage collector interacts with pthread stacks. There has > > been a bug open for a long time (over 14 years) which has never been > > fixed: > > > > https://redmine.ruby-lang.org/issues/2294#note-34 > > > > As that discussion is rather hard to follow, the basic problem is > > this: > > > > When you embed Ruby you must tell the interpreter where "the stack" > > is. You do this by calling RUBY_INIT_STACK early on (which we do in > > load(), on the main thread). This marks the top of stack. > > > > From time to time Ruby's garbage collector will scan the stack > > starting at the current stack pointer, and going up til it reaches the > > previously marked top of stack. It scans this memory looking for > > roots etc. > > > > This strategy fails completely if you have threads. In a thread > > (using a different stack entirely) it scans randomly across the heap, > > resulting in weird behaviour at best, but more usually straight > > crashes. > > Urgh, horrible. > > > > > We've had a Ruby plugin for a long time, but it broke with Ruby 1.9, > > when Ruby changed from some kind of cooperative threading to using > > native threads (and therefore separate stacks). > > > > The tests have been disabled since early 2018 (nbdkit < 1.2). I > > re-enabled the tests just now (Ruby 3.3) but it's broken in the same > > way as ever. > > > > The only changes that have happened since the tests were disabled are > > mechanical ones / project-wide cleanups. > > > > It currently crashes in open(). > > > > It seems highly unlikely that anyone is using this plugin. > > IIUC, the '--threads' arg is just a per-client limit. So if > there are multiple clients, you'll still get many threads > using ruby even with "--threads = 1".
Yes, and in fact you'll get an extra thread even with one client. Using a background thread outside of nbdkit's control and funnelling requests through that (see my follow up) seems to be the only way. > If spawning nbdkit with --single though, you'll get a separate > process per client, and then with --threads=1, you could can > sure that only 1 thread is calling into ruby. True, although --single is of very limited use. Basically it forces you to use xinetd or systemd or libnbd's nbd_connect_command. > Is there a way a plugin can declare that it is non-threadsafe, > and thus allow nbdkit to refuse to run it without --single > and --threads=1, being set ? It's possible, but the use case would be so limiting that I'm not sure if the cure is better than the disease. > I guess that will complicate your test suite though unless > you can skip all tests which rely on multi-connections and > threads. Rich. > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html _______________________________________________ Libguestfs mailing list -- guestfs@lists.libguestfs.org To unsubscribe send an email to guestfs-le...@lists.libguestfs.org
