Re: [Xen-devel] libxl - avoid calling block script
Marek Marczykowski-Górecki writes ("Re: [Xen-devel] libxl - avoid calling block
script"):
> On Fri, Feb 09, 2018 at 11:03:55AM +, Roger Pau Monné wrote:
> > Really adding Ian and Wei.
> >
> > On Fri, Feb 09, 2018 at 10:55:24AM +, Roger Pau Monné wrote:
> > > So the problem is creation time for domains that have quite a lot of
> > > disks attached. Adding Ian and Wei who know more about the async
> > > dispatch system, but I think (at least from a technical PoV) it
> > > should be possible to parallelize device attachment and thus hotplug
> > > script execution. Devices are independent from each other.
>
> In theory yes, but in practice block script (at least on Linux) takes a
> lock and serialize execution...
Indeed.
> > > Also the Linux hotplug scripts in general seem extremely convoluted,
> > > I'm not sure whether we could gain some speed there just by
> > > simplification.
>
> Well, we're comparing a bunch of fork+exec(), including starting bash
> (default /bin/sh on most systems), with just a single stat() call...
> Handling scripts in libxl itself also takes some time (in my case libxl
> live in libvirt, which may or may not have an impact). For a domU with
> 4 disks, getting rid of hotplug scripts saved about 2s of startup time.
The scripts themselves are terribly terribly slow. They are as Roger
says incredibly convoluted. I'm sure your 2s is right, but almost all
of that will be actual script execution.
I am not opposed to moving the functionality for very simplest case
into libxl.
But I think from your pov it would be worth trying a simple shell
script which doesn't take a lock, but just provides the physical
device information.
Ian.
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] libxl - avoid calling block script
On Fri, Feb 23, 2018 at 09:14:03PM +0100, Marek Marczykowski-Górecki wrote: > On Fri, Feb 23, 2018 at 06:28:56PM +, Wei Liu wrote: > > On Fri, Feb 09, 2018 at 12:35:13PM +0100, Marek Marczykowski-Górecki wrote: > > > On Fri, Feb 09, 2018 at 11:03:55AM +, Roger Pau Monné wrote: > > > > Really adding Ian and Wei. > > > > > > > > On Fri, Feb 09, 2018 at 10:55:24AM +, Roger Pau Monné wrote: > > > > > So the problem is creation time for domains that have quite a lot of > > > > > disks attached. Adding Ian and Wei who know more about the async > > > > > dispatch system, but I think (at least from a technical PoV) it > > > > > should be possible to parallelize device attachment and thus hotplug > > > > > script execution. Devices are independent from each other. > > > > > > In theory yes, but in practice block script (at least on Linux) takes a > > > lock and serialize execution... > > > > > > > > Also the Linux hotplug scripts in general seem extremely convoluted, > > > > > I'm not sure whether we could gain some speed there just by > > > > > simplification. > > > > > > Well, we're comparing a bunch of fork+exec(), including starting bash > > > (default /bin/sh on most systems), with just a single stat() call... > > > Handling scripts in libxl itself also takes some time (in my case libxl > > > live in libvirt, which may or may not have an impact). For a domU with > > > 4 disks, getting rid of hotplug scripts saved about 2s of startup time. > > > > > > > Sorry for the late reply. > > > > If you really don't want block scripts, can you not specify a script > > that only does "exit 0"? That seems to be easier than modifying libxl > > and it is also useable in older versions of Xen. > > But this is only one part of the picture. Something needs to set > physical-device xenstore entry. Libxl did that before, but it was > removed (see original message in this thread). I may write alternative > simplified block script for such case and measure performance of it, but > this feels overly complex, especially when libxl already have everything > it needs to quickly fill that xenstore entry. Keep in mind hotplug scripts can be written in any language, they don't need to be shell scripts. They are written in shell ATM because it's easier to modify, but I think it would be perfectly normal to instead have binary executables (ie: written in C) for performance reasons. Roger. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] libxl - avoid calling block script
On Fri, Feb 23, 2018 at 06:28:56PM +, Wei Liu wrote: > On Fri, Feb 09, 2018 at 12:35:13PM +0100, Marek Marczykowski-Górecki wrote: > > On Fri, Feb 09, 2018 at 11:03:55AM +, Roger Pau Monné wrote: > > > Really adding Ian and Wei. > > > > > > On Fri, Feb 09, 2018 at 10:55:24AM +, Roger Pau Monné wrote: > > > > So the problem is creation time for domains that have quite a lot of > > > > disks attached. Adding Ian and Wei who know more about the async > > > > dispatch system, but I think (at least from a technical PoV) it > > > > should be possible to parallelize device attachment and thus hotplug > > > > script execution. Devices are independent from each other. > > > > In theory yes, but in practice block script (at least on Linux) takes a > > lock and serialize execution... > > > > > > Also the Linux hotplug scripts in general seem extremely convoluted, > > > > I'm not sure whether we could gain some speed there just by > > > > simplification. > > > > Well, we're comparing a bunch of fork+exec(), including starting bash > > (default /bin/sh on most systems), with just a single stat() call... > > Handling scripts in libxl itself also takes some time (in my case libxl > > live in libvirt, which may or may not have an impact). For a domU with > > 4 disks, getting rid of hotplug scripts saved about 2s of startup time. > > > > Sorry for the late reply. > > If you really don't want block scripts, can you not specify a script > that only does "exit 0"? That seems to be easier than modifying libxl > and it is also useable in older versions of Xen. But this is only one part of the picture. Something needs to set physical-device xenstore entry. Libxl did that before, but it was removed (see original message in this thread). I may write alternative simplified block script for such case and measure performance of it, but this feels overly complex, especially when libxl already have everything it needs to quickly fill that xenstore entry. Anyway, I'll go that way for now. -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? signature.asc Description: PGP signature ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] libxl - avoid calling block script
On Fri, Feb 09, 2018 at 12:35:13PM +0100, Marek Marczykowski-Górecki wrote: > On Fri, Feb 09, 2018 at 11:03:55AM +, Roger Pau Monné wrote: > > Really adding Ian and Wei. > > > > On Fri, Feb 09, 2018 at 10:55:24AM +, Roger Pau Monné wrote: > > > So the problem is creation time for domains that have quite a lot of > > > disks attached. Adding Ian and Wei who know more about the async > > > dispatch system, but I think (at least from a technical PoV) it > > > should be possible to parallelize device attachment and thus hotplug > > > script execution. Devices are independent from each other. > > In theory yes, but in practice block script (at least on Linux) takes a > lock and serialize execution... > > > > Also the Linux hotplug scripts in general seem extremely convoluted, > > > I'm not sure whether we could gain some speed there just by > > > simplification. > > Well, we're comparing a bunch of fork+exec(), including starting bash > (default /bin/sh on most systems), with just a single stat() call... > Handling scripts in libxl itself also takes some time (in my case libxl > live in libvirt, which may or may not have an impact). For a domU with > 4 disks, getting rid of hotplug scripts saved about 2s of startup time. > Sorry for the late reply. If you really don't want block scripts, can you not specify a script that only does "exit 0"? That seems to be easier than modifying libxl and it is also useable in older versions of Xen. Wei. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] libxl - avoid calling block script
On Fri, Feb 09, 2018 at 11:03:55AM +, Roger Pau Monné wrote: > Really adding Ian and Wei. > > On Fri, Feb 09, 2018 at 10:55:24AM +, Roger Pau Monné wrote: > > So the problem is creation time for domains that have quite a lot of > > disks attached. Adding Ian and Wei who know more about the async > > dispatch system, but I think (at least from a technical PoV) it > > should be possible to parallelize device attachment and thus hotplug > > script execution. Devices are independent from each other. In theory yes, but in practice block script (at least on Linux) takes a lock and serialize execution... > > Also the Linux hotplug scripts in general seem extremely convoluted, > > I'm not sure whether we could gain some speed there just by > > simplification. Well, we're comparing a bunch of fork+exec(), including starting bash (default /bin/sh on most systems), with just a single stat() call... Handling scripts in libxl itself also takes some time (in my case libxl live in libvirt, which may or may not have an impact). For a domU with 4 disks, getting rid of hotplug scripts saved about 2s of startup time. -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? signature.asc Description: PGP signature ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] libxl - avoid calling block script
Really adding Ian and Wei. On Fri, Feb 09, 2018 at 10:55:24AM +, Roger Pau Monné wrote: > On Fri, Feb 09, 2018 at 02:02:42AM +0100, Marek Marczykowski-Górecki wrote: > > Hi, > > > > I'd like to avoid calling block script to speed up domain startup a > > little (there may be multiple disks, all already being block devices). > > Right now I have restored setting physical-device xenstore entry in > > libxl (by reverting [1]), > > This will work for Linux, but will break for FreeBSD, which expects > physical-device-path instead of physical-device as the output of > hotplug scripts. > > > then applying the patch below (it's on 4.8). > > This works well for my case, but maybe there is some option to have it > > in vanilla Xen? Right now, this require explicit "script=block" to call > > the script (for example to setup loop device). > > > > Alternative idea I have is setting disk->script="block" early > > (in libxl_device_disk_init()?), so default do not change, but it's still > > possible to change it to NULL and avoid calling the script. The problem > > is libxl_device_disk_init() is a generated and I don't see how it could > > be modified... Any hints? > > > > Yet another idea is having some specific value for disk->script, > > that would avoid calling it, but I find this much less elegant solution. > > So the problem is creation time for domains that have quite a lot of > disks attached. Adding Ian and Wei who know more about the async > dispatch system, but I think (at least from a technical PoV) it > should be possible to parallelize device attachment and thus hotplug > script execution. Devices are independent from each other. > > Also the Linux hotplug scripts in general seem extremely convoluted, > I'm not sure whether we could gain some speed there just by > simplification. > > Thanks, Roger. > > ___ > Xen-devel mailing list > Xen-devel@lists.xenproject.org > https://lists.xenproject.org/mailman/listinfo/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] libxl - avoid calling block script
On Fri, Feb 09, 2018 at 02:02:42AM +0100, Marek Marczykowski-Górecki wrote: > Hi, > > I'd like to avoid calling block script to speed up domain startup a > little (there may be multiple disks, all already being block devices). > Right now I have restored setting physical-device xenstore entry in > libxl (by reverting [1]), This will work for Linux, but will break for FreeBSD, which expects physical-device-path instead of physical-device as the output of hotplug scripts. > then applying the patch below (it's on 4.8). > This works well for my case, but maybe there is some option to have it > in vanilla Xen? Right now, this require explicit "script=block" to call > the script (for example to setup loop device). > > Alternative idea I have is setting disk->script="block" early > (in libxl_device_disk_init()?), so default do not change, but it's still > possible to change it to NULL and avoid calling the script. The problem > is libxl_device_disk_init() is a generated and I don't see how it could > be modified... Any hints? > > Yet another idea is having some specific value for disk->script, > that would avoid calling it, but I find this much less elegant solution. So the problem is creation time for domains that have quite a lot of disks attached. Adding Ian and Wei who know more about the async dispatch system, but I think (at least from a technical PoV) it should be possible to parallelize device attachment and thus hotplug script execution. Devices are independent from each other. Also the Linux hotplug scripts in general seem extremely convoluted, I'm not sure whether we could gain some speed there just by simplification. Thanks, Roger. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
