Re: Understanding CPU and memory usage in Bhyve
Peter Grehan wrote: > Hi Roman, > > > But the summary is: is there a way to figure out how much CPU time > > bhyve and the guest spends on host CPUs N (N = 0, 1, ...)? > > I don't think FreeBSD records that for a process, and bhyve doesn't > record the guest vCPU time on individual host CPUs (it's an aggregate > over all CPUs). > > Do you know if Linux supports that ? As far as I understand, Linux supports that through Cgroups, specifically, quoting: http://docs.fedoraproject.org/en-US/Fedora/15/html/Resource_Management_Guide/sec-cpuacct.html cpuacct.usage_percpu reports the CPU time (in nanoseconds) consumed on each CPU by all tasks in this cgroup (including tasks lower in the hierarchy). Roman Bogorodskiy pgppooiAZ8GBU.pgp Description: PGP signature
Re: Understanding CPU and memory usage in Bhyve
Hi Roman, But the summary is: is there a way to figure out how much CPU time bhyve and the guest spends on host CPUs N (N = 0, 1, ...)? I don't think FreeBSD records that for a process, and bhyve doesn't record the guest vCPU time on individual host CPUs (it's an aggregate over all CPUs). Do you know if Linux supports that ? later, ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Understanding CPU and memory usage in Bhyve
Peter Grehan wrote: > Hi Roman, > > > For the CPU, there's some info provided by bhyvectl, e.g.: > > > > /usr/sbin/bhyvectl --vm=bhyve --get-stats --cpu=1 > > > > in the field 'vcpu total runtime', for example: > > > > vcpu total runtime 8178870653 > > > > Two question about that: > > > > 1. What are the units used here? Looks like it's ticks, so if I want to > > convert this number, to, say, nanoseconds, would it be right to do > > something like: > > > > X * 1000**3 / kern.clockrate.stathz > > > > (X beeing the value returned by bhyvectl and kern.clockrate is a > > sysctl name)? > > It's in TSC units so you can convert to nsecs using the > machdep.tsc_freq sysctl node. > > > 2. This value shows only vcpu time, without hypervisor time? > > That's correct. To get the hypervisor and vCPU time together, you can > use existing CPU usage reporting (e.g. that used by top et al). Thanks for the clarification, it's very helpful. However, I need some more help with that. I'm trying to implement some domain stats reporting for the bhyve driver in libvirt and it looks like I choose a wrong approach initially. It's described here: https://www.redhat.com/archives/libvir-list/2014-April/msg00912.html But the summary is: is there a way to figure out how much CPU time bhyve and the guest spends on host CPUs N (N = 0, 1, ...)? > > For the memory stats, it's less obvious. Does guest always use an amount > > of memory specified at startup with '-m' (i.e. bhyve -m 1G) or it could > > use less? > > That's the maximum it can use. We currently don't expose the incore > amount from the vmspace representing the guest's physical - that should > be added to bhyvectl soon. Looking forward to that. > > What is the format of bhyvectl --get-{low,high}mem output? > > The number reported there isn't to do with guest usage, but how the > memory is divided between < 4G and above 4G. For example, the default > for bhyve is to use up to 3G below 4G, provide 1GB for PCI MMIO decode, > and then put the remainder of guest memory above 4G. Here's an example > for an 8G VM, with 3G of guest mem below 4G (starting address 0), and 5G > above 4G (starting address 4G): > > lowmem 0x/3221225472 > highmem 0x0001/5368709120 > > > PS I found it a little confusing that bhyvectl displays vcpu0 stats by > > default if --cpu is not provided, expected it provide info of all vcpus > > in the guest by default. > > Yes, that's a relic of the days when there was only a single vCPU > supported :( > > > Also, didn't find a way to get a number of vcpus in a running guest. > > Until the vCPU state is exposed by bhyvectl, or we provide a sysctl, > you can use heuristics: the number of vCPU threads for the bhyve > process, or scan all vCPUs and only count those that have a non-zero RIP. > > later, > > Peter. > Thanks, Roman Bogorodskiy pgpXkSY0G17fG.pgp Description: PGP signature
Re: Understanding CPU and memory usage in Bhyve
Hi Roman, For the CPU, there's some info provided by bhyvectl, e.g.: /usr/sbin/bhyvectl --vm=bhyve --get-stats --cpu=1 in the field 'vcpu total runtime', for example: vcpu total runtime 8178870653 Two question about that: 1. What are the units used here? Looks like it's ticks, so if I want to convert this number, to, say, nanoseconds, would it be right to do something like: X * 1000**3 / kern.clockrate.stathz (X beeing the value returned by bhyvectl and kern.clockrate is a sysctl name)? It's in TSC units so you can convert to nsecs using the machdep.tsc_freq sysctl node. 2. This value shows only vcpu time, without hypervisor time? That's correct. To get the hypervisor and vCPU time together, you can use existing CPU usage reporting (e.g. that used by top et al). For the memory stats, it's less obvious. Does guest always use an amount of memory specified at startup with '-m' (i.e. bhyve -m 1G) or it could use less? That's the maximum it can use. We currently don't expose the incore amount from the vmspace representing the guest's physical - that should be added to bhyvectl soon. What is the format of bhyvectl --get-{low,high}mem output? The number reported there isn't to do with guest usage, but how the memory is divided between < 4G and above 4G. For example, the default for bhyve is to use up to 3G below 4G, provide 1GB for PCI MMIO decode, and then put the remainder of guest memory above 4G. Here's an example for an 8G VM, with 3G of guest mem below 4G (starting address 0), and 5G above 4G (starting address 4G): lowmem 0x/3221225472 highmem 0x0001/5368709120 PS I found it a little confusing that bhyvectl displays vcpu0 stats by default if --cpu is not provided, expected it provide info of all vcpus in the guest by default. Yes, that's a relic of the days when there was only a single vCPU supported :( Also, didn't find a way to get a number of vcpus in a running guest. Until the vCPU state is exposed by bhyvectl, or we provide a sysctl, you can use heuristics: the number of vCPU threads for the bhyve process, or scan all vCPUs and only count those that have a non-zero RIP. later, Peter. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Understanding CPU and memory usage in Bhyve
Hi, I'm trying to understand how to obtain resource usage information for a bhyve guest. For the CPU, there's some info provided by bhyvectl, e.g.: /usr/sbin/bhyvectl --vm=bhyve --get-stats --cpu=1 in the field 'vcpu total runtime', for example: vcpu total runtime 8178870653 Two question about that: 1. What are the units used here? Looks like it's ticks, so if I want to convert this number, to, say, nanoseconds, would it be right to do something like: X * 1000**3 / kern.clockrate.stathz (X beeing the value returned by bhyvectl and kern.clockrate is a sysctl name)? 2. This value shows only vcpu time, without hypervisor time? For the memory stats, it's less obvious. Does guest always use an amount of memory specified at startup with '-m' (i.e. bhyve -m 1G) or it could use less? What is the format of bhyvectl --get-{low,high}mem output? PS I found it a little confusing that bhyvectl displays vcpu0 stats by default if --cpu is not provided, expected it provide info of all vcpus in the guest by default. Also, didn't find a way to get a number of vcpus in a running guest. Roman Bogorodskiy pgpdf09dm2nHF.pgp Description: PGP signature