Re: type of vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin, vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout on AMD64 r320730
On Mon, Jul 10, 2017 at 10:51:43AM -0300, Otac??lio wrote: > Em 10/07/2017 06:49, Konstantin Belousov escreveu: > > On Mon, Jul 10, 2017 at 06:45:28PM +0900, Tomoaki AOKI wrote: > >> Hm, for example, sysutils/lsof (userspace app) depends on kernel > >> source, and I thought the one Otacilio mentioned is something like it. > > lsof purpose is to dig into a kernel memory to gather information about > > the process state and opened files. To do this, lsof authors have to use > > internal kernel data structures. > > > > The library which is discussed in the thread uses public interfaces, > > but then tries to pack the returned data into internal kernel structure. > > If this is true, then the breakage is on the library side. > > Hello > > This following simple program behaves in the same manner that xosview. > It shows corrects answers for pages_in and pages_out on FreeBSD 11 amd64 > and FreeBSD10.3 i386. On FreeBSD 12 r320730 it shows the error answer > that I related previously. If the lines > > pageinfo[0] = (uint64_t)vm.v_vnodepgsin + (uint64_t)vm.v_swappgsin; > pageinfo[1] = (uint64_t)vm.v_vnodepgsout + (uint64_t)vm.v_swappgsout; > > are patched to > > pageinfo[0] = (uint32_t)((uint64_t)vm.v_vnodepgsin + > (uint64_t)vm.v_swappgsin); > pageinfo[1] = (uint32_t)((uint64_t)vm.v_vnodepgsout + > (uint64_t)vm.v_swappgsout); > > The program works on all FreeBSD version. My doubt is why now this type > cast (uint32_t) is necessary? > > []'s > > -Otacilio > > #include > #include > #include > > #define _WANT_VMMETER > > #include > > /* meminfo[5] = { active, inactive, wired, cached, free } */ > /* pageinfo[2] = { pages_in, pages_out } */ > void BSDGetPageStats(uint64_t *meminfo, uint64_t *pageinfo) { > struct vmmeter vm; > size_t size = sizeof(unsigned int); > #defineGET_VM_STATS(name) sysctlbyname("vm.stats.vm." #name, > &vm.name, &size, NULL, 0) Use of struct vmmeter is wrong, as discussed. The structure is for kernel internal interfaces. The behaviour changed because the structure member was of type uint32_t but become a pointer. You initialize the size of the storage to receive MIB value to sizeof(int), which leaves half of the pointer storage not filled. Stop using vmmeter, use correct types for the MIBs values, and pass correct size to sysctl(3) call. > GET_VM_STATS(v_active_count); > GET_VM_STATS(v_inactive_count); > GET_VM_STATS(v_wire_count); > GET_VM_STATS(v_free_count); > GET_VM_STATS(v_page_size); > GET_VM_STATS(v_vnodepgsin); > GET_VM_STATS(v_vnodepgsout); > GET_VM_STATS(v_swappgsin); > GET_VM_STATS(v_swappgsout); > > if (meminfo) { > meminfo[0] = (uint64_t)vm.v_active_count * vm.v_page_size; > meminfo[1] = (uint64_t)vm.v_inactive_count * vm.v_page_size; > meminfo[2] = (uint64_t)vm.v_wire_count * vm.v_page_size; > meminfo[4] = (uint64_t)vm.v_free_count * vm.v_page_size; > } > if (pageinfo) { > pageinfo[0] = (uint64_t)vm.v_vnodepgsin + (uint64_t)vm.v_swappgsin; > pageinfo[1] = (uint64_t)vm.v_vnodepgsout + > (uint64_t)vm.v_swappgsout; > } > } > > int main(int argc, char **argv){ > uint64_t meminfo[5]; > uint64_t pageinfo[2]; > > BSDGetPageStats(meminfo, pageinfo); > > printf("active memory = %lu bytes\n", meminfo[0]); > printf("inactive memory = %lu bytes\n", meminfo[1]); > printf("wired memory = %lu bytes\n", meminfo[2]); > printf("cached memory = %lu bytes\n", meminfo[3]); > printf("free memory = %lu bytes\n", meminfo[4]); > > printf("\npages_in= %lu bytes\n", pageinfo[0]); > printf("pages_out= %lu bytes\n", pageinfo[1]); > return 0; > } ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: type of vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin, vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout on AMD64 r320730
Em 10/07/2017 06:49, Konstantin Belousov escreveu: On Mon, Jul 10, 2017 at 06:45:28PM +0900, Tomoaki AOKI wrote: Hm, for example, sysutils/lsof (userspace app) depends on kernel source, and I thought the one Otacilio mentioned is something like it. lsof purpose is to dig into a kernel memory to gather information about the process state and opened files. To do this, lsof authors have to use internal kernel data structures. The library which is discussed in the thread uses public interfaces, but then tries to pack the returned data into internal kernel structure. If this is true, then the breakage is on the library side. Hello This following simple program behaves in the same manner that xosview. It shows corrects answers for pages_in and pages_out on FreeBSD 11 amd64 and FreeBSD10.3 i386. On FreeBSD 12 r320730 it shows the error answer that I related previously. If the lines pageinfo[0] = (uint64_t)vm.v_vnodepgsin + (uint64_t)vm.v_swappgsin; pageinfo[1] = (uint64_t)vm.v_vnodepgsout + (uint64_t)vm.v_swappgsout; are patched to pageinfo[0] = (uint32_t)((uint64_t)vm.v_vnodepgsin + (uint64_t)vm.v_swappgsin); pageinfo[1] = (uint32_t)((uint64_t)vm.v_vnodepgsout + (uint64_t)vm.v_swappgsout); The program works on all FreeBSD version. My doubt is why now this type cast (uint32_t) is necessary? []'s -Otacilio #include #include #include #define _WANT_VMMETER #include /* meminfo[5] = { active, inactive, wired, cached, free } */ /* pageinfo[2] = { pages_in, pages_out } */ void BSDGetPageStats(uint64_t *meminfo, uint64_t *pageinfo) { struct vmmeter vm; size_t size = sizeof(unsigned int); #defineGET_VM_STATS(name) sysctlbyname("vm.stats.vm." #name, &vm.name, &size, NULL, 0) GET_VM_STATS(v_active_count); GET_VM_STATS(v_inactive_count); GET_VM_STATS(v_wire_count); GET_VM_STATS(v_free_count); GET_VM_STATS(v_page_size); GET_VM_STATS(v_vnodepgsin); GET_VM_STATS(v_vnodepgsout); GET_VM_STATS(v_swappgsin); GET_VM_STATS(v_swappgsout); if (meminfo) { meminfo[0] = (uint64_t)vm.v_active_count * vm.v_page_size; meminfo[1] = (uint64_t)vm.v_inactive_count * vm.v_page_size; meminfo[2] = (uint64_t)vm.v_wire_count * vm.v_page_size; meminfo[4] = (uint64_t)vm.v_free_count * vm.v_page_size; } if (pageinfo) { pageinfo[0] = (uint64_t)vm.v_vnodepgsin + (uint64_t)vm.v_swappgsin; pageinfo[1] = (uint64_t)vm.v_vnodepgsout + (uint64_t)vm.v_swappgsout; } } int main(int argc, char **argv){ uint64_t meminfo[5]; uint64_t pageinfo[2]; BSDGetPageStats(meminfo, pageinfo); printf("active memory = %lu bytes\n", meminfo[0]); printf("inactive memory = %lu bytes\n", meminfo[1]); printf("wired memory = %lu bytes\n", meminfo[2]); printf("cached memory = %lu bytes\n", meminfo[3]); printf("free memory = %lu bytes\n", meminfo[4]); printf("\npages_in= %lu bytes\n", pageinfo[0]); printf("pages_out= %lu bytes\n", pageinfo[1]); return 0; } ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: type of vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin, vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout on AMD64 r320730
On Mon, Jul 10, 2017 at 06:45:28PM +0900, Tomoaki AOKI wrote: > Hm, for example, sysutils/lsof (userspace app) depends on kernel > source, and I thought the one Otacilio mentioned is something like it. lsof purpose is to dig into a kernel memory to gather information about the process state and opened files. To do this, lsof authors have to use internal kernel data structures. The library which is discussed in the thread uses public interfaces, but then tries to pack the returned data into internal kernel structure. If this is true, then the breakage is on the library side. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: type of vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin, vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout on AMD64 r320730
Hm, for example, sysutils/lsof (userspace app) depends on kernel source, and I thought the one Otacilio mentioned is something like it. Anyway, we should wait for Otacilio's response. ;-) On Mon, 10 Jul 2017 10:18:07 +0300 Konstantin Belousov wrote: > On Mon, Jul 10, 2017 at 03:40:46PM +0900, Tomoaki AOKI wrote: > > Hi. > > > > Members v_swappgsin and v_vnodepgsin are declared on sys/sys/vmmeter.h > > as > > > > u_int on stable/11@r320798 [1] > > counter_u64_t on head@r320861 [2] > > > > respectively. > > > > Diggin in further, on head@r320861, counter_u64_t is typedef'ed as > > uint64_t * at line 32 of sys/sys/counter.h [3] like below. > > > > typedef uint64_t *counter_u64_t; > > > > So they're "pointers" on head, while "values" on stable/11. > > > > So, as you're on head, it seems you're casting "pointers (addresses)" to > > uint64_t "value". It could be huge and non-expected values. > > Your code would work as expected on stable/11. > > What ? counter_u64_t in userspace in nonsense. > > Whole struct vmmeter is useless in userspace, it is not exported by the > sysctls and is only used in kernel. Sysctl MIBs export individual > counters which should be used with appropriate type, but besides native > uint64_t, 32bit requests are also supported. > > This is why I asked about the type of the vm.v_something and asked for > the explicit fragment of the code which reads the MIBs. > > > > > [1] > > https://svnweb.freebsd.org/base/stable/11/sys/sys/vmmeter.h?annotate=320610 > > > > [2] > > https://svnweb.freebsd.org/base/head/sys/sys/vmmeter.h?annotate=317061 > > > > [3] > > https://svnweb.freebsd.org/base/head/sys/sys/counter.h?annotate=309745 > > > > > > On Sun, 9 Jul 2017 23:15:18 -0300 > > Otac?$B".lio wrote: > > > > > Dears > > > > > > I'm the maintainer of xosview and I'm debugging rather weird behavior > > > from it in the latest FreeBSD 12 revisions (12.0-CURRENT #0 r320730 > > > AMD64) . The problem is occurring on the lines responsible for > > > collecting statistics about paging. These lines follow: > > > > > > If (pageinfo) { > > > Pageinfo [0] = (uint64_t) vm.v_vnodepgsin + (uint64_t) vm.v_swappgsin; > > > Pageinfo [1] = (uint64_t) vm.v_vnodepgsout + (uint64_t) vm.v_swappgsout; > > > } > > > > > > This code works on FreeBSD 11 and until a short time ago it works at 12. > > > But now it returns extremely large values ?$B".".when interpreted as > > > 64-bit > > > values. A debugging has shown that for this sysctl call the following > > > values ?$B".".are returned in the terminal: > > > sysctl vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin > > > vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout > > > Vm.stats.vm.v_vnodepgsin: 47432 > > > Vm.stats.vm.v_swappgsin: 0 > > > Vm.stats.vm.v_vnodepgsout: 19 > > > Vm.stats.vm.v_swappgsout: 0 > > > > > > While the code returns things like: > > > > > > Pageinf [0] = 34359785800; Pageinfo [1] = 140733193388051 > > > Pageinf [0] = 34359785800; Pageinfo [1] = 2678138638516092947 > > > > > > After some tests I found that if I change the code to use a typecast to > > > (uint32_t) then Xosview works correctly.: > > > If (pageinfo) { > > > Pageinfo [0] = (uint32_t) ((uint64_t) vm.v_vnodepgsin + (uint64_t) > > > vm.v_swappgsin); > > > Pageinfo [1] = (uint32_t) ((uint64_t) vm.v_vnodepgsout + (uint64_t) > > > vm.v_swappgsout); > > > } > > > > > > For me, it seems that some code in the kernel is storing values ?$B".".as > > > 32 > > > bits where it should be 64 bits. Is this behavior correct? > > > > > > []'s > > > > > > -Otac?$B".lio > > > ___ > > > freebsd-current@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > > > > > > > > > > > -- > > Tomoaki AOKI > > ___ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > -- Tomoaki AOKI ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: type of vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin, vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout on AMD64 r320730
On Mon, Jul 10, 2017 at 03:40:46PM +0900, Tomoaki AOKI wrote: > Hi. > > Members v_swappgsin and v_vnodepgsin are declared on sys/sys/vmmeter.h > as > > u_int on stable/11@r320798 [1] > counter_u64_t on head@r320861 [2] > > respectively. > > Diggin in further, on head@r320861, counter_u64_t is typedef'ed as > uint64_t * at line 32 of sys/sys/counter.h [3] like below. > > typedef uint64_t *counter_u64_t; > > So they're "pointers" on head, while "values" on stable/11. > > So, as you're on head, it seems you're casting "pointers (addresses)" to > uint64_t "value". It could be huge and non-expected values. > Your code would work as expected on stable/11. What ? counter_u64_t in userspace in nonsense. Whole struct vmmeter is useless in userspace, it is not exported by the sysctls and is only used in kernel. Sysctl MIBs export individual counters which should be used with appropriate type, but besides native uint64_t, 32bit requests are also supported. This is why I asked about the type of the vm.v_something and asked for the explicit fragment of the code which reads the MIBs. > > [1] > https://svnweb.freebsd.org/base/stable/11/sys/sys/vmmeter.h?annotate=320610 > > [2] > https://svnweb.freebsd.org/base/head/sys/sys/vmmeter.h?annotate=317061 > > [3] > https://svnweb.freebsd.org/base/head/sys/sys/counter.h?annotate=309745 > > > On Sun, 9 Jul 2017 23:15:18 -0300 > Otac?$B".lio wrote: > > > Dears > > > > I'm the maintainer of xosview and I'm debugging rather weird behavior > > from it in the latest FreeBSD 12 revisions (12.0-CURRENT #0 r320730 > > AMD64) . The problem is occurring on the lines responsible for > > collecting statistics about paging. These lines follow: > > > > If (pageinfo) { > > Pageinfo [0] = (uint64_t) vm.v_vnodepgsin + (uint64_t) vm.v_swappgsin; > > Pageinfo [1] = (uint64_t) vm.v_vnodepgsout + (uint64_t) vm.v_swappgsout; > > } > > > > This code works on FreeBSD 11 and until a short time ago it works at 12. > > But now it returns extremely large values ?$B".".when interpreted as 64-bit > > values. A debugging has shown that for this sysctl call the following > > values ?$B".".are returned in the terminal: > > sysctl vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin > > vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout > > Vm.stats.vm.v_vnodepgsin: 47432 > > Vm.stats.vm.v_swappgsin: 0 > > Vm.stats.vm.v_vnodepgsout: 19 > > Vm.stats.vm.v_swappgsout: 0 > > > > While the code returns things like: > > > > Pageinf [0] = 34359785800; Pageinfo [1] = 140733193388051 > > Pageinf [0] = 34359785800; Pageinfo [1] = 2678138638516092947 > > > > After some tests I found that if I change the code to use a typecast to > > (uint32_t) then Xosview works correctly.: > > If (pageinfo) { > > Pageinfo [0] = (uint32_t) ((uint64_t) vm.v_vnodepgsin + (uint64_t) > > vm.v_swappgsin); > > Pageinfo [1] = (uint32_t) ((uint64_t) vm.v_vnodepgsout + (uint64_t) > > vm.v_swappgsout); > > } > > > > For me, it seems that some code in the kernel is storing values ?$B".".as > > 32 > > bits where it should be 64 bits. Is this behavior correct? > > > > []'s > > > > -Otac?$B".lio > > ___ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > > > > > > -- > Tomoaki AOKI > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: type of vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin, vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout on AMD64 r320730
Hi. Members v_swappgsin and v_vnodepgsin are declared on sys/sys/vmmeter.h as u_int on stable/11@r320798 [1] counter_u64_t on head@r320861 [2] respectively. Diggin in further, on head@r320861, counter_u64_t is typedef'ed as uint64_t * at line 32 of sys/sys/counter.h [3] like below. typedef uint64_t *counter_u64_t; So they're "pointers" on head, while "values" on stable/11. So, as you're on head, it seems you're casting "pointers (addresses)" to uint64_t "value". It could be huge and non-expected values. Your code would work as expected on stable/11. [1] https://svnweb.freebsd.org/base/stable/11/sys/sys/vmmeter.h?annotate=320610 [2] https://svnweb.freebsd.org/base/head/sys/sys/vmmeter.h?annotate=317061 [3] https://svnweb.freebsd.org/base/head/sys/sys/counter.h?annotate=309745 On Sun, 9 Jul 2017 23:15:18 -0300 Otac〓lio wrote: > Dears > > I'm the maintainer of xosview and I'm debugging rather weird behavior > from it in the latest FreeBSD 12 revisions (12.0-CURRENT #0 r320730 > AMD64) . The problem is occurring on the lines responsible for > collecting statistics about paging. These lines follow: > > If (pageinfo) { > Pageinfo [0] = (uint64_t) vm.v_vnodepgsin + (uint64_t) vm.v_swappgsin; > Pageinfo [1] = (uint64_t) vm.v_vnodepgsout + (uint64_t) vm.v_swappgsout; > } > > This code works on FreeBSD 11 and until a short time ago it works at 12. > But now it returns extremely large values 〓〓when interpreted as 64-bit > values. A debugging has shown that for this sysctl call the following > values 〓〓are returned in the terminal: > sysctl vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin > vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout > Vm.stats.vm.v_vnodepgsin: 47432 > Vm.stats.vm.v_swappgsin: 0 > Vm.stats.vm.v_vnodepgsout: 19 > Vm.stats.vm.v_swappgsout: 0 > > While the code returns things like: > > Pageinf [0] = 34359785800; Pageinfo [1] = 140733193388051 > Pageinf [0] = 34359785800; Pageinfo [1] = 2678138638516092947 > > After some tests I found that if I change the code to use a typecast to > (uint32_t) then Xosview works correctly.: > If (pageinfo) { > Pageinfo [0] = (uint32_t) ((uint64_t) vm.v_vnodepgsin + (uint64_t) > vm.v_swappgsin); > Pageinfo [1] = (uint32_t) ((uint64_t) vm.v_vnodepgsout + (uint64_t) > vm.v_swappgsout); > } > > For me, it seems that some code in the kernel is storing values 〓〓as 32 > bits where it should be 64 bits. Is this behavior correct? > > []'s > > -Otac〓lio > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > > -- Tomoaki AOKI ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: type of vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin, vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout on AMD64 r320730
On Sun, Jul 09, 2017 at 11:15:18PM -0300, Otac??lio wrote: > Dears > > I'm the maintainer of xosview and I'm debugging rather weird behavior > from it in the latest FreeBSD 12 revisions (12.0-CURRENT #0 r320730 > AMD64) . The problem is occurring on the lines responsible for > collecting statistics about paging. These lines follow: > > If (pageinfo) { > Pageinfo [0] = (uint64_t) vm.v_vnodepgsin + (uint64_t) vm.v_swappgsin; Why these casts are needed ? What is the type of vm.v_vnodepgsin ? > Pageinfo [1] = (uint64_t) vm.v_vnodepgsout + (uint64_t) vm.v_swappgsout; > } > > This code works on FreeBSD 11 and until a short time ago it works at 12. > But now it returns extremely large values ??when interpreted as 64-bit > values. A debugging has shown that for this sysctl call the following > values ??are returned in the terminal: > sysctl vm.stats.vm.v_vnodepgsin vm.stats.vm.v_swappgsin > vm.stats.vm.v_vnodepgsout vm.stats.vm.v_swappgsout > Vm.stats.vm.v_vnodepgsin: 47432 > Vm.stats.vm.v_swappgsin: 0 > Vm.stats.vm.v_vnodepgsout: 19 > Vm.stats.vm.v_swappgsout: 0 > > While the code returns things like: > > Pageinf [0] = 34359785800; Pageinfo [1] = 140733193388051 > Pageinf [0] = 34359785800; Pageinfo [1] = 2678138638516092947 > > After some tests I found that if I change the code to use a typecast to > (uint32_t) then Xosview works correctly.: > If (pageinfo) { > Pageinfo [0] = (uint32_t) ((uint64_t) vm.v_vnodepgsin + (uint64_t) > vm.v_swappgsin); > Pageinfo [1] = (uint32_t) ((uint64_t) vm.v_vnodepgsout + (uint64_t) > vm.v_swappgsout); > } > > For me, it seems that some code in the kernel is storing values ??as 32 > bits where it should be 64 bits. Is this behavior correct? Show the _exact_ code fragment that reads the MIBs values. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"