On Tue, 12 Jun 2012 21:13:42 -0400, Josh Boyer wrote:
> On Tue, Jun 12, 2012 at 5:18 PM, Eric B Munson <emun...@mgebm.net>
> wrote:
>> On Tue, 12 Jun 2012 15:34:32 -0400, Josh Boyer wrote:
>>>
>>> Hi,
>>>
>>> I've been poking around at the libhugetlbfs testcases and there are
>>> a
>>> small number of failures that I can't figure out. The primary one
>>> is
>>> the malloc test. From all appearances, it seems that the custom
>>> hugetlbfs_morecore function isn't getting called when the test
>>> calls
>>> malloc, and it fails because the page wasn't allocated from the
>>> hugepages. You can see below that the INFO call in the
>>> hugetlbfs_morecore function never prints anything:
>>>
>>> [jwboyer@zod obj64]$ sudo
>>> LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:../../obj64/
>>> LD_PRELOAD=../../obj64/libhugetlbfs.so HUGETLB_MORECORE=yes
>>> HUGETLB_VERBOSE=4 ./malloc
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: Found pagesize 2048
>>> kB
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: Parsed kernel
>>> version:
>>> [3] . [3] . [7]
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: Feature
>>> private_reservations is present in this kernel
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: Feature
>>> noreserve_safe
>>> is present in this kernel
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: Feature map_hugetlb
>>> is
>>> present in this kernel
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: Kernel has
>>> MAP_PRIVATE
>>> reservations. Disabling heap prefaulting.
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: Kernel supports
>>> MAP_HUGETLB
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: HUGETLB_SHARE=0,
>>> sharing disabled
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO:
>>> HUGETLB_NO_RESERVE=no,
>>> reservations enabled
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: No segments were
>>> appropriate for remapping
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: setup_morecore():
>>> heapaddr = 0xe00000
>>> Starting testcase "./malloc", pid 22061
>>> HUGETLB_MORECORE=yes
>>> HUGETLB_RESTRICT_EXE=(null)
>>> expect_hugepage=1
I missed it last time through, but this line is the problem. Note that
earlier the morecore_setup function said that the heap base is at
0xe00000, and we allocated below that. If the __morecore hook was
called, there would be more debug output here about what it was asked to
do.
>>> malloc(4) = 0xd89010
>>> FAIL Address is not hugepage
>>> [jwboyer@zod obj64]$
>>>
>>> This is with the latest git head of libhugetlbfs, and I have 32
>>> hugepages setup and free:
>>>
>>> [jwboyer@zod glibc]$ grep Huge /proc/meminfo
>>> AnonHugePages: 299008 kB
>>> HugePages_Total: 32
>>> HugePages_Free: 32
>>> HugePages_Rsvd: 0
>>> HugePages_Surp: 0
>>> Hugepagesize: 2048 kB
>>> [jwboyer@zod glibc]$
>>>
>>> I ran the test in gdb as well, and my breakpoint on
>>> hugetlbfs_morecore
>>> never triggered after main was started. Perhaps this is a
>>> side-effect
>>> of glibc already having pre-allocated heap from arenas?
>>>
>>> Has anyone else seen this?
>>
>> Quick question, do you have hugetlbfs mounted?
>
> Yep.
>
> [jwboyer@zod glibc]$ mount | grep hugetlb
> hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
> [jwboyer@zod glibc]$
>
> I would have thought that was obvious from:
>
>>> libhugetlbfs [zod.bos.redhat.com:22061]: INFO: Found pagesize 2048
>>> kB
Not necessarily, there is usually a small portion after that that
prints the mount point, that is why I asked.
>
> All of the other tests in the testsuite complete fine with the
> exception of the malloc HUGETLB_MEMCORE tests where expect_hugepage=1
> and the readahead_reserve.sh test. The latter seems to be known
> broken
> or at least reported as such in 2010 with no replies.
>
> Again, it seems that the hugetlbfs_memcore function is never called
> for
> the malloc tests. I even ran it under gdb, set a breakpoint at main,
> then set one at hugetlbfs_memcore after I was in main and it never
> hit
> that breakpoint.
The setup for this hasn't changed in a number of years, the only thing
I can come up with is that glibc is ignoring the __morecore hook. I am
looking into why this might happen.
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
