Okay nm the first part, it does actually allocate large pages and use
them in the code segment now, didn't relink the binary I was using.
Output now looks like
$ hugectl --text --bss --data --heap --shm --share-text -v -v -v
./test 100000000
hugectl: INFO: HUGETLB_VERBOSE='5'
hugectl: INFO: LD_LIBRARY_PATH='/usr/lib32:/usr/lib64:'
hugectl: INFO: HUGETLB_ELFMAP='R:W'
hugectl: INFO: HUGETLB_MORECORE='yes'
hugectl: INFO: HUGETLB_SHM='yes'
hugectl: WARNING: LD_PRELOAD not appropriate for this map combination
hugectl: INFO: HUGETLB_SHARE='1'
libhugetlbfs [dmlb2000ts:31341]: INFO: Parsed kernel version: [2] . [6] . [30]
libhugetlbfs [dmlb2000ts:31341]: INFO: Feature private_reservations is
present in this kernel
libhugetlbfs [dmlb2000ts:31341]: INFO: Kernel has MAP_PRIVATE
reservations. Disabling heap prefaulting.
libhugetlbfs [dmlb2000ts:31341]: INFO: HUGETLB_SHARE=1, sharing
enabled for only read-only segments
libhugetlbfs [dmlb2000ts:31341]: INFO: Segment 0 (phdr 2):
0x400000-0x4008fc (filesz=0x8fc) (prot = 0x5)
libhugetlbfs [dmlb2000ts:31341]: INFO: Segment 1 (phdr 3):
0x800900-0x80800b60 (filesz=0x228) (prot = 0x3)
libhugetlbfs [dmlb2000ts:31341]: INFO: libhugetlbfs version: 2.4
libhugetlbfs [dmlb2000ts:31342]: WARNING: Unable to verify address
range 0x800000 - 0x800000. Not empty?
libhugetlbfs [dmlb2000ts:31342]: INFO: Mapped hugeseg at
0x2aaaaac00000. Copying 0x228 bytes and 0 extra bytes from
0x800900...done
libhugetlbfs [dmlb2000ts:31341]: INFO: Prepare succeeded
libhugetlbfs [dmlb2000ts:31341]: INFO: setup_morecore(): heapaddr = 0x82000000
0
However, I try this same binary on an older kernel and it doesn't look the same.
$ hugectl -v -v -v --text --bss --heap --shm --data ./test 100000000
hugectl: INFO: HUGETLB_VERBOSE='5'
hugectl: INFO:
LD_LIBRARY_PATH='/usr/local/lib:/usr/local/lib64:/usr/mpi/gcc/mvapich2-1.2p1/lib:'
hugectl: INFO: HUGETLB_ELFMAP='R:W'
hugectl: INFO: HUGETLB_MORECORE='yes'
hugectl: INFO: HUGETLB_SHM='yes'
hugectl: WARNING: LD_PRELOAD not appropriate for this map combination
libhugetlbfs [cut1n10:24262]: INFO: Parsed kernel version: [2] . [6] . [18]
libhugetlbfs [cut1n10:24262]: INFO: HUGETLB_SHARE=0, sharing disabled
libhugetlbfs [cut1n10:24262]: INFO: Segment 0 (phdr 2):
0x400000-0x4008bc (filesz=0x8bc) (prot = 0x5)
libhugetlbfs [cut1n10:24262]: INFO: Segment 1 (phdr 3):
0x5008c0-0x80500af0 (filesz=0x220) (prot = 0x3)
libhugetlbfs [cut1n10:24262]: WARNING: Layout problem with segments 0 and 1:
Segments would overlap
libhugetlbfs [cut1n10:24262]: INFO: No segments were appropriate for remapping
libhugetlbfs [cut1n10:24262]: INFO: setup_morecore(): heapaddr = 0x98000000
0
When was large page support for code segments supported?
Thanks,
- David Brown
On Thu, Jun 25, 2009 at 10:31 AM, David Brown<dmlb2...@gmail.com> wrote:
> I fixed the memory issue and the application is running but its not
> allocating large pages.
>
> $ hugectl --text --bss --data --heap --shm -v -v -v ./test 100000000
> hugectl: INFO: HUGETLB_VERBOSE='5'
> hugectl: INFO: LD_LIBRARY_PATH='/usr/lib32:/usr/lib64:'
> hugectl: INFO: HUGETLB_ELFMAP='R:W'
> hugectl: INFO: HUGETLB_MORECORE='yes'
> hugectl: INFO: HUGETLB_SHM='yes'
> hugectl: WARNING: LD_PRELOAD not appropriate for this map combination
> libhugetlbfs [dmlb2000ts:15538]: INFO: Parsed kernel version: [2] . [6] . [30]
> libhugetlbfs [dmlb2000ts:15538]: INFO: Feature private_reservations is
> present in this kernel
> libhugetlbfs [dmlb2000ts:15538]: INFO: HUGETLB_SHARE=0, sharing disabled
> libhugetlbfs [dmlb2000ts:15538]: INFO: Segment 0 (phdr 2):
> 0x400000-0x4008bc (filesz=0x8bc) (prot = 0x5)
> libhugetlbfs [dmlb2000ts:15538]: INFO: Segment 1 (phdr 3):
> 0x5008c0-0x80500af0 (filesz=0x220) (prot = 0x3)
> libhugetlbfs [dmlb2000ts:15538]: WARNING: Layout problem with segments 0 and
> 1:
> Segments would overlap
> libhugetlbfs [dmlb2000ts:15538]: INFO: No segments were appropriate
> for remapping
> libhugetlbfs [dmlb2000ts:15538]: INFO: Kernel has MAP_PRIVATE
> reservations. Disabling heap prefaulting.
> libhugetlbfs [dmlb2000ts:15538]: INFO: setup_morecore(): heapaddr = 0x80e00000
> 0
>
>
> So I'm guessing make check is supposed to be run as root? since there
> seems to be mount commands and echoing stuff out to proc that is
> performed.
>
> I'd be nice if if you could just use some of the same checks in
> libhugetlbfs.so to find a mount and see if you can mmap in it then run
> the tests if you aren't root.
>
> ( as me )
> ********** TEST SUMMARY
> * 2M
> * 32-bit 64-bit
> * Total testcases: 0 98
> * Skipped: 0 0
> * PASS: 0 68
> * FAIL: 0 5
> * Killed by signal: 0 0
> * Bad configuration: 0 11
> * Expected FAIL: 0 0
> * Unexpected PASS: 0 0
> * Strange test result: 0 14
> **********
>
> ( as root )
> ********** TEST SUMMARY
> * 2M
> * 32-bit 64-bit
> * Total testcases: 0 98
> * Skipped: 0 0
> * PASS: 0 81
> * FAIL: 0 0
> * Killed by signal: 0 0
> * Bad configuration: 0 3
> * Expected FAIL: 0 0
> * Unexpected PASS: 0 0
> * Strange test result: 0 14
> **********
>
> Hmmm, not sure what tests were what do you want the full output?
>
> Thanks,
> - David Brown
>
> On Wed, Jun 24, 2009 at 9:09 PM, David
> Gibson<da...@gibson.dropbear.id.au> wrote:
>> On Thu, Jun 25, 2009 at 03:50:14AM +0000, David Brown wrote:
>>> I've been playing around with large pages and libhugetlbfs and have
>>> been having no such luck getting my code segment to be backed by large
>>> pages.
>>>
>>> I have a small example code that doesn't seem to be loading.
>>
>> First, does the libhugetlbfs pass ("make check").
>>
>> [snip]
>>> $ hugectl -v -v --text --bss --heap --shm --data ./test 10000000
>>> hugectl: INFO: HUGETLB_VERBOSE='4'
>>> hugectl: INFO: LD_LIBRARY_PATH='/usr/local/lib:/usr/local/lib64:'
>>> hugectl: INFO: HUGETLB_ELFMAP='R:W'
>>> hugectl: INFO: HUGETLB_MORECORE='yes'
>>> hugectl: INFO: HUGETLB_SHM='yes'
>>> hugectl: WARNING: LD_PRELOAD not appropriate for this map combination
>>> libhugetlbfs [uber-gamer:31047]: INFO: Parsed kernel version: [2] . [6] .
>>> [30]
>>> libhugetlbfs [uber-gamer:31047]: INFO: Feature private_reservations is
>>> present in this kernel
>>> libhugetlbfs [uber-gamer:31047]: INFO: Kernel has MAP_PRIVATE
>>> reservations. Disabling heap prefaulting.
>>> libhugetlbfs [uber-gamer:31047]: INFO: HUGETLB_SHARE=0, sharing disabled
>>> libhugetlbfs [uber-gamer:31047]: INFO: Segment 0 (phdr 2):
>>> 0x400000-0x40089c (filesz=0x89c) (prot = 0x5)
>>> libhugetlbfs [uber-gamer:31047]: INFO: Segment 1 (phdr 3):
>>> 0x8008a0-0x40800ae0 (filesz=0x220) (prot = 0x3)
>>> libhugetlbfs [uber-gamer:31047]: INFO: libhugetlbfs version: 2.4
>>> libhugetlbfs [uber-gamer:31048]: INFO: Mapped hugeseg at
>>> 0x2aaaaac00000. Copying 0x89c bytes and 0 extra bytes from
>>> 0x400000...done
>>> libhugetlbfs [uber-gamer:31047]: INFO: Prepare succeeded
>>> libhugetlbfs [uber-gamer:31049]: WARNING: Unable to verify address
>>> range 0x800000 - 0x800000. Not empty?
>>> libhugetlbfs [uber-gamer:31049]: INFO: Mapped hugeseg at
>>> 0x2aaaaac00000. Copying 0x220 bytes and 0 extra bytes from
>>> 0x8008a0...done
>>> libhugetlbfs [uber-gamer:31047]: INFO: Prepare succeeded
>>> Failed to map hugepage segment 1: 800000-40a00000 (errno=12)
>>> Aborted
>>
>> errno 12 is ENOMEM. This suggests you don't have enough hugepages
>> allocated to back the program's segments.
>>
>> --
>> David Gibson | I'll have my music baroque, and my code
>> david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
>> | _way_ _around_!
>> http://www.ozlabs.org/~dgibson
>>
>
------------------------------------------------------------------------------
_______________________________________________
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
