I sent the previous mail too early! I got it to work for root user by setting "/proc/sys/vm/nr_hugepages" to 20 as suggested in Documentation/vm/hugetlbpage.txt. Now I'm able to mmap it and it reflects the same in /proc/meminfo as well! :)
HTH, -mandeep On Mon, Aug 18, 2014 at 1:23 PM, Mandeep Sandhu <[email protected] > wrote: > I tried your program and got the same result, i.e "Resource temporarily > unavailable" for an ordinary user and "Cannot allocate memory" if run as > root. > > However, reading the Huge TLB pages doc > (Documentation/vm/hugetlbpage.txt), I saw the following para which might > explain the error: > ... > Once a number of huge pages have been pre-allocated to the kernel huge page > pool, a user with appropriate privilege can use either the mmap system call > or shared memory system calls to use the huge pages. See the discussion of > Using Huge Pages, below. > > The administrator can allocate persistent huge pages on the kernel boot > command line by specifying the "hugepages=N" parameter, where 'N' = the > number of huge pages requested. This is the most reliable method of > allocating huge pages as memory has not yet become fragmented. > ... > ... > > I don't have my kernel booted with it, so that might explain why mmap fails > for MAP_HUGETLB. > > I can't restart my machine with the new flag right now, but maybe you can try. > > HTH, > > -mandeep > > > > > On Mon, Aug 18, 2014 at 11:30 AM, Kevin O'Gorman <[email protected]> > wrote: > >> Now I'm befuddled. I modified the source so that it was easy to try >> different combinations. If I sudo it, I can map a 1GiB (1<<30) private >> anonymous region if I sudo the program. I cannot find any combination with >> MAP_HUGETLB that works at all. >> >> Here it is in working form, with an anonymous map and no backing file. >> The mmap request is for a single byte, so it works for ordinary users Try >> uncommenting the MAP_HUGETLB line, and I'd love to hear about a version >> that works, even if it's only for root. >> >> Since I'm aiming for a huge memory region, I figured HUGETLB would make >> sense to limit the overhead. >> >> ============================ CUT HERE ================ >> /** >> * @file >> * <pre>"Find out the limits on locked memory" >> * Last Modified: Mon Aug 18 11:21:23 PDT 2014 >> * @author Kevin O'Gorman >> */ >> >> #define _GNU_SOURCE /* enable some of the mmap flags */ >> #include <unistd.h> >> #include <stdlib.h> >> #include <stdio.h> >> #include <string.h> >> #include <sys/mman.h> >> >> /* for getrlimit(2) */ >> #include <sys/time.h> >> #include <sys/resource.h> >> >> /* for open(2) */ >> #include <sys/types.h> >> #include <sys/stat.h> >> #include <sys/fcntl.h> >> >> int >> main(int argc, char *argv[]) >> { >> void *where; >> size_t length = 1; >> struct rlimit rlim; >> int flags = 0; >> int fd = -1; >> >> // fd = open("solvedtaq", O_RDWR); if (fd == -1) perror("open"); >> printf("fd is %d\n", fd); >> >> // flags |= MAP_HUGETLB; >> flags |= MAP_PRIVATE; >> flags |= MAP_LOCKED; >> flags |= MAP_ANONYMOUS; >> >> if (getrlimit(RLIMIT_MEMLOCK, &rlim) == 0) { >> printf("RLIMIT_MEMLOCK: hard: %lld, soft: %lld\n", (long >> long)rlim.rlim_max, (long long)rlim.rlim_cur); >> } else { >> perror("getrlimit failed"); >> } >> >> where = mmap(NULL, length, PROT_READ | PROT_WRITE, flags, fd, 0); >> if (where != MAP_FAILED) { >> printf("Mapped at %p\n", where); >> } else { >> perror("Mapping failed"); >> } >> >> return EXIT_SUCCESS; >> } >> ============================ CUT HERE ================ >> >> >> >> On Mon, Aug 18, 2014 at 10:24 AM, Kevin O'Gorman <[email protected]> >> wrote: >> >>> >>> On Mon, Aug 18, 2014 at 9:46 AM, <[email protected]> wrote: >>> >>>> On Mon, 18 Aug 2014 08:15:12 -0700, "Kevin O'Gorman" said: >>>> >>>> > maybe someone here can help me figure out how to map a big (really >>>> big) >>>> > work area and lock it in memory. >>>> >>>> What problem are you trying to solve by doing this? It usually doesn't >>>> make >>>> sense to lock more than a small chunk in memory - that's mostly for >>>> crypto >>>> programs like PGP that are paranoid about keys being written to swap >>>> spaces, >>>> and a *very* few highly performance oriented programs. 99.8% of the >>>> time, >>>> the kernel will do a reasonable job of making sure the right pages are >>>> in >>>> memory. >>>> >>>> Plus, programmers almost always over-estimate what they *really* need >>>> to lock >>>> down. The end result is that performance ends up being *worse*, >>>> because they >>>> lock out more than needed, leaving all the rest of the processes on the >>>> system >>>> fighting over a too-small pool of pages. Really sucks when you >>>> actually have >>>> to read from disk every I/O because you no longer have an in-core file >>>> cache :) >>>> >>>> I'm solving a huge alpha-beta search problem, and want a huge cache >>> table tocheck permutations. That's why I increased RAM from 8 to 32 GB, >>> and would >>> be happy to use the extra 24GB all for the cache. (Cache table is like >>> a hash table but with collisions causing replacement rather than chaining). >>> >>> Even in my small tests, the cache is giving me speedups in the >>> neighborhood of >>> a million, so I'm going big. Because this is about speed, I do not >>> *ever* want it to >>> swap. >>> >>> >>>> > I wrote a little test program to try this out, and it fails. As a >>>> regular >>>> > user, perror() tells me some "resource is temporarily unavailable". >>>> As >>>> > root, it says it "cannot allocate memory". I'm only asking for 1 >>>> byte. >>>> >>>> > where = mmap(NULL, length, PROT_READ | PROT_WRITE, MAP_PRIVATE | >>>> > MAP_ANONYMOUS | MAP_HUGETLB | MAP_LOCKED | MAP_POPULATE, -1, 0); >>>> >>>> Remember that no matter what "length" you pass, even if it's 1 byte, the >>>> kernel has to allocate at least enough integer pages to cover the >>>> request. >>>> You ask for 4100 bytes, it has to allocate 2 pages because 4096 isn't >>>> quite big enough. >>>> >>>> And if you ask for MAP_HUGETLB, you're going to get at least 1 page. >>>> Which >>>> is probably 2M in size. And that can be painful if the default max >>>> lockable >>>> memory is less than 2M (at least on my Fedora Rawhide box, it's all of >>>> 64K). There's also some interaction danger with MAP_POPULATE and memory >>>> fragmentation. But both POPULATE and HUGETLB have a high chance of >>>> not playing well with LOCKED. >>>> >>> >>> Good catch. Indeed, removing MAP_HUGETLB allows the program to >>> succeed. I'll try increasing the resource limit, but may have to have to >>> sudo it, do the mmap() and then chusr to my real ID. >>> >>> The fd of -1 that some folks thought might be a problem is suggested in >>> the man page for use with MAP_ANONYMOUS. That flag means I'm not using a >>> backing file, after all. >>> >>> -- >>> Kevin O'Gorman >>> >>> programmer, n. an organism that transmutes caffeine into software. >>> Please consider the environment before printing this email. >>> >> >> >> >> -- >> Kevin O'Gorman >> >> programmer, n. an organism that transmutes caffeine into software. >> Please consider the environment before printing this email. >> >> _______________________________________________ >> Kernelnewbies mailing list >> [email protected] >> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies >> >> >
_______________________________________________ Kernelnewbies mailing list [email protected] http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
