Re: premature ENOMEM
Solved!!! Peter Lei of Cisco noticed this application uses mmap rather than malloc so I could get what I wanted (go beyond 200 MB without ENOMEM) if I adjusted sysctl -w vm.max_proc_mmap to be a higher value. Thanks so much for everyone's help, brad On Fri, Feb 26, 2010 at 8:12 AM, Brad Penoff wrote: > Thanks Anton for the advice about a low kern.maxdsiz . Does that work > on your machine? I tried to give instructions in the original post so > that anyone can tune their machine and try to see what I am seeing. > Can others try to see on their own machines if the suggestions work? > I'll do anything to help (even provide machines, if need be). I'm > surprised how difficult this is FreeBSD compared to Linux and Mac OS > X, but I think FreeBSD will be the most performant if I can get over > this hump. > > On my machine, I had kern.maxdsiz, kern.maxssiz, and kern.dfldsiz all > set to 2147483648 to match the 2 GB on my machine; that wasn't working > though. I tried to modify but had much worse results... > > I set this: > kern.maxdsiz="100" > kern.maxssiz="2147483648" > kern.dfldsiz="2147483648" > ... and NFS didn't load. > > I set this: > kern.maxdsiz="0" > kern.maxssiz="2147483648" > kern.dfldsiz="2147483648" > ... and sshd didn't load! > > What combination should I try or what combination did you have in > mind? Please excuse my ignorance. > > Also, I don't understand the link you sent... was it mistyped? It > seems to go to some index and not any advice on this matter... > http://docs.freebsd.org/cgi/mid.cgi?200207291839.g6TIduVw055637 > > Thanks again for your help and ahead of time to anyone else's... I'm > getting a bit desperate and I'm surprised how difficult this is to get > working in FreeBSD when it was so simple in Linux and Mac OS X. The > thing is, I typically use FreeBSD for the best performance but I can't > even get it to run there, at the moment. > > Thanks, > brad > > > On Thu, Feb 25, 2010 at 10:49 PM, Brad Penoff wrote: >> Greetings, >> >> I have a 32-bit machine with 2 GB running FreeBSD 8. I have a complex >> application that starts getting ENOMEM once the resident memory is >> about 200 MB. I adjusted the appropriate /boot/loader.conf and >> /etc/login.conf settings resulting in an increase in the "limit" >> values to around 2 GB, but still the complex application gives me >> ENOMEM at 200 MB. Including swap space, I should be able to handle 3 >> GB in an application. I need help understanding how I can enable this >> or why I cannot. >> >> I had this same problem on FreeBSD 6.3 about a year and a half ago, >> but I found an application-level work-around particular for FreeBSD >> (the full thread is here http://markmail.org/message/5nsld7pb25m5bfja >> ; but we found no general solution unfortunately ). This >> application-level work-around is no longer sufficient so I wanted to >> revisit the root of this problem I was seeing and hopefully ask this >> community of smart hackers to give me a hand here... >> >> I have created a simplified application that demonstrates this exact >> same problem and made it easy to reproduce. I was hoping that some >> kind and helpful person could take a look at it and help me out. It >> doesn't require any editing of code or anything... In the end, I just >> need to know how to configure my FreeBSD box with 2 GB of RAM to not >> give me ENOMEM for this application when it is using only 200 MB, >> something that never happened in the previous unresolved thread >> reverenced above. >> >> This simplified application has a deliberate memory leak but the goal >> is for it to not give ENOMEM at 200 MB (I want to be able to use about >> 1.5 GB). I have created a tarball... I think it may only work on a >> 32-bit machine. >> >> download/wget http://cs.ubc.ca/~penoff/reslim.tgz >> tar zxf reslim.tgz >> cd reslim >> gmake (or make on Linux) >> sh ./myumem.sh >> >> On my system, when this prints our an error message, in "top", I only >> see about 200 MB of both "SIZE" and "RES" (they are a few numbers >> off). The goal is for "SIZE" to be 1.5 GB or higher; "RES" is up to >> the OS. For example, on Linux, "SIZE" (called "VIRT") goes as high as >> 2.8 GB before the system just becomes slow and unusable (never saw >> ENOMEM). For FreeBSD, what can I change to increase the possible >> "SIZE" before I see ENOMEM? I have tried increasing >> /boot/loader.conf values to match my 2 GB RAM (kern.maxdsiz, >> kern.maxssiz, kern.dfldsiz), as well as unlimited values in >> /etc/login.conf, but still 200 MB seems a hard limit. Any ideas? Why >> is it not going beyond 200 MB? Resident memory is equal to memory >> size used by the app so why is it never going to disk to swap before >> ENOMEM comes up? >> >> Thanks a million for any help or advice you could lend. I would be >> more than willing to help in any way, even providing a machine to try >> this on (I could sponsor an emulab.net account). Anxious to hear what >> options exist. >> >> Thanks aga
Re: premature ENOMEM
Thanks Anton for the advice about a low kern.maxdsiz . Does that work on your machine? I tried to give instructions in the original post so that anyone can tune their machine and try to see what I am seeing. Can others try to see on their own machines if the suggestions work? I'll do anything to help (even provide machines, if need be). I'm surprised how difficult this is FreeBSD compared to Linux and Mac OS X, but I think FreeBSD will be the most performant if I can get over this hump. On my machine, I had kern.maxdsiz, kern.maxssiz, and kern.dfldsiz all set to 2147483648 to match the 2 GB on my machine; that wasn't working though. I tried to modify but had much worse results... I set this: kern.maxdsiz="100" kern.maxssiz="2147483648" kern.dfldsiz="2147483648" ... and NFS didn't load. I set this: kern.maxdsiz="0" kern.maxssiz="2147483648" kern.dfldsiz="2147483648" ... and sshd didn't load! What combination should I try or what combination did you have in mind? Please excuse my ignorance. Also, I don't understand the link you sent... was it mistyped? It seems to go to some index and not any advice on this matter... http://docs.freebsd.org/cgi/mid.cgi?200207291839.g6TIduVw055637 Thanks again for your help and ahead of time to anyone else's... I'm getting a bit desperate and I'm surprised how difficult this is to get working in FreeBSD when it was so simple in Linux and Mac OS X. The thing is, I typically use FreeBSD for the best performance but I can't even get it to run there, at the moment. Thanks, brad On Thu, Feb 25, 2010 at 10:49 PM, Brad Penoff wrote: > Greetings, > > I have a 32-bit machine with 2 GB running FreeBSD 8. I have a complex > application that starts getting ENOMEM once the resident memory is > about 200 MB. I adjusted the appropriate /boot/loader.conf and > /etc/login.conf settings resulting in an increase in the "limit" > values to around 2 GB, but still the complex application gives me > ENOMEM at 200 MB. Including swap space, I should be able to handle 3 > GB in an application. I need help understanding how I can enable this > or why I cannot. > > I had this same problem on FreeBSD 6.3 about a year and a half ago, > but I found an application-level work-around particular for FreeBSD > (the full thread is here http://markmail.org/message/5nsld7pb25m5bfja > ; but we found no general solution unfortunately ). This > application-level work-around is no longer sufficient so I wanted to > revisit the root of this problem I was seeing and hopefully ask this > community of smart hackers to give me a hand here... > > I have created a simplified application that demonstrates this exact > same problem and made it easy to reproduce. I was hoping that some > kind and helpful person could take a look at it and help me out. It > doesn't require any editing of code or anything... In the end, I just > need to know how to configure my FreeBSD box with 2 GB of RAM to not > give me ENOMEM for this application when it is using only 200 MB, > something that never happened in the previous unresolved thread > reverenced above. > > This simplified application has a deliberate memory leak but the goal > is for it to not give ENOMEM at 200 MB (I want to be able to use about > 1.5 GB). I have created a tarball... I think it may only work on a > 32-bit machine. > > download/wget http://cs.ubc.ca/~penoff/reslim.tgz > tar zxf reslim.tgz > cd reslim > gmake (or make on Linux) > sh ./myumem.sh > > On my system, when this prints our an error message, in "top", I only > see about 200 MB of both "SIZE" and "RES" (they are a few numbers > off). The goal is for "SIZE" to be 1.5 GB or higher; "RES" is up to > the OS. For example, on Linux, "SIZE" (called "VIRT") goes as high as > 2.8 GB before the system just becomes slow and unusable (never saw > ENOMEM). For FreeBSD, what can I change to increase the possible > "SIZE" before I see ENOMEM? I have tried increasing > /boot/loader.conf values to match my 2 GB RAM (kern.maxdsiz, > kern.maxssiz, kern.dfldsiz), as well as unlimited values in > /etc/login.conf, but still 200 MB seems a hard limit. Any ideas? Why > is it not going beyond 200 MB? Resident memory is equal to memory > size used by the app so why is it never going to disk to swap before > ENOMEM comes up? > > Thanks a million for any help or advice you could lend. I would be > more than willing to help in any way, even providing a machine to try > this on (I could sponsor an emulab.net account). Anxious to hear what > options exist. > > Thanks again, > brad > ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: premature ENOMEM
On Thu, 25 Feb 2010 22:49:22 -0800, Brad Penoff wrote: BP> I have a 32-bit machine with 2 GB running FreeBSD 8. I have a complex BP> application that starts getting ENOMEM once the resident memory is BP> about 200 MB. I adjusted the appropriate /boot/loader.conf and BP> /etc/login.conf settings resulting in an increase in the "limit" BP> values to around 2 GB, but still the complex application gives me BP> ENOMEM at 200 MB. Including swap space, I should be able to handle 3 BP> GB in an application. I need help understanding how I can enable this BP> or why I cannot. Don't forget that increase of kern.maxdsiz will decrease space for mmap. And by default malloc in FreeBSD 8 uses mmap. Try to set kern.maxdsiz to small value (may be 0 will be ok). http://docs.freebsd.org/cgi/mid.cgi?200207291839.g6TIduVw055637 -- WBR, Anton Yuzhaninov ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
premature ENOMEM
Greetings, I have a 32-bit machine with 2 GB running FreeBSD 8. I have a complex application that starts getting ENOMEM once the resident memory is about 200 MB. I adjusted the appropriate /boot/loader.conf and /etc/login.conf settings resulting in an increase in the "limit" values to around 2 GB, but still the complex application gives me ENOMEM at 200 MB. Including swap space, I should be able to handle 3 GB in an application. I need help understanding how I can enable this or why I cannot. I had this same problem on FreeBSD 6.3 about a year and a half ago, but I found an application-level work-around particular for FreeBSD (the full thread is here http://markmail.org/message/5nsld7pb25m5bfja ; but we found no general solution unfortunately ). This application-level work-around is no longer sufficient so I wanted to revisit the root of this problem I was seeing and hopefully ask this community of smart hackers to give me a hand here... I have created a simplified application that demonstrates this exact same problem and made it easy to reproduce. I was hoping that some kind and helpful person could take a look at it and help me out. It doesn't require any editing of code or anything... In the end, I just need to know how to configure my FreeBSD box with 2 GB of RAM to not give me ENOMEM for this application when it is using only 200 MB, something that never happened in the previous unresolved thread reverenced above. This simplified application has a deliberate memory leak but the goal is for it to not give ENOMEM at 200 MB (I want to be able to use about 1.5 GB). I have created a tarball... I think it may only work on a 32-bit machine. download/wget http://cs.ubc.ca/~penoff/reslim.tgz tar zxf reslim.tgz cd reslim gmake (or make on Linux) sh ./myumem.sh On my system, when this prints our an error message, in "top", I only see about 200 MB of both "SIZE" and "RES" (they are a few numbers off). The goal is for "SIZE" to be 1.5 GB or higher; "RES" is up to the OS. For example, on Linux, "SIZE" (called "VIRT") goes as high as 2.8 GB before the system just becomes slow and unusable (never saw ENOMEM). For FreeBSD, what can I change to increase the possible "SIZE" before I see ENOMEM? I have tried increasing /boot/loader.conf values to match my 2 GB RAM (kern.maxdsiz, kern.maxssiz, kern.dfldsiz), as well as unlimited values in /etc/login.conf, but still 200 MB seems a hard limit. Any ideas? Why is it not going beyond 200 MB? Resident memory is equal to memory size used by the app so why is it never going to disk to swap before ENOMEM comes up? Thanks a million for any help or advice you could lend. I would be more than willing to help in any way, even providing a machine to try this on (I could sponsor an emulab.net account). Anxious to hear what options exist. Thanks again, brad ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
