Re: Increase memory limit ?
From: Chuck Swiger [EMAIL PROTECTED] Hmm. Are you certain: 12-sec# dd if=/dev/zero of=/var/cache/bar bs=1024k count=64 64+0 records in 64+0 records out 67108864 bytes transferred in 18.790176 secs (3571487 bytes/sec) 13-sec# ls -l /var/cache/bar -rw-r--r-- 1 root wheel 67108864 Jan 16 22:44 /var/cache/bar ...since that should have created a 64 MB swapfile. Yeah, you are right. It is only 64MB. Anyway, it sounds like you've made good progress. (Congratulations.) Thanks, Chuck, you are very helpful! Frank _ MSN 8 helps eliminate e-mail viruses. Get 2 months FREE* http://join.msn.com/?page=features/virus To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
I would like to thank the help from the following friends: Chuck Swiger [EMAIL PROTECTED],Duncan Anker [EMAIL PROTECTED],Roman Neuhauser [EMAIL PROTECTED],Greg 'groggy' Lehey [EMAIL PROTECTED] for solving the memory limit problem. Thanks very much! Sincerely, Frank _ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
Frank Li wrote: [ ... ] Cool! I created an additional swapfile according to its instruction. Everything is smooth. Just one thing not quite understood. # dd if=/dev/zero of=/usr/swap0 bs=1024k count=64 It actually created a 64GB swapfile. Hmm. Are you certain: 12-sec# dd if=/dev/zero of=/var/cache/bar bs=1024k count=64 64+0 records in 64+0 records out 67108864 bytes transferred in 18.790176 secs (3571487 bytes/sec) 13-sec# ls -l /var/cache/bar -rw-r--r-- 1 root wheel 67108864 Jan 16 22:44 /var/cache/bar ...since that should have created a 64 MB swapfile. As we knew virtual memory space on 32bit bsd cannot exceed 2GB, is 64GB for multiple processes/programs, or multiple users? The 2 GB of user-addressable virtual address space is per-process. Anyway, it sounds like you've made good progress. (Congratulations.) -Chuck To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
put your replies *below* the text you reply to. most questions@ subscribers are used to reading from top to bottom, from left to right. # [EMAIL PROTECTED] / 2003-01-15 22:02:49 -0600: 32-bit systems implementing VM typically could increase user-mode address space up to 2 GB, although variants on that and other things (ie, where devices get mapped into memory) make that only an approximation. For FreeBSD: # Certain applications can grow to be larger than the 128M limit # that FreeBSD initially imposes. Below are some options to # allow that limit to grow to 256MB, and can be increased further # with changing the parameters. MAXDSIZ is the maximum that the # limit can be set to, and the DFLDSIZ is the default value for # the limit. MAXSSIZ is the maximum that the stack limit can be # set to. You might want to set the default lower than the max, # and explicitly set the maximum with a shell command for processes # that regularly exceed the limit like INND. # options MAXDSIZ=(256*1024*1024) options MAXSSIZ=(256*1024*1024) options DFLDSIZ=(256*1024*1024) Depending on what you're doing, you might well find using a 64-bit platform (Alpha hardware? Solaris on SPARC?) to be more appropriate... Which file should I put in the following ? in your kernel configuration file, look in /usr/src/sys/i386/conf/ -- If you cc me or remove the list(s) completely I'll most likely ignore your message.see http://www.eyrie.org./~eagle/faqs/questions.html To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
Frank Li wrote: [ ... ] Should I add more physical memory (if so should I reinstll OS)? If you're actually doing something where the 512 MB datasize limit matters to you, adding more physical memory will almost certainly speed things up. No, don't reinstall; even Windows doesn't make you reinstall the OS just because you added memory. :-) Can I just set some parameters(what?) and recompile kernel and increase these limits ? 32-bit systems implementing VM typically could increase user-mode address space up to 2 GB, although variants on that and other things (ie, where devices get mapped into memory) make that only an approximation. For FreeBSD: # Certain applications can grow to be larger than the 128M limit # that FreeBSD initially imposes. Below are some options to # allow that limit to grow to 256MB, and can be increased further # with changing the parameters. MAXDSIZ is the maximum that the # limit can be set to, and the DFLDSIZ is the default value for # the limit. MAXSSIZ is the maximum that the stack limit can be # set to. You might want to set the default lower than the max, # and explicitly set the maximum with a shell command for processes # that regularly exceed the limit like INND. # options MAXDSIZ=(256*1024*1024) options MAXSSIZ=(256*1024*1024) options DFLDSIZ=(256*1024*1024) Depending on what you're doing, you might well find using a 64-bit platform (Alpha hardware? Solaris on SPARC?) to be more appropriate... -Chuck To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
Thanks, Roman, options MAXDSIZ=(256*1024*1024) options MAXSSIZ=(256*1024*1024) options DFLDSIZ=(256*1024*1024) Depending on what you're doing, you might well find using a 64-bit platform (Alpha hardware? Solaris on SPARC?) to be more appropriate... Which file should I put in the following ? in your kernel configuration file, look in /usr/src/sys/i386/conf/ So, I need to recompile kernel ? I guess in this case I'd better back up things in case the kernel no longer boots up... Frank _ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
On Fri, 2003-01-17 at 07:59, Frank Li wrote: Thanks, Roman, options MAXDSIZ=(256*1024*1024) options MAXSSIZ=(256*1024*1024) options DFLDSIZ=(256*1024*1024) Depending on what you're doing, you might well find using a 64-bit platform (Alpha hardware? Solaris on SPARC?) to be more appropriate... Which file should I put in the following ? in your kernel configuration file, look in /usr/src/sys/i386/conf/ So, I need to recompile kernel ? I guess in this case I'd better back up things in case the kernel no longer boots up... You can also tune this at boot time by adding them to the file /boot/loader.conf. e.g. kern.maxdsiz=(256*1024*1024) Although I am not 100% certain whether the tunables parser actually calculates that, I haven't checked the source too closely. I play it safe by using the actual number. On other thing to beware of, whether you set this in the kernal or /boot/loader.conf, is that you cannot go over 2^31-1 because FreeBSD uses a signed int for this value (I tried setting it to 2G once and MySQL refused to start - MySQL AB verified the problem). -- The information contained in this email is confidential. If you are not the intended recipient, you may not disclose or use the information in this email in any way. Dark Blue Sea does not guarantee the integrity of any emails or attached files. The views or opinions expressed are the author's own and may not reflect the views or opinions of Dark Blue Sea. Dark Blue Sea does not warrant that any attachments are free from viruses or other defects. You assume all liability for any loss, damage or other consequences which may arise from opening or using the attachments. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
Thanks for all of your replies,
Now I can do it through recompiling the kernel and the limits did increase.
I haven't tried whether it can increase over 2G (I would love that if it
can!). The code I used cannot be easily changed to reduce memory
consumption but I think 2G would probably be enough for some time. Indeed,
I found the max value I could set is 2048*1024*1024-1, bigger than that will
result in a negative value shown by limit.
You can also tune this at boot time by adding them to the file
/boot/loader.conf. e.g.
kern.maxdsiz=(256*1024*1024)
I added this (though it was actually 1024*1024*1024) to /boot/loader.conf
and also /boot/defaults/loader.conf, and booted the GENERIC kernel. The
limits were still the old ones (i.e.512M). It did not work I don't know
why, but I am already happy as recompiling kernel worked anyway.
The final problem now becomes: Initially I made the swap partition to be 1GB
and there is no more free partition on my harddisk. This value now actually
becomes the bottleneck. Repartitioning will lose all my data and also
applications installed, and thus is very high cost operation. Is there any
easy way to increase the swap partition ? Say, add one more harddisk(need
to tell the OS?)? For me, the easiest way I prefer is to use a file (under
/usr say, as I have lots of space there) associated with a disk device by
vnconfig for the swapping. It seems not difficult to do so. By looking at
/etc/rc, I noticed the following:
# Add additional swapfile, if configured.
#
case ${swapfile} in
[Nn][Oo] | '')
;;
*)
if [ -w ${swapfile} -a -c /dev/vn0b ]; then
echo Adding ${swapfile} as additional swap
vnconfig -e /dev/vn0b ${swapfile} swap
fi
;;
esac
The swapfile will be associated as an additional swap device (i.e./dev/vn0b
which does exist in my system). But I didn't find where the variable
${swapfile} is set. Do you know ?
I tried adding one line swapfile=/usr/someone/swap1 and created a file
/usr/someone/swap1 with vi editor by typing some characters in. Then boot
the system, I saw the booting message said:
Adding /usr/someone/swap1 as additional swap
vnconfig:
swapon
:
Device not configured:
Doing intial network setup
I am not sure if it actually already worked or not.
Thanks!
Frank
_
Protect your PC - get McAfee.com VirusScan Online
http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
On Fri, 2003-01-17 at 12:08, Frank Li wrote: Thanks for all of your replies, Now I can do it through recompiling the kernel and the limits did increase. I haven't tried whether it can increase over 2G (I would love that if it can!). The code I used cannot be easily changed to reduce memory consumption but I think 2G would probably be enough for some time. Indeed, I found the max value I could set is 2048*1024*1024-1, bigger than that will result in a negative value shown by limit. Yes, 2G - 1 is the maximum size a signed int can hold ... mind you, any program needing more that 2G of memory really should be implementing on-disk storage of data for itself. You can also tune this at boot time by adding them to the file /boot/loader.conf. e.g. kern.maxdsiz=(256*1024*1024) I added this (though it was actually 1024*1024*1024) to /boot/loader.conf and also /boot/defaults/loader.conf, and booted the GENERIC kernel. Yes, I just copied the numbers from what was given above - I should have substituted 1024 or more in there. The limits were still the old ones (i.e.512M). It did not work I don't know why, but I am already happy as recompiling kernel worked anyway. It may be the case that you cannot specify the value like that in loader.conf, in which case we must do the math ourselves. I am glad your problem is solved. The final problem now becomes: Initially I made the swap partition to be 1GB and there is no more free partition on my harddisk. This value now actually becomes the bottleneck. Repartitioning will lose all my data and also applications installed, and thus is very high cost operation. Is there any easy way to increase the swap partition ? Read this, it may help: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/adding-swap-space.html -- The information contained in this email is confidential. If you are not the intended recipient, you may not disclose or use the information in this email in any way. Dark Blue Sea does not guarantee the integrity of any emails or attached files. The views or opinions expressed are the author's own and may not reflect the views or opinions of Dark Blue Sea. Dark Blue Sea does not warrant that any attachments are free from viruses or other defects. You assume all liability for any loss, damage or other consequences which may arise from opening or using the attachments. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Increase memory limit ?
The situation is as follows: Physical memory is 128M, OS is FreeBSD 4.4. My C++ simulation code mallocs large amount of memory. When running, I found top shows SIZE is 514M, RES is 176M. Then the code dumpped a core. The code itself should have no problem. It's not complicated and it runs very well when running shorter. Top shows: Mem: 180M Active, 21M Inact, 32M Wired, 13M Cache, 35M Buf, 656K Free Abort trap - core dumped . PID USERNAME PRI NICE SIZERES STATETIME WCPUCPU COMMAND 66351 dxu -22 0 514M 127M swread 0:11 36.96% 27.39% simulation limit shows: cputime unlimited filesizeunlimited datasize524288 kbytes stacksize 65536 kbytes coredumpsizeunlimited memoryuse unlimited descriptors 1064 memorylockedunlimited maxproc 531 I tried to increase the above limits but failed. They seem to be already the maximum. Should I add more physical memory (if so should I reinstll OS)? Can I just set some parameters(what?) and recompile kernel and increase these limits ? Thanks a lot! Frank _ The new MSN 8 is here: Try it free* for 2 months http://join.msn.com/?page=dept/dialup To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
On Wednesday, 15 January 2003 at 18:36:31 -0600, Frank Li wrote: The situation is as follows: Physical memory is 128M, OS is FreeBSD 4.4. My C++ simulation code mallocs large amount of memory. When running, I found top shows SIZE is 514M, RES is 176M. That's rather difficult on a machine with only 128 MB. Then the code dumpped a core. The code itself should have no problem. It's not complicated and it runs very well when running shorter. Top shows: Mem: 180M Active, 21M Inact, 32M Wired, 13M Cache, 35M Buf, 656K Free This shows that you have much more than 128 MB of memory in the machine. And how much swap? Abort trap - core dumped . PID USERNAME PRI NICE SIZERES STATETIME WCPUCPU COMMAND 66351 dxu -22 0 514M 127M swread 0:11 36.96% 27.39% simulation limit shows: cputime unlimited filesizeunlimited datasize524288 kbytes stacksize 65536 kbytes coredumpsizeunlimited memoryuse unlimited descriptors 1064 memorylockedunlimited maxproc 531 I tried to increase the above limits but failed. They seem to be already the maximum. No, you need to set them in /etc/login.conf. Should I add more physical memory That might speed things up, but it shouldn't make any other difference. (if so should I reinstll OS)? NO! You don't reinstall the system, ever. Well, almost never. Just reboot. Can I just set some parameters(what?) and recompile kernel and increase these limits ? No. Just reboot. The real issue is why your simulation is dying. Do you have any messages in /var/log/messages indicating that you're out of swap? Greg -- When replying to this message, please copy the original recipients. If you don't, I may ignore the reply or reply to the original recipients. For more information, see http://www.lemis.com/questions.html See complete headers for address and phone numbers To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
Thanks, Greg, From: Greg 'groggy' Lehey [EMAIL PROTECTED] Top shows: Mem: 180M Active, 21M Inact, 32M Wired, 13M Cache, 35M Buf, 656K Free This shows that you have much more than 128 MB of memory in the machine. I checked it again (by rebooting it and seeing its booting messages, stupid method, any good method ? uname -a did not give me the physical memory size). The physical memory is actually 256MB. And how much swap? I forgot. Any command to check that ? Here is the partition info though: Filesystem 1K-blocks UsedAvail Capacity Mounted on /dev/ad0s1a19839980463 10206544%/ /dev/ad0s1f19839930485 15204317%/tmp /dev/ad0s1g 42032558 7019929 3165002518%/usr /dev/ad0s1e19839928338 15419016%/var procfs 440 100%/proc I vaguely remember it's about 2 times of physical memory as recommended by BSD. So it should be about 512MB. How much should I set it, in this case now ? I need a lot of memory. I tried to increase the above limits but failed. They seem to be already the maximum. No, you need to set them in /etc/login.conf. Here is what I see in this file. It seems already maximum. :cputime=unlimited:\ :datasize=unlimited:\ :stacksize=unlimited:\ :memorylocked=unlimited:\ :memoryuse=unlimited:\ :filesize=unlimited:\ :coredumpsize=unlimited:\ :openfiles=unlimited:\ :maxproc=unlimited:\ :sbsize=unlimited:\ :priority=0:\ The real issue is why your simulation is dying. Do you have any messages in /var/log/messages indicating that you're out of swap? Yes. Here is what I found in the log: Dec 30 01:05:05 machi /kernel: pid 28666 (simulation), uid 1000, was killed: out of swap space Dec 30 01:05:05 machi /kernel: swap_pager_getswapspace: failed So, how can I increase swap space? Thanks very much! Frank _ MSN 8 with e-mail virus protection service: 2 months FREE* http://join.msn.com/?page=features/virus To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
Thanks, Chuck, Which file should I put in the following ? Frank 32-bit systems implementing VM typically could increase user-mode address space up to 2 GB, although variants on that and other things (ie, where devices get mapped into memory) make that only an approximation. For FreeBSD: # Certain applications can grow to be larger than the 128M limit # that FreeBSD initially imposes. Below are some options to # allow that limit to grow to 256MB, and can be increased further # with changing the parameters. MAXDSIZ is the maximum that the # limit can be set to, and the DFLDSIZ is the default value for # the limit. MAXSSIZ is the maximum that the stack limit can be # set to. You might want to set the default lower than the max, # and explicitly set the maximum with a shell command for processes # that regularly exceed the limit like INND. # options MAXDSIZ=(256*1024*1024) options MAXSSIZ=(256*1024*1024) options DFLDSIZ=(256*1024*1024) Depending on what you're doing, you might well find using a 64-bit platform (Alpha hardware? Solaris on SPARC?) to be more appropriate... -Chuck _ MSN 8 helps eliminate e-mail viruses. Get 2 months FREE* http://join.msn.com/?page=features/virus To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: Increase memory limit ?
On Wednesday, 15 January 2003 at 21:54:54 -0600, Frank Li wrote: Thanks, Greg, From: Greg 'groggy' Lehey [EMAIL PROTECTED] Top shows: Mem: 180M Active, 21M Inact, 32M Wired, 13M Cache, 35M Buf, 656K Free This shows that you have much more than 128 MB of memory in the machine. I checked it again (by rebooting it and seeing its booting messages, stupid method, any good method ? uname -a did not give me the physical memory size). No, you need dmesg for that. For example: CPU: AMD Athlon(tm) XP processor 1700+ (1462.51-MHz 686-class CPU) Origin = AuthenticAMD Id = 0x662 Stepping = 2 Features=0x383f9ffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE AMD Features=0xc048MP,AMIE,DSP,3DNow! real memory = 536870912 (524288K bytes) avail memory = 513773568 (501732K bytes) The physical memory is actually 256MB. And how much swap? I forgot. Any command to check that ? pstat -s. But it was the line below the one you quoted from top. I vaguely remember it's about 2 times of physical memory as recommended by BSD. So it should be about 512MB. How much should I set it, in this case now ? It depends on what you need. I need a lot of memory. In your case, I'd create more swap. But first you should see how much you're using. I tried to increase the above limits but failed. They seem to be already the maximum. No, you need to set them in /etc/login.conf. Here is what I see in this file. It seems already maximum. :cputime=unlimited:\ :datasize=unlimited:\ :stacksize=unlimited:\ :memorylocked=unlimited:\ :memoryuse=unlimited:\ :filesize=unlimited:\ :coredumpsize=unlimited:\ :openfiles=unlimited:\ :maxproc=unlimited:\ :sbsize=unlimited:\ :priority=0:\ That's not the entire file. There are multiple classes. But this doesn't seem to be your issue. The real issue is why your simulation is dying. Do you have any messages in /var/log/messages indicating that you're out of swap? Yes. Here is what I found in the log: Dec 30 01:05:05 machi /kernel: pid 28666 (simulation), uid 1000, was killed: out of swap space Dec 30 01:05:05 machi /kernel: swap_pager_getswapspace: failed So, how can I increase swap space? Create another swap partition. If you really can't do that, you'll have to create a vnode device with vnconfig(8), but I don't recommend that. In this case, yes, adding more memory might solve the problem. Greg -- When replying to this message, please copy the original recipients. If you don't, I may ignore the reply or reply to the original recipients. For more information, see http://www.lemis.com/questions.html See complete headers for address and phone numbers To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
