Re: FreeBSD 9.1 excessive memory allocations [SOLVED]
Quoth Unga unga...@yahoo.com: I think you may be reading too much into the malloc manpage.� When it mentions the use of per-thread small-object caches to avoid locking it's talking about performance, not thread safety.� Allocations of all sizes are thread-safe, the library just assumes that huge allocations are rare enough that it doesn't use extra per-thread resources to avoid locking for them, it just uses locking for huge blocks. Good to note all allocations are thread safe in FreeBSD. Is it by some standard that malloc should be thread safe regardless the OS (BSDs, Linux, Windows, Android, etc)? POSIX (well, SUSv4 at least) says that malloc and free must be threadsafe. Note that Windows is not a POSIX system, though I belive malloc is also always threadsafe on Windows. Ben ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: FreeBSD 9.1 excessive memory allocations [SOLVED]
On Wed, Mar 27, 2013 at 10:39:16PM -0700, Unga wrote: I think you may be reading too much into the malloc manpage.? When it mentions the use of per-thread small-object caches to avoid locking it's talking about performance, not thread safety.? Allocations of all sizes are thread-safe, the library just assumes that huge allocations are rare enough that it doesn't use extra per-thread resources to avoid locking for them, it just uses locking for huge blocks. -- Ian Good to note all allocations are thread safe in FreeBSD. Is it by some standard that malloc should be thread safe regardless the OS (BSDs, Linux, Windows, Android, etc)? The Internet is sometimes a useful resource. http://stackoverflow.com/questions/855763/malloc-thread-safe http://stackoverflow.com/questions/987444/about-thread-safety-in-malloc-and-free http://msdn.microsoft.com/en-us/library/ms235505.aspx http://msdn.microsoft.com/en-us/library/abx4dbyh.aspx On the BSDs, with gcc -pthread you should be fine. Don't ask me about Clang/LLVM. On Linux, glibc's routines are apparently thread-safe barring if the software was built with -DNO_THREADS (which seems very unlikely given the risks). Android is Linux. On Windows, assuming you're using CRT libraries, the routines are thread-safe. I get the impression the threaded library versions are also thread-safe for their *alloc equivalents. I believe HeapAlloc is thread-safe, and am pretty sure Global* is thread-safe (barring very very old Wine environments). Solaris's routines are also thread-safe, barring you're using standard malloc and not mtmalloc or ptmalloc or libumem (try searching the web for things like Solaris mtmalloc and read). All these OSes in some way or another have support for multi-threaded malloc implementations for increased performance (this is separate from locking/lock contention) where, AFAIK, you get to do the locking yourself. -- | Jeremy Chadwick j...@koitsu.org | | UNIX Systems Administratorhttp://jdc.koitsu.org/ | | Mountain View, CA, US| | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: FreeBSD 9.1 excessive memory allocations [SOLVED]
- Original Message - From: Ian Lepore i...@freebsd.org To: Unga unga...@yahoo.com Cc: freebsd-stable@freebsd.org freebsd-stable@FreeBSD.org Sent: Wednesday, March 27, 2013 2:06 PM Subject: Re: FreeBSD 9.1 excessive memory allocations On Tue, 2013-03-26 at 11:35 -0700, Unga wrote: Hi all I have a heavily threaded C application, developed on an Intel Core i5 laptop (2 cores) running FreeBSD 8.1-RELEASE. When this application compile and run on another Intel Core i7 laptop (4 cores) running FreeBSD 9.1-RELEASE, this application immediately starts grabbing memory by over 100MB per second and soon exit with not enough RAM. Both laptops having 4GB RAM. All malloc and free are mutex locked. Very rarely this problem happens on the i5 (2 cores) laptop too, but on the i7 laptop, it happens every time. Appreciate any feedback to identify and fix this issue. Best regards Unga Too many moving parts, you need to partition the problem. Is it the change in OS (and especially libraries) that causes the problem, or the change in the number of cores (more concurrent threads) is exposing some sort of application-side race condition? Given the fact that it does occur on 2 cores + freebsd 8.1, even if more rarely, it's almost surely an application problem. Perhaps you could use a tool such as valgrind to help track down the runaway allocations? Another way to expose thread race problems is to force more thread context switches. A blunt tool for doing so is to set hz=5000 or even higher in /boot/loader.conf. I've never done that on a desktop or laptop system, only on embedded systems, but it should still work okay. If changing the application code is easier, you can get a similar effect by creating a thread whose only job is to preempt other threads, by using rtprio_thread() to set it to real time priority, then just have it sleep for short random intervals (microseconds), all it does is wakes up and goes right back to sleep. Also, FYI, there's no need to use a mutex in your application to protect allocations. The memory allocator in libc is thread-safe, and in fact is particularly designed for efficient multi-threaded allocation. -- Ian Dear Tony, Alexander, Jan, Ian and Jeremy Thank you very much for your very valuable comments. Problem seems to be solved. But require more testing. 1. Fixed an application-level crucial bug. This is nearly a 7000 lines C app. It was really hard to see as the application is designed with 8 dedicated threads. 2. At run-time, this application shoots up to about 400 dynamic threads on the i7 machine, whereas on the i5 machine, it only shoots up 57 dynamic threads. It was bit scaring, therefore, made a hard limit on total number of threads to 64. Regarding mutex locks on allocations, as per the malloc(3), it says small and medium size allocations are done from per thread caches, therefore, thread-safe. My allocations are large in nature, about 5-7MB. Best regards Unga ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: FreeBSD 9.1 excessive memory allocations [SOLVED]
On Wed, 27 Mar 2013 19:33:46 +0100, Unga unga...@yahoo.com wrote: - Original Message - From: Ian Lepore i...@freebsd.org To: Unga unga...@yahoo.com Cc: freebsd-stable@freebsd.org freebsd-stable@FreeBSD.org Sent: Wednesday, March 27, 2013 2:06 PM Subject: Re: FreeBSD 9.1 excessive memory allocations On Tue, 2013-03-26 at 11:35 -0700, Unga wrote: Hi all I have a heavily threaded C application, developed on an Intel Core i5 laptop (2 cores) running FreeBSD 8.1-RELEASE. When this application compile and run on another Intel Core i7 laptop (4 cores) running FreeBSD 9.1-RELEASE, this application immediately starts grabbing memory by over 100MB per second and soon exit with not enough RAM. Both laptops having 4GB RAM. All malloc and free are mutex locked. Very rarely this problem happens on the i5 (2 cores) laptop too, but on the i7 laptop, it happens every time. Appreciate any feedback to identify and fix this issue. Best regards Unga Too many moving parts, you need to partition the problem. Is it the change in OS (and especially libraries) that causes the problem, or the change in the number of cores (more concurrent threads) is exposing some sort of application-side race condition? Given the fact that it does occur on 2 cores + freebsd 8.1, even if more rarely, it's almost surely an application problem. Perhaps you could use a tool such as valgrind to help track down the runaway allocations? Another way to expose thread race problems is to force more thread context switches. A blunt tool for doing so is to set hz=5000 or even higher in /boot/loader.conf. I've never done that on a desktop or laptop system, only on embedded systems, but it should still work okay. If changing the application code is easier, you can get a similar effect by creating a thread whose only job is to preempt other threads, by using rtprio_thread() to set it to real time priority, then just have it sleep for short random intervals (microseconds), all it does is wakes up and goes right back to sleep. Also, FYI, there's no need to use a mutex in your application to protect allocations. The memory allocator in libc is thread-safe, and in fact is particularly designed for efficient multi-threaded allocation. -- Ian Dear Tony, Alexander, Jan, Ian and Jeremy Thank you very much for your very valuable comments. Problem seems to be solved. But require more testing. 1. Fixed an application-level crucial bug. This is nearly a 7000 lines C app. It was really hard to see as the application is designed with 8 dedicated threads. 2. At run-time, this application shoots up to about 400 dynamic threads on the i7 machine, whereas on the i5 machine, it only shoots up 57 dynamic threads. It was bit scaring, therefore, made a hard limit on total number of threads to 64. Regarding mutex locks on allocations, as per the malloc(3), it says small and medium size allocations are done from per thread caches, therefore, thread-safe. My allocations are large in nature, about 5-7MB. The per thread caches are for speed. Not for thread-safety. Ronald. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: FreeBSD 9.1 excessive memory allocations [SOLVED]
On Wed, 2013-03-27 at 11:33 -0700, Unga wrote: - Original Message - From: Ian Lepore i...@freebsd.org To: Unga unga...@yahoo.com Cc: freebsd-stable@freebsd.org freebsd-stable@FreeBSD.org Sent: Wednesday, March 27, 2013 2:06 PM Subject: Re: FreeBSD 9.1 excessive memory allocations On Tue, 2013-03-26 at 11:35 -0700, Unga wrote: Hi all I have a heavily threaded C application, developed on an Intel Core i5 laptop (2 cores) running FreeBSD 8.1-RELEASE. When this application compile and run on another Intel Core i7 laptop (4 cores) running FreeBSD 9.1-RELEASE, this application immediately starts grabbing memory by over 100MB per second and soon exit with not enough RAM. Both laptops having 4GB RAM. All malloc and free are mutex locked. Very rarely this problem happens on the i5 (2 cores) laptop too, but on the i7 laptop, it happens every time. Appreciate any feedback to identify and fix this issue. Best regards Unga Too many moving parts, you need to partition the problem. Is it the change in OS (and especially libraries) that causes the problem, or the change in the number of cores (more concurrent threads) is exposing some sort of application-side race condition? Given the fact that it does occur on 2 cores + freebsd 8.1, even if more rarely, it's almost surely an application problem. Perhaps you could use a tool such as valgrind to help track down the runaway allocations? Another way to expose thread race problems is to force more thread context switches. A blunt tool for doing so is to set hz=5000 or even higher in /boot/loader.conf. I've never done that on a desktop or laptop system, only on embedded systems, but it should still work okay. If changing the application code is easier, you can get a similar effect by creating a thread whose only job is to preempt other threads, by using rtprio_thread() to set it to real time priority, then just have it sleep for short random intervals (microseconds), all it does is wakes up and goes right back to sleep. Also, FYI, there's no need to use a mutex in your application to protect allocations. The memory allocator in libc is thread-safe, and in fact is particularly designed for efficient multi-threaded allocation. -- Ian Dear Tony, Alexander, Jan, Ian and Jeremy Thank you very much for your very valuable comments. Problem seems to be solved. But require more testing. 1. Fixed an application-level crucial bug. This is nearly a 7000 lines C app. It was really hard to see as the application is designed with 8 dedicated threads. 2. At run-time, this application shoots up to about 400 dynamic threads on the i7 machine, whereas on the i5 machine, it only shoots up 57 dynamic threads. It was bit scaring, therefore, made a hard limit on total number of threads to 64. Regarding mutex locks on allocations, as per the malloc(3), it says small and medium size allocations are done from per thread caches, therefore, thread-safe. My allocations are large in nature, about 5-7MB. Best regards Unga I think you may be reading too much into the malloc manpage. When it mentions the use of per-thread small-object caches to avoid locking it's talking about performance, not thread safety. Allocations of all sizes are thread-safe, the library just assumes that huge allocations are rare enough that it doesn't use extra per-thread resources to avoid locking for them, it just uses locking for huge blocks. -- Ian ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: FreeBSD 9.1 excessive memory allocations [SOLVED]
I think you may be reading too much into the malloc manpage. When it mentions the use of per-thread small-object caches to avoid locking it's talking about performance, not thread safety. Allocations of all sizes are thread-safe, the library just assumes that huge allocations are rare enough that it doesn't use extra per-thread resources to avoid locking for them, it just uses locking for huge blocks. -- Ian Good to note all allocations are thread safe in FreeBSD. Is it by some standard that malloc should be thread safe regardless the OS (BSDs, Linux, Windows, Android, etc)? Unga ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
