Elizabeth Mattijsen wrote: >> Shared memory is not about sharing the pre-allocated memory pool (heap >> memory). Once you re-use a bit of preallocated memory the sharing goes >> away. > > > I think the phrase is Copy-On-Write, right?
yup > And since RAM is allocated > in chunks, let's assume 4K for the sake of the argument, depends on OS/version, could be 8k or different. > changing a > single byte in such a chunk causes the entire chunk to be unshared. yup. but this could never happen on the pages of read-only text, think libc.so, of course it has its own heap memory and then it's like perl. > In > older Linux kernels, I believe to have seen that when a byte gets > changed in a chunk of any child, that chunk becomes changed for _all_ > children. Newer kernels only unshare it for that particular child. > Again, if I'm not mistaken and someone please correct me if I'm wrong... I doubt this could be right, child processes cannot affect each other. Only if you change the parent, all child processes will unshare their data. > Since Perl is basically all data, you would need to find a way of > localizing all memory that is changing to as few memory chunks as > possible. My idea was just that: by filling up all "used" memory before > spawning children, you would use op some memory, but that would be > shared between all children and thus not so bad. But by doing this, you > would hopefully cause all changing data to be localized to newly > allocated memory by the children. Wish someone with more Perl guts > experience could tell me if that really is an idea that could work or > not... sort of. You can help by pre-compiling your code, then your opcode tree could be shared, if it happen to land on the page with no real dynamic variables that can change. But this won't work for true variables. Doug has plans for a much improved opcode tree sharing for mod_perl 2.0, the details are kept as a secret so far :) Watch the dev list for future revelations. Also see: http://perl.apache.org/guide/performance.html#Know_Your_Operating_System >> Shared memory is about 'text'/read-only memory pages which never get >> modified and pages that can get modified but as long as they aren't >> modified they are shared. Unfortunately (in this aspect, but >> fortunately for many other aspects) Perl is not a strongly-typed (or >> whatever you call it) language, therefore it's extremely hard to share >> memory, because in Perl almost everything is data. Though as you could >> see Bill was able to share 43M out of 50M which is damn good! > > > As a proof of concept I have run more than 100 200MB+ children on a 1 GB > RAM machine and had sharing go up so high causing the "top" number of > bytes field for shared memory to cycle through its 32-bit range multiple > times... ;-) . It was _real_ fast (had all of its data that it needed > as Perl hashes and lists) and ran ok until something would start an > avalanche effect and it would all go down in a whirlwind of swapping. > So in the end, it didn't work reliably enough ;-( But man, was it fast > when it ran... ;-) :) This topic is covered (will be) in the upcoming mod_perl book, where we include the following reference materials which you may find helpful for understanding the shared memory concepts. Chapters II (Processes) and IV (Memory Management) in I<Operating Systems: Design And Implementation>, 2nd edition. By Andrew S. Tanenbaum, Albert S. Woodhull. Prentice Hall: ISBN 0-13-638677-6 Chapter 4 (memory Management) in I<Modern Operating Systems>, 2nd edition by Andrew S. Tanenbaum. Prentice Hall: ISBN: 0-13-031358-0 Chapters 7 (Process Control) and 9 (Memory Management Policies) in I<Design of the UNIX Operating System> 1st edition, by Maurice J. Bach. Prentice Hall: ISBN 0-13-201799-7. The Solaris memory system, sizing, tools and architecture: http://www.sun.com/sun-on-net/performance/vmsizing.pdf If you have other interesting resources please let me know. Don't you love mod_perl for what it makes you learn :) _____________________________________________________________________ Stas Bekman JAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide http://perl.apache.org/guide mailto:[EMAIL PROTECTED] http://ticketmaster.com http://apacheweek.com http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
