RE: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-08 Thread Giampaolo Tomassoni
Memory management is tricky though. Hard to tell which values sum up to the real thing. Probably best meter on Linux is the actual free value highlighted below? Check it before starting amavisd/spamd/whatnot and check it again after running for a while. Also double check it after killing

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-08 Thread Matus UHLAR - fantomas
On Thu, 7 Mar 2013 14:18:12 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: I'm not talking about the semantics but about the implementation. Simply said, vfork() was developed to avoid process memory copying used at fork(). on linux, fork() does NOT copy process memory. On 07.03.13

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread Matus UHLAR - fantomas
On Thu, 7 Mar 2013 08:57:28 +0100 Giampaolo Tomassoni giampa...@tomassoni.biz wrote: I don't see too many differences with running more SA processes with linuxes (in which a fork() is actually a vfork()). On 07.03.13 07:01, David F. Skoll wrote: I don't believe that's true. Do you have

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread David F. Skoll
On Thu, 7 Mar 2013 13:47:55 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: the implementation of fork() in linux makes it nearly the same as vfork(). That is completely wrong. Just because modern forks use copy-on-write doesn't make them anything at all like vfork; the semantics are

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread Matus UHLAR - fantomas
On Thu, 7 Mar 2013 13:47:55 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: the implementation of fork() in linux makes it nearly the same as vfork(). On 07.03.13 07:53, David F. Skoll wrote: That is completely wrong. Just because modern forks use copy-on-write doesn't make them

RE: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread Giampaolo Tomassoni
On Thu, 7 Mar 2013 13:47:55 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: the implementation of fork() in linux makes it nearly the same as vfork(). That is completely wrong. Just because modern forks use copy-on-write doesn't make them anything at all like vfork; the

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread David F. Skoll
On Thu, 7 Mar 2013 14:18:12 +0100 Matus UHLAR - fantomas uh...@fantomas.sk wrote: I'm not talking about the semantics but about the implementation. Simply said, vfork() was developed to avoid process memory copying used at fork(). on linux, fork() does NOT copy process memory. vfork() also

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread Henrik K
On Thu, Mar 07, 2013 at 09:48:19AM -0500, David F. Skoll wrote: I think if you measure what happens to Perl processes that fork a number of children to handle requests, you'll see that there's very little memory sharing after a short while. Please let's stop the techno-theorizing and provide

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread David F. Skoll
On Thu, 7 Mar 2013 17:47:22 +0200 Henrik K h...@hege.li wrote: Memory measured with free (without buffers/cache etc): begin 2588084 end 1296756 About 25MB non-shared memory used per child, Are you sure your measurements are correct? I use MIMEDefang which also has a preforked-children

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread Henrik K
On Thu, Mar 07, 2013 at 11:37:33AM -0500, David F. Skoll wrote: On Thu, 7 Mar 2013 17:47:22 +0200 Henrik K h...@hege.li wrote: Memory measured with free (without buffers/cache etc): begin 2588084 end 1296756 About 25MB non-shared memory used per child, Are you sure your

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread David F. Skoll
On Thu, 7 Mar 2013 18:56:45 +0200 Henrik K h...@hege.li wrote: You provide no data how you end up with the 4MB etc. And MD is not SA, it might do all sorts of funky stuff. I wrote MD, so I'm pretty sure it's not doing any funky stuff. How about actually trying the provided spamd line

RE: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread Giampaolo Tomassoni
On Thu, Mar 07, 2013 at 11:37:33AM -0500, David F. Skoll wrote: On Thu, 7 Mar 2013 17:47:22 +0200 Henrik K h...@hege.li wrote: Memory measured with free (without buffers/cache etc): begin 2588084 end 1296756 About 25MB non-shared memory used per child, Are you sure your

Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread Henrik K
On Thu, Mar 07, 2013 at 07:02:00PM +0100, Giampaolo Tomassoni wrote: I just got a snip into my amavisd's 5 children /proc/pid/smaps file, summing together the count of Private_{Clean|Dirty} pages. I got this: p1: 74,164 kb p2: 70,772 kb p3: 71,548 kb p4: 74,064

R: Re: fork is vfork? (was Re: With similar rules, rspamd is about ten times faster than SpamAssassin.)

2013-03-07 Thread Giampaolo Tomassoni
The Private_ entries in /proc/.../smaps are reported to be the right choice here: they report only pages allocated while not shared with any other process. Ie, the ones touched after fork and the new allocated ones. Also, smaps is a relatively new proc entry, meant exactly to cope with all the