All,
Been a long time since I posted anything but I wanted to give some number
for the group as well.
Just an FYI it also maybe hardware and version dependent, that is to say
that the performance measurements are not cpu bound therefor it may be IO
that is saturated which will happen on either
I belive I have found the memleak for Peters issue:
Index: dbmail-mailbox.c
===
--- dbmail-mailbox.c(revision 2574)
+++ dbmail-mailbox.c(working copy)
@@ -1312,7 +1312,7 @@
GTree * dbmail_mailbox_get_set(struct
cool! But rev 2579 doesn't seem to include this particular issue as well
as what may be one with dbmail_imap_session_bodyfetch_rewind which I
don't have a quick answer as to how much of an issue it is?
-leif
Paul J Stevens wrote:
Leif Jackson wrote:
I belive I have found the memleak
instance? If not I supsect g_mime_init and shutdown in the session will
cause untold problems.
Just my .02
Thanks,
Leif
Paul J Stevens wrote:
Leif Jackson wrote:
I belive I have found the memleak for Peters issue:
It's one instance of a whole class of leakage we uncovered here over
(imapcommands.c:1790)
==28403==by 0x804D0DF: IMAPClientHandler (imap4.c:290)
==28403==
I am out of time for today but maybe Paul or Aaron will see something
and we can put this one to rest. Thanks for you help peter.
-leif
Peter Rabbitson wrote:
Leif Jackson wrote:
Peter try 2579 with my
it was cause you tried to use my patch from the old code and
Paul's changes might have not helped. let me know
Thanks,
Leif
Peter Rabbitson wrote:
Leif Jackson wrote:
Peter try 2579 with my changes from the previous message.
Same. Logs at
http://rabbit.us/pool/misc
Peter,
You need to install libtool, I got the same thing on one of my devel
boxes, once libtool is installed re-run autoreconf. This svn version has
now been running for a while on a semi-production box and it looks like
imapd doesn't leak memory anymore.
-leif
Peter Rabbitson wrote:
it will make a file per pid
and log the leaks that arn't suppressed by the valgrind suppession file.
That should help us track this down futher. As arron said the options
that you specified may also be needed but the tool should be memcheck.
Thanks for helping track this down.
Thanks,
Leif
Humm looks like it might be in the ic_fetch:
==26918== 37,800 bytes in 54 blocks are still reachable in loss record
23 of 27
==26918==at 0x40235AB: realloc (vg_replace_malloc.c:306)
==26918==by 0x428CF4A: vasprintf (in /lib/i686/cmov/libc-2.5.so)
==26918==by 0x41708C6: g_vasprintf
Anton Paul,
humm sorry I missed the fact it was a static variable, but there is
some wierdness with this code then and gmime/glib as far as valgrind is
concerned. I will move on to other things for now.
Thanks,
Leif
Paul J Stevens wrote:
Anton Zakatov wrote:
Paul and Leif, in
Paul J Stevens wrote:
Leif,
could you plz resent this patch as attachment? I can't get it to apply
like this.
You can simply do 'svn diff misc.c' to produce valid patches. No need to
keep an original tree around.
Thats really cool, still learning svn. Anyway the svn diff for misc.c is
from the ingres driver (unfinished) there's not much to see in
the trunk atm.
Paul J Stevens wrote:
Leif,
You should stay away from trunk for now. I merge changes into the trunk
from time to time but it's the dbmail_2_2_branch where current
development is taking place.
Leif Jackson wrote
Cool I just got the 2_2 branch from svn is this suposed to have the
configure script and buildtools? usualy i use autoreconf -f -i to get
the configure script. It seems the buildtools and configure script are
not allowing me to build on my x86_64 I have a -fPIC error, Aaron don't
worrie about
Aaron it seem the path I sent with the config initializers got taken
back out other than the change to dsn.c that is all this patch is.
Otherwise the 2_2 branch is looking good as far as memleaks :)
I was looking at the dbmail-message.c code and saw a new function
convert_8bit_field_to_utf8
[EMAIL PROTECTED] wrote:
--
aaron - 24-Mar-07 22:19
--
Oh crap, I've confirmed that I SIGHUP handling very badly.
Probably will rearrange the code more
Just a suggestion :) I belive without closing it everywhere in that
function there are still leaks from iconv.
-leif
diff -urN dbmail-svn-2.2.4-2480.leif2/misc.c
dbmail-svn-2.2.4-2480.leif3/misc.c
--- dbmail-svn-2.2.4-2480.leif2/misc.c 2007-03-24 18:40:07.0 -0400
+++
Paul Aaron,
I was reviewing the current trunk 2474 from today, to see if I can help
make 2.2.5 bug free :) Anyway I came across some possible memleak issues
patch is attached. This is from valgrind I deduced the issues. Please
verify and apply to the trunk if you see fit.
Thanks,
Leif
I just pulled a fresh copy to make sure that it wasn't me this time paul
but as I see it in trunk _ic_select in imapcommands.c
looks like this:
/* show idx of first unseen msg (if present) */
if (ud-mailbox.exists) {
if (! (key = db_first_unseen(ud-mailbox.uid))) {
(ud-mailbox.uid))) {
+ if ((key = db_first_unseen(ud-mailbox.uid)) == (u64_t)
(-1)) {
dbmail_imap_session_printf(self, * BYE internal
dbase error\r\n);
return -1;
}
Thanks,
Leif Jackson
=
db_first_unseen(ud-mailbox.uid))) { + if ((key =
db_first_unseen(ud-mailbox.uid)) == (u64_t) (-1)) {
dbmail_imap_session_printf(self, * BYE internal dbase error\r\n); return
-1;
}
Thanks,
Leif Jackson
___
Dbmail-dev mailing list
Aaron,
From your wiki page:
On the issue of a database connection pool, I think that issue is mooted
by this architecture. There are relatively few commands that do not
require any database interaction, and the ones that dont are so fast to
process as to not warrant the additional abstraction
Paul,
even though these warning are harmless they should be fixed, this is
minor yes but still will save you and aaron and the list time not
answering this question again so I belive this is the best fix:
diff -urNb dbmail-svn-2.1.8-2270.orig/modules/dbmysql.c
can you please also get the row is NULL above the result is NULL I missed
that in my patch.
Thanks,
Leif
On Tue, September 19, 2006 9:23 am, Paul J Stevens wrote:
fix applied (to dbpgsql.c as well).
thanks,
Leif Jackson wrote:
Paul,
even though these warning are harmless they should
We could make the query string dynamicly size it self with a realloc or
somthing to that effect, then the question just would be would that open
up a DDoS of some sort... But just for the utils and such that might come
accross a query larger than the default it might solve and issue such as
this
Guys,
It seems current trunk is broken for compiling with sieve, the define
IMAP_NFLASGS is missing where should that be? also typo on the g_free.
Thanks,
Leif
diff -urNab dbmail-svn-2.1.7-2218.orig/modules/sortsieve.c
dbmail-svn-2.1.7-2218/modules/sortsieve.c
---
DK,
I have them at 95 with a chkconfig version of the dbmail init scripts on
CentOS 4.3 64bit But as long as they are after your network and if sql is
on the same machine your mysql startup it should be fine at any init
number.
-leif
On Wed, July 26, 2006 7:37 pm, DK wrote:
One more thing
Martin,
Based on the link with the logs and your message in question I tried to
reproduce this issue under valgrind and was unable to with 2.1.7
dbmail-smtp as well as the retrival of the message after storage matches
the original. So I would guess that this sig9 is somewhere in your exim
chain
Paul Aaron,
It seems that there is a minor memory leak in the 2.1.7 release. Sorry.
see minor patch below.
diff -urNab dbmail-2.1.7.orig/db.c dbmail-2.1.7/db.c
--- dbmail-2.1.7.orig/db.c 2006-07-24 11:35:41.0 -0400
+++ dbmail-2.1.7/db.c 2006-07-26 02:04:47.596208079 -0400
@@
On Wed, July 26, 2006 3:38 am, UEMURA (fka. MAENAKA) Tetsuya wrote:
Posted on Wed, 26 Jul 2006 02:55:20 -0400 (EDT)
by author Leif Jackson [EMAIL PROTECTED]
It seems that there is a minor memory leak in the 2.1.7 release. Sorry.
see minor patch below.
Leif, your patch isn't a minor
Cool! :)
-leif
On Sat, May 13, 2006 9:43 pm, Aaron Stone wrote:
Thanks! There was even one more in dbmail-imapsession, if !ln failed
then re was not freed, about three lines above your fix. Now in SVN.
Aaron
On Sat, 2006-05-13 at 15:28 -0400, Leif Jackson wrote:
Guys,
Hey looking
tests in 140.009s
FAILED (failures=1)
Is this truly a failure? or can you update testimap.py to use the correct
check? ..etc.
Thanks,
leif
On Tue, February 28, 2006 5:29 pm, Leif Jackson wrote:
On Tue, February 28, 2006 4:59 pm, Aaron Stone wrote:
On Tue, Feb 28, 2006, Leif Jackson [EMAIL
On Tue, February 28, 2006 5:54 pm, Leif Jackson wrote:
Aaron Paul,
I belive I have come up with a fix for this and I have also found a few
extra possible memory leaks, I will submit a patch for your review here in
a bit
what i tried didn't quite workout right so Aaron if you have some
See attached valgrind suppression file. I have added the libmodule leaks
to the suppression file. So it make debugging the other stuff a bit
better. Didn't know if you guys had already done this Should the
suppressions file be included in contrib ?
Aaron ..etc. I have also attached the
On Wed, March 1, 2006 1:03 am, Aaron Stone wrote:
On Tue, 2006-02-28 at 19:02 -0500, Leif Jackson wrote:
On Tue, February 28, 2006 5:54 pm, Leif Jackson wrote:
I belive I have come up with a fix for this and I have also found a
few extra possible memory leaks, I will submit a patch for your
a
patch if I can resolve them or a updated dbmail.supp if they are glib
related.
Thanks!
-leif
On Wed, March 1, 2006 1:32 am, Leif Jackson wrote:
On Wed, March 1, 2006 1:03 am, Aaron Stone wrote:
On Tue, 2006-02-28 at 19:02 -0500, Leif Jackson wrote:
On Tue, February 28, 2006 5:54 pm, Leif
*) dsnuser-address);
dm_free((char *) dsnuser-mailbox);
*/
dsnuser-address = NULL;
Thanks,
Leif
p.s. fix these 2 minor issues and then rel 2.1.4 :) blah..blah just my .02
as always.
On Wed, March 1, 2006 3:09 am, Leif Jackson wrote:
Disregard this. It appears it is all calloc
On Tue, February 28, 2006 3:42 pm, Aaron Stone wrote:
On Tue, Feb 28, 2006, Paul J Stevens [EMAIL PROTECTED] said:
Aaron Stone wrote:
This fixes it for me:
--- ../dbmail/imapd.c 2006-02-28 04:15:22.556837664 -0800
+++ imapd.c 2006-02-28 10:16:50.398784536 -0800
case 0: /* child
On Tue, February 28, 2006 4:59 pm, Aaron Stone wrote:
On Tue, Feb 28, 2006, Leif Jackson [EMAIL PROTECTED] said:
Guys before 2.1.4 rel, what about that memory leak issue in imapd?, did
you find a workaround? This is the malloc for () keys in the command
parser I am talking about.
Oh. Hrmm
On Tue, February 21, 2006 1:42 am, Oleg Lapshin wrote:
But there is error: dbmysql.c,db_query: res was not freed after the
previous query!
This isn't really an error just a warning message to developers about the
fact that on the previous call to db_query they didn't free the result
before
On Fri, February 17, 2006 10:29 pm, Aaron Stone wrote:
On Fri, 2006-02-17 at 17:35 -0500, Leif Jackson wrote:
- dbmail_message_set_header(msg,
Return-Path, from-data);
+ dbmail_message_set_header(msg,
Return-Path
On Sat, February 18, 2006 2:57 am, Aaron Stone wrote:
On Sat, 2006-02-18 at 01:59 -0500, Leif Jackson wrote:
On Fri, February 17, 2006 10:29 pm, Aaron Stone wrote:
I'm trying to figure out why sortmodule.c doesn't seem to be building
unless --with-sieve is specified. The Makefile.am
On Sat, February 18, 2006 3:33 pm, Aaron Stone wrote:
On Sat, 2006-02-18 at 04:47 -0500, Leif Jackson wrote:
Either that or #ifdef (SIEVE || SHARED). The jury is out.
You know as painfull as it can be this may be a good time to work out a
loadable at runtime module system using function
) $(am__objects_3) \
[EMAIL PROTECTED]@ $(am__objects_4)
libdbmail_la_OBJECTS = $(am_libdbmail_la_OBJECTS)
@[EMAIL PROTECTED] = check_dbmail_common$(EXEEXT) \
@WITHCHECK_TRUE@ check_dbmail_server$(EXEEXT) \
On Sat, February 18, 2006 3:58 pm, Leif Jackson wrote:
On Sat, February 18, 2006 3:33 pm
Current svn build problems... minor fixes.. Without sieve enabled there is
a missing ifdef and the Return-Path argument is of the wrong type...
diff -urN dbmail-svn-2.1.3-1987.orig/lmtp.c dbmail-svn-2.1.3-1987/lmtp.c
--- dbmail-svn-2.1.3-1987.orig/lmtp.c 2006-02-17 17:23:43.0 -0500
+++
On Fri, February 17, 2006 5:35 pm, Leif Jackson wrote:
Current svn build problems... minor fixes.. Without sieve enabled there
is a missing ifdef and the Return-Path argument is of the wrong type...
diff -urN dbmail-svn-2.1.3-1987.orig/lmtp.c dbmail-svn-2.1.3-1987/lmtp.c
--- dbmail-svn-2.1.3
Aaron,
Thanks for getting this into CVS, odd tho the md5 issue didn't seem to
leak anywhere else. But I am sure it would have. Any ideas for the
currently supressed build_args_array_ext leak, this is kinda a big issue
in my mind as it leaks anywhere from 16bytes to 1600bytes on each
connection
Hey guys Just taking a new look at the code you guys have been working
so hard on. Looks really good! I have a minor patch that doesn't even
warrent a bug report so I have attached it here.
On another note I wanted to know if either Arron or Paul have tried the
current svn trunk on x86_64 bit
to
work on the code for 2.0 as it is limted by the parsing system and other
issues people are improving on in 2.1.
just my .02
-leif
I'm putting the two strands of this thread back together ;-) ...
On Tue, Jan 25, 2005, Leif Jackson [EMAIL PROTECTED] said:
That US-ASCII isn't recognized
On Tue, Jan 25, 2005, Paul J Stevens [EMAIL PROTECTED] said:
I was pleasantly surprised by your command trace yesterday. Apparently
c-client is smart. Smart enough to hide server-errors from the client.
The fact that
A01 SORT (REVERSE ARRIVAL) US-ASCII ALL
failed is very important in this
Aaron Stone wrote:
On Tue, Jan 25, 2005, Paul J Stevens [EMAIL PROTECTED] said:
I was pleasantly surprised by your command trace yesterday. Apparently
c-client is smart. Smart enough to hide server-errors from the client.
The fact that
A01 SORT (REVERSE ARRIVAL) US-ASCII ALL
failed is very
Aaron,
I've reverted _ic_sort behaviour to pre-2.0.3. Can you confirm this wrt
twig?
This has given me something to think about:
- Leif interleaved the sort with the search code. There are some valid
reason's for doing so, but we need to
treat the search arguments fundamentally
On Wed, Sep 29, 2004 at 04:24:38PM +0200, Paul J Stevens wrote:
Agreed. And that's what I'm working on already. However, doing so would
be
a lot easier if we implement some kind of test-frame.
Its not a solution, tho it may help. A real solution as I said earlier is
to simplify processes,
Just check out the nfg-0-1 branch. Its in a unfinished state at the
moment, so
you won't be able to compile it completely yet. Like I said, I'm removing
the
mime, cache, and list code, so you can imagine there's a lot of changes
going
into imapcommands.c and imaputils.c.
OK will do going
Leif Jackson wrote:
Could you please explain how Oracle's connection overhead directly
translates to we need threads? A new thread still has to open up a new
connection to the DB. It sounds to me that Oracle's connection overhead
(or Postgresql or whatever) is more an argument for persistent
Leif Jackson wrote:
Ok, so you are using thread pools to implement persistent connections.
I would still like to hear more details about the 60% speedup of the
threaded pop3 daemon vs the non-threaded. Exactly what was 60% faster?
I still think that the Multi-Process Model can be just
Gerrit P. Haase wrote:
Leif Jackson wrote:
Hello,
please consider using one single process with threads, for each client
one thread, instead the slower 'fork a child for every client' or
Apache like pre-fork technique.
Our current 2.1 cvs head has pre-fork and I am currently working
Along with this I would like to start a discussion on the possiblity for
doing away with the fork code and going to a pthread model with a thread
pool, I have already starting some proof of concept code on my own, but
with the performance on threads in linux 2.6.8 and the performance of
threads on
If we're using Glib, then let's also use Glib's threading interface. It
gives us transparency over a variety of thread implementations and
platforms, and provides for graceful failure in case there is no thread
implementation at all.
The issue is that glib's thead pool won't work for an
Paul,
On the default where if the prefix is not specified it needs to be
changed to look like this. Didn't feel like doing a diff. This is because
we need pople to be able to specify a prefix of your fix doesn't allow
for this I have changed my config.c function to look like such:
void
Paul,
Glad it will work in the current CVS head, thanks for putting it in
there... As you said and I had planed it is just a stop gap till we have
header caching. But the btree works well it needs some work for other
clients than sqmail as other clients use diffrent sort parameters and I
didn't
to the codebase for 2.1 asap. This requires the
addition of table_prefix to the dbmail.conf
Thanks,
Leif Jackson
dbmail-2.1cvs091404-dynamicprefix.diff
Description: Binary data
On Tue, 14 Sep 2004 15:59:51 -0400 (EDT), Leif Jackson
[EMAIL PROTECTED] wrote:
Hi All,
Paul or whomever is the one to review and add patches these days, I
have
made a patch for current CVS head (2.1) that allows for dynamic table
prefixes as I am tired of patching the source each time I
Guys,
I disagree with the statement that LTDL makes it impossible to debug. I
fact it is quite easy to debug dynamic libraries with gdb, as well as
valgrind. What about this code can you not figure out. I haven't taken
the time to look at it yet since I only use MySQL but I understand the
need
Yes, it makes the code more complicated, but I think it's only because
we
still haven't fully adapted our thinking to our model. It occurred to me
today
that if we were MIME parsing at delivery time, we could also store the
mime
structure of a message in the database. An IMAP BODY.PEEK for
I am betting that my mysql auto-reconnect change that went into 2.x devel
tree didn't get into 1.2.x Ilja? maybe we can get out a 1.2.7b anyway in
the db_connect area you have to add a line like...
/* auto re-connect */
conn.reconnect = 1;
this is right after the mysql_init in
I just posed a message to the list for John's issue you are having the
same one. if Ilja is around maybe we can get out a 1.2.7b with this minor
fix in it. See the other message I sent to the list.
-leif
Hello,
I use dbmail for a couple of days and have been monitoring the maillog
very
I have attached the same version 2 patch for the sort command I sent the
list a few days ago, but this is against current cvs so it should apply
cleanly. Might be nice to sneek it into 2.0 for the people who use dbmail
for a webmail backend, but that is up to you guys :) It is usable I just
don't
beta patch to 2.0rc4.
Thanks,
Leif Jackson.
Micah,
I'm running the very same configuration here - SM 1.4.2 and dbmail 1.2.5.
Have you experienced perfomance problems?
For benchmarking purposes, i created 30 folders and filled each one of
them with 100 to 1500 messages (a total of 16000 messages
to migrate my dbmail 1.2.5 to the 2.0 RC4 if I
can apply your speedup patch (server side search) - where can i get it?
Is it bundled in the 2.0 RC4 source?
Thanks in advance,
Augusto Bott
Leif Jackson wrote:
Augusto Micah,
This issues has to do with the fact that until dbmail 2.1 (where
All,
Here is version two of the imap SORT command i got to thinking about
that first one using msort on the list, and that was just stupid. So I
made up a quick binary tree and all is much happier now. This still
needs a lot of work and still needs the header caching but at least now
it does
SORT (ARRIVAL) ISO-8859-1 ALL
p.p.s. see
http://www.ietf.org/internet-drafts/draft-ietf-imapext-sort-14.txt
Yay!!! You are a life saver Leif.
SB
Leif Jackson [EMAIL PROTECTED] wrote:
This observation was made before :)
SORT and THREAD will have to be implemented for version 2.1
All,
I am with arron on this one, I have been looking to implement a sqlrelay
interface for a while now, the benifit of it is that it gives us an easy
migration to other RDBM's such as oracle, which is relatively hard to
code natively for. As Aaron pointed out it also solves some if not all of
Ed,
I read your spec, seems like this will work fine but make sure you add
and implement XOVER section 2.8 from rfc2980 this is the _most_ important
command in an nntp server as far as current clients today. Be carefully
to take into account that article numbers are not uniq on the servers out
Ed, For good reason,
I used to work for one of the worlds largest NNTP providers. And believe
me from experience we do not want to get into NNTP stuff. The reason none
of the currently available NNTP server code uses an RDBM is that the file
IO is so heavy and requires so much IO that it takes
This observation was made before :)
SORT and THREAD will have to be implemented for version 2.1
Or as a patch to 2.0 if I can get it done shortly, I really need it for
our application which is a web based mail desktop system. So my client is
asking for this almost every day. I hope to have
linked (uses shared libs), not stripped
[EMAIL PROTECTED]:~/dbmail-2.0rc3 ./testmd5
;lkajsdf;kljasdf
asdf
0a8c8e4fd15849ec600d9aac8e53bf93
[EMAIL PROTECTED]:~/dbmail-2.0rc3
Leif Jackson [EMAIL PROTECTED] said:
Humm well it isn't our code that's not 64bit clean, as I emailed
(SYSV),
not
stripped
testmd5: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
dynamically linked (uses shared libs), not stripped
[EMAIL PROTECTED]:~/dbmail-2.0rc3 ./testmd5
;lkajsdf;kljasdf
asdf
0a8c8e4fd15849ec600d9aac8e53bf93
[EMAIL PROTECTED]:~/dbmail-2.0rc3
Leif Jackson
an endianness issue, too... I've got a Sparc at
school,
I'll give it a whirl on that machine, too. And my Alpha is out of reach
again,
so I can't double check the results on it :-(
Aaron
Leif Jackson [EMAIL PROTECTED] said:
odd, just checked with your same chars pasted into the test app
Aaron,
I would try valgrind, they should have it installed. It does well on all
kinds of bounds checking, as well as memory and cache checks.
-Leif
Hey,
So I whipped up a little wrapper program around read_header() and
makemd5()
that crashes on the Opteron server at SourceForge, but
/sodabrew/dbmail-2.0rc3/testmd5
;lkajsdf;kljasdf
asdf
b3dd95bad20e039aa898a75cdab51a4d
Program received signal SIGSEGV, Segmentation fault.
0x in ?? ()
Leif Jackson [EMAIL PROTECTED] said:
Aaron,
I would try valgrind, they should have it installed. It does well on
all
(SYSV), not
stripped
testmd5: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV),
dynamically
linked (uses shared libs), not stripped
Leif Jackson [EMAIL PROTECTED] said:
Valgrind emulates a Pentium instruction set, so it's not useful on an
x86-64
processor. They have gdb 5.3
delivered twice.
Ilja
Leif Jackson wrote:
Ilja Aaron,
I am looking for this patch. I cannot access it from sourceforge? Or
do
I
have to checkout from cvs differently than the default dbmail root? I
would be glad to look at this Also Aaron, i was working on your
sieve
code
Ilja,
Correction I wiped my cvs out and re-checked it out and the changes are
there. I was expecting the update command to merge the changes is this
not the case with the dbmail cvs tree?
On the list.c stuff I see this with the dmalloc lib we talked about a
while back, do you knotice a state
Nevermind on the not exiting that was a issue I was causing with the
debuging options. thx.
-leif
Ilja,
Correction I wiped my cvs out and re-checked it out and the changes are
there. I was expecting the update command to merge the changes is this
not the case with the dbmail cvs tree?
the documentation :-\
You mind sendming me a tarball offline? Or sending me a place I can get
your current working tree? Thx. I understand about getting busy
Aaron
Leif Jackson [EMAIL PROTECTED] said:
Ilja Aaron,
I am looking for this patch. I cannot access it from sourceforge? Or do
I
Roel,
I noticed that some of the original patch I sent you a while ago for
memory leaks missed these few db_free_result's I have attached the patch
for current cvs. Thanks. I am still trying to track down the issue with
the list.c node_add it still seems that there is an instance in the
codebase
86 matches
Mail list logo
