Re: [Dbmail] Sieve rules break address expansion routing
Paul J Stevens wrote: > Peter Rabbitson wrote: >> I am using latest dbmail 2.2.15. I just realized that *any* sieve >> rule present for an account (even if the rule is disabled), implicitly >> disabels any support for addr+folder@ type of address expansion routing. > > I assume you mean that having an active sieve-script will disable > subaddress delivery. That is correct. The default-mailbox, derived from > the subaddress does not propagate into the sieve layer. I am not sure what you mean here (especially since you are attaching a patch further down). Basically my question is - what is the precedence here? Can Sieve *and* subaddressing coexist? Or it is solely one or the other? >> The way things are now, if a user wants to take advantage of sieve, >> they will have to populate the sieve rulebook with *all* + extensions >> used so far, so the mail delivery will continue as before. This is... >> suboptimal :) > > I've just pushed a fix for this. > > > Try the patch attached plz. My test-setup got screwed up, and I am not brave enough to test this on prod. I will get around to this in a couple of weeks. Thanks! ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
[Dbmail] Sieve rules break address expansion routing
I am using latest dbmail 2.2.15. I just realized that *any* sieve rule present for an account (even if the rule is disabled), implicitly disabels any support for addr+folder@ type of address expansion routing. The way things are now, if a user wants to take advantage of sieve, they will have to populate the sieve rulebook with *all* + extensions used so far, so the mail delivery will continue as before. This is... suboptimal :) Any ideas? Thanks! Peter ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
[Dbmail] SQL query for DBMail 2.2.x status reports
I needed an efficient way to present an overview of the current state of
the database. I came up with these 2 queries, figured they may be useful
for someone else too. Performance is rather good, it takes me 1.3 secs
for the outer query on ~1mil phys messages with ~40 users. Formatting in
the select list is mysql-specific and can of course be changed/removed:
The first query is where the actual aggregation takes place (and the one
that takes the 1.3 secs):
SELECT
b.owner_idnr AS `user_idnr`,
COUNT(m.message_idnr) AS `msg_count`,
MAX(m.message_idnr) AS `last_msg_id`,
COUNT(m2.message_idnr) AS `unread_count`,
MIN(m2.message_idnr) AS `first_unread_id`
FROM dbmail_mailboxes b
JOIN dbmail_messages m
ON ( m.mailbox_idnr = b.mailbox_idnr AND m.status <= 1 )
LEFT JOIN dbmail_messages m2
ON ( m.message_idnr = m2.message_idnr AND m2.seen_flag = 0 )
GROUP BY b.owner_idnr
This is the reporting query which relies on the aggregator
(for readability assume that the aggregator is saved as a view STATS):
SELECT
RIGHT( a.alias, LENGTH(a.alias) - LOCATE('@',a.alias) ) AS `Domain`,
a.alias AS `Address`,
IFNULL( u.userid, a.deliver_to ) AS `Deliver To
(username or forward)`,
IFNULL( DATE(u.last_login),'N/A' )AS `User Last
Login`,
LPAD( IFNULL( FORMAT(s.msg_count,0),'N/A' ), 14, ' ') AS `Total Messages`,
IFNULL( DATE(p_last.internal_date),'N/A' )AS `Most Recent
Message`,
LPAD( IFNULL( FORMAT(s.unread_count,0), 'N/A' ), 14, ' ') AS `Unread
Messages`,
IFNULL( DATE(p_unread.internal_date), 'N/A' ) AS `Oldest Unread
Message`
FROM dbmail_aliases a
LEFT JOIN dbmail_users u
ON a.deliver_to = u.user_idnr
LEFT JOIN STATS s
ON u.user_idnr = s.user_idnr
LEFT JOIN dbmail_messages m_last
ON s.last_msg_id = m_last.message_idnr
LEFT JOIN dbmail_physmessage p_last
ON p_last.id = m_last.physmessage_id
LEFT JOIN dbmail_messages m_unread
ON s.first_unread_id = m_unread.message_idnr
LEFT JOIN dbmail_physmessage p_unread
ON p_unread.id = m_unread.physmessage_id
ORDER BY Domain, ISNULL(u.user_idnr),a.alias
___
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: [Dbmail] Problems working with a non-local (tcp-reachable) mysql database
Peter Rabbitson wrote: > I filed a bug about a week ago [1], and I did some more tests and am still > mystified as to what could be causing this. I would like to know if someone > else has seen such problems before, it might be something as simple as > a misconfigured mysql on my part. All details and log excerpts are in the > bugreport. > > [1] http://www.dbmail.org/mantis/view.php?id=824 Please close this bug. The problem turned out to be a BIZARRE issue with the network card driver: http://marc.info/?l=linux-netdev&m=126321290613982 ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: [Dbmail] Problems working with a non-local (tcp-reachable) mysql database
Paul J Stevens wrote: > I don't have a clue. I use non-local tcp connections from dbmail-2.2 to > mysql servers (32bit) all the time without any problems like this. > > Only thing I havent tested is connecting from a 32bit mysql client to a > 64 bit mysql server > > Upgrading to 5.1 seems to make the issue disappear entirely. Now all I need to know if mysql 5.1 is still a disaster a year after it was released. If it performs acceptably - I'll just close the bug and move on. ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: [Dbmail] Problems working with a non-local (tcp-reachable) mysql database
Paul J Stevens wrote: > Jorge, Peter, > > I don't believe there's a problem with the query. > > Peter, try connection to the remove database using the mysql client > (mysql -h somehost...) and run the query mentioned below. That way you > will evade any dbmail issues. > > My guess is this is a libmysqlclient versioning issue. > Nope, all queries are running fine standalone. In fact as you can see in the full dbmail-util log in the bugreport, the same query runs *several times* before actually failing. Any other ideas on how to pin this down? ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
[Dbmail] Problems working with a non-local (tcp-reachable) mysql database
I filed a bug about a week ago [1], and I did some more tests and am still mystified as to what could be causing this. I would like to know if someone else has seen such problems before, it might be something as simple as a misconfigured mysql on my part. All details and log excerpts are in the bugreport. [1] http://www.dbmail.org/mantis/view.php?id=824 ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: [Dbmail] :/
Piotr Wadas wrote: > > Well, after some tests for a few months on selected accounts with dbmail > 2.3.6 The 2.3 series is an experimental branch, akin to the 2.5.x series of the linux kernel back in the day. Most subscribers of this list are quite happy with the production-ready 2.2.x series. Be aware that 2.2.12 introduced a regression, which apparently has been fixed and 2.2.13 is to be officially released today. Cheers ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
[Dbmail] LDAP workflow
Hi, I am toying with the idea to migrate all my services (login, web, samba, dbmail) to a centralized ldap setup. While I read a lot about ldap design and general operation principles, I can't quite wrap my head around it. So here goes: Is there some recommended/best practice way of seting up ldap schemas for dbmail consumption? Also is there some sort of GUI tool for ldap administration? While I am comfortable with the CLI, some of the people who will need to eventually administer users are not, and anything "flashy" and easy to use is a plus. Thanks! ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: [Dbmail] 2ip Fiber Web Hosting at $199
7CREWZ wrote: > > > I'd understand if this was a spam-and-run drive-by. But this is an actual f*** user who got help here before[1]. Just sick. [1]: http://www.mail-archive.com/search?l=dbmail%40dbmail.org&q=7CREWZ ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: [Dbmail] buggy postfix sql recipients query against dbmail_aliases
Aleksander Kamenik wrote: > Thanks for the fast reply. > > Peter Rabbitson wrote: >> Just my 2c - I have never seen this error, as I do not blindly accept mail >> for my subdomains (unlike the default): >> >> r...@arzamas:~# postconf -d | grep parent_domain_matches_subdomains >> parent_domain_matches_subdomains = >> debug_peer_list,fast_flush_domains,mynetworks,permit_mx_backup_networks,qmqpd_authorized_clients,smtpd_access_maps >> r...@arzamas:~# postconf -n | grep parent_domain_matches_subdomains >> parent_domain_matches_subdomains = >> >> With this settings the user gets a 554 Relay Denied unless the domain >> is explicitly listed in _domains= > > Ok, I set the parent_domain_matches_subdomains to blank. > > # postconf -n | grep parent_domain_matches_subdomains > parent_domain_matches_subdomains = > # netcat localhost 25 > 220 mail.krediidiinfo.ee ESMTP Postfix > HELO mail.krediidiinfo.ee > 250 mail.krediidiinfo.ee > MAIL FROM: > 250 Ok > RCPT TO: > 250 Ok > QUIT > 221 Bye > > Still accepted the address. All *_domains settings are the default, that > is empty. The domains I receive mail for are defined in mydestination. > > Yes, I reloaded and restarted postfix, the settings are definitely in > effect. > > Why does postfix still accept the address? I looked at my postfix > settings and couldn't find anything useful. > > PS: In my previous mail I forgot to mention I'm using mysql: > local_recipient_maps = proxy:mysql:/etc/postfix/sql-recipients.cf > Can you show your postconf -n (edit whatever information you want to hide) ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: [Dbmail] buggy postfix sql recipients query against dbmail_aliases
Aleksander Kamenik wrote: > Hi, I have the following standard settings for postfix doing the RCPT > TO: check via sql against the dbmail_aliases table. > > sql-recipients.cf: > > user = xxx > password = xxx > hosts = 127.0.0.1 > dbname = xxx > table = dbmail_aliases > select_field = alias > where_field = alias > > > Here's a working and buggy example: > > # netcat localhost 25 > 220 mail.krediidiinfo.ee ESMTP Postfix > HELO mail.krediidiinfo.ee > 250 mail.krediidiinfo.ee > MAIL FROM: > 250 Ok > RCPT TO: > 550 : Recipient address rejected: User > unknown in local recipient table > RCPT TO: > 250 Ok > QUIT > 221 Bye > > The second response to RCPT TO: should have been a 550 too, not 250. The > dbmail_aliases table contains only proper "u...@domain" entries, no > wildcard stuff like simply a "@domain" or similar. > > So a there's a problem, when a subhost is specified in the address. > Another issue is with when no user is specified. Here's an example: > > # netcat localhost 25 > 220 mail.krediidiinfo.ee ESMTP Postfix > HELO mail.krediidiinfo.ee > 250 mail.krediidiinfo.ee > MAIL FROM: > 250 Ok > RCPT TO:<@krediidiinfo.ee> > 250 Ok > QUIT > 221 Bye > > These wrong addresses are often expanded from user typos like > "@u...@domain" for example or whatever combination. Spamming postfix > with these invalid addresses would result in a DOS. > > This is postfix 2.2.9 and dbmail 2.2.11 on SLES10SP2. > > > Could a postfix/sql knowledgeable person please refine the > sql-recipients file, so invalid addresses would be denied immediately? I > have no idea how to do this. Just my 2c - I have never seen this error, as I do not blindly accept mail for my subdomains (unlike the default): r...@arzamas:~# postconf -d | grep parent_domain_matches_subdomains parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks,permit_mx_backup_networks,qmqpd_authorized_clients,smtpd_access_maps r...@arzamas:~# postconf -n | grep parent_domain_matches_subdomains parent_domain_matches_subdomains = With this settings the user gets a 554 Relay Denied unless the domain is explicitly listed in _domains= ___ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
Re: [Dbmail] [Slightly OffTopic] Backing up dbmail ( error 2013 )
Marc Dirix wrote: > This is not the same. > >> 1) flush all pending writes and lock the database as read-only > > Postgres does not flush pending writes, and also does not set the > database read-only. > Is this considered a good thing...? I mean I am about to save away the actual database binaries. >> 2) force a binlog rotation so you know exactly where the snapshot was taken >> 3) make a LV snapshot (takes care of all sync()s at fs level) >> 4) unlock everything (the lock persists while the snapshto is created, >> which lasts about half a second) >> >> Then I go ahead and slowly backup/compress/whatever I want the database >> files from the snapshot volume >> >> >> When shit hits the fan: >> === >> >> 1) Stop the database >> 2) Unpack the mysql data directory saved from the latest snapshot (since >> the snapshot was made within a read-lock, the database is guaranteed to >> be consistent) > > With PITR you can recover to *any* point in time. Not just your snapshot > time. It is completely different. PITR is a query log which starts from > your snapshot onward. So sunday snapshot day (without locking or > read-only) then continueing with query logs up to crash day. Again - how do you take a consistent snapshot on Sunday without ensuring a lock? > However, I would not even think about using it with dbmail. Because it > slows down your database. For every INSERT and UPDATE query the data has > to be saved twice. Once in the database and once in the log. > Not sure about PG - the penalty in MySQL is in the low single digits. It's not like fsync() is called on every logwrite. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] [Slightly OffTopic] Backing up dbmail ( error 2013 )
Niblett, David A wrote: > Dan, > > I know you are using mysql, but if ever you were to move to > Postgres, I'm using the Point in Time Recovery (PITR) setup. > This has been great for making backups when your database > is large (mine is around 52G). > > Restore could take you longer, but my opinion is, if I'm to > the point that I have to restore it's better to have the > data and take longer to restore, than to not or cause > performance hits every day to backup the data. > > Basically the PITR saves the transaction logs of the database. > So I just copy the database every Sunday early morning, then > just archive out the transaction logs to another drive as they > fill up. > > If I should have to recover, I just copy back in the last Sunday > copy of the DB and tell Postgres to playback the transaction log > files. The really nice thing is if I think it was a DB corruption > problem, I can say restore everything to 5m before the crash. > > Just another thought for you. > I do absolutely the same with my 60GB mysql database: in mysql.cnf: = log_bin = /space/system/mysql/binlog/mysql-bin * log_bin_index = /space/system/mysql/binlog/mysql-bin.index * server_id = 1 expire_logs_days= 20 max_binlog_size = 500M * You can use the defaults, I am just copying my complete example here in my backup script: === /usr/bin/mysql --defaults-extra-file=/etc/mysql/debian.cnf \ --execute "FLUSH TABLES WITH READ LOCK, LOGS; SYSTEM /sbin/lvcreate --snapshot --size $S_LV_SIZE --chunksize 512K --name $S_LV $VG/$LV &>/dev/null ; UNLOCK TABLES;" where the variables are just fill-ins as per the lvcreate manpage. WHat this does is: 1) flush all pending writes and lock the database as read-only 2) force a binlog rotation so you know exactly where the snapshot was taken 3) make a LV snapshot (takes care of all sync()s at fs level) 4) unlock everything (the lock persists while the snapshto is created, which lasts about half a second) Then I go ahead and slowly backup/compress/whatever I want the database files from the snapshot volume When shit hits the fan: === 1) Stop the database 2) Unpack the mysql data directory saved from the latest snapshot (since the snapshot was made within a read-lock, the database is guaranteed to be consistent) 3) Start the database, but make sure no connections are accepted (as the data is healthy but old) 4) gather all binlog files since the snapshot was taken (correlate the backup script run time with the snapshot timestamos) 5) replay them using the wonderful mysqlbinlog utility 6) go back to sleep ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] mysql + slow queries - END OF THE STORY
Giulio Ferro wrote: > Ok, the problem was mysql5.1, as many of you suggested. > I've dumped the dbmail db, downgraded to 5.0.67 and imported > back again. > The explain statement now yields the same result both with the > "order by" and without. The system seems to work ok, I hope I'll > get no more slow queries in the next days... > Please file a bug report with MySQL and let us know the ticket number, so interested individuals can track its progress. It would be an unpleasant surprise to upgrade into this mess :) Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] mysql + slow queries
Peter Rabbitson wrote: > Giulio Ferro wrote: >> Paul J Stevens wrote: >>> It's not the number of rows in messageblks that is your problem, but the >>> rediculous rowscan in the explain output. Could you run that 'explain >>> select' again? >>> >> Now, this is interesting: >> --- >> >> explain select messageblk, is_header from dbmail_messageblks where >> physmessage_id=364483 order by messageblk_idnr; >> ++-++---+-+-+-+--+-+-+ >> >> | id | select_type | table | type | >> possible_keys | key | key_len >> | ref | rows| Extra | >> ++-++---+-+-+-+--+-+-+ >> >> | 1 | SIMPLE | dbmail_messageblks | index | >> physmessage_id_index,physmessage_id_is_header_index | PRIMARY | 8 >> | NULL | 1605498 | Using where | >> ++-++---+-+-+-+--+-+-+ >> >> 1 row in set (0.00 sec) >> --- >> >> >> >> but if I now run: >> --- >> >> explain select messageblk, is_header from dbmail_messageblks where >> physmessage_id=364483;++-++--+-++-+---+--+---+ >> >> | id | select_type | table | type | >> possible_keys | >> key| key_len | ref | rows | Extra | >> ++-++--+-++-+---+--+---+ >> >> | 1 | SIMPLE | dbmail_messageblks | ref | >> physmessage_id_index,physmessage_id_is_header_index | >> physmessage_id_is_header_index | 8 | const |1 | | >> ++-++--+-++-+---+--+---+ >> >> 1 row in set (0.00 sec) >> --- >> >> >> May the problem be in how the db server treats the "order by" clause? >> >> I'm currently running mysql 5.1.28. I may try to downgrade to 5.0.x, but >> I'm >> loath to do that. Maybe I should contact the mysql guys to raise a bug >> or something? > > It might be 5.1. Here are my 2 cents: > > mysql> select count(*) from dbmail_messageblks where messageblk_idnr = 500; > +--+ > | count(*) | > +--+ > |1 | > +--+ > 1 row in set (0.42 sec) > > mysql> explain SELECT messageblk, is_header FROM dbmail_messageblks > WHERE physmessage_id = 500; > ++-++--+-+--+-+---+--+---+ > | id | select_type | table | type | possible_keys > | key | key_len | ref | > rows | Extra | > ++-++--+-+--+-+---+--+---+ > | 1 | SIMPLE | dbmail_messageblks | ref | > physmessage_id_index,physmessage_id_is_header_index | > physmessage_id_index | 8 | const |2 | | > ++-++--+-+--+-+---+--+---+ > 1 row in set (0.73 sec) > > mysql> explain SELECT messageblk, is_header FROM dbmail_messageblks > WHERE physmessage_id = 500 ORDER BY messageblk_idnr; > ++-++--+-+--+-+---+--+-+ > | id | select_type | table
Re: [Dbmail] mysql + slow queries
Giulio Ferro wrote: > Paul J Stevens wrote: >> It's not the number of rows in messageblks that is your problem, but the >> rediculous rowscan in the explain output. Could you run that 'explain >> select' again? >> > > Now, this is interesting: > --- > > explain select messageblk, is_header from dbmail_messageblks where > physmessage_id=364483 order by messageblk_idnr; > ++-++---+-+-+-+--+-+-+ > > | id | select_type | table | type | > possible_keys | key | key_len > | ref | rows| Extra | > ++-++---+-+-+-+--+-+-+ > > | 1 | SIMPLE | dbmail_messageblks | index | > physmessage_id_index,physmessage_id_is_header_index | PRIMARY | 8 > | NULL | 1605498 | Using where | > ++-++---+-+-+-+--+-+-+ > > 1 row in set (0.00 sec) > --- > > > > but if I now run: > --- > > explain select messageblk, is_header from dbmail_messageblks where > physmessage_id=364483;++-++--+-++-+---+--+---+ > > | id | select_type | table | type | > possible_keys | > key| key_len | ref | rows | Extra | > ++-++--+-++-+---+--+---+ > > | 1 | SIMPLE | dbmail_messageblks | ref | > physmessage_id_index,physmessage_id_is_header_index | > physmessage_id_is_header_index | 8 | const |1 | | > ++-++--+-++-+---+--+---+ > > 1 row in set (0.00 sec) > --- > > > May the problem be in how the db server treats the "order by" clause? > > I'm currently running mysql 5.1.28. I may try to downgrade to 5.0.x, but > I'm > loath to do that. Maybe I should contact the mysql guys to raise a bug > or something? It might be 5.1. Here are my 2 cents: mysql> select count(*) from dbmail_messageblks where messageblk_idnr = 500; +--+ | count(*) | +--+ |1 | +--+ 1 row in set (0.42 sec) mysql> explain SELECT messageblk, is_header FROM dbmail_messageblks WHERE physmessage_id = 500; ++-++--+-+--+-+---+--+---+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | ++-++--+-+--+-+---+--+---+ | 1 | SIMPLE | dbmail_messageblks | ref | physmessage_id_index,physmessage_id_is_header_index | physmessage_id_index | 8 | const |2 | | ++-++--+-+--+-+---+--+---+ 1 row in set (0.73 sec) mysql> explain SELECT messageblk, is_header FROM dbmail_messageblks WHERE physmessage_id = 500 ORDER BY messageblk_idnr; ++-++--+-+--+-+---+--+-+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | ++-++--+-+--+-+---+--+-+ | 1 | SIMPLE | dbmail_messageblks | ref | physmessage_id_index,physmessage_id_is_header_index | physmessage_id_index | 8 | const |2 | Using where | ++-++--+-+--+-+---+--+-+ 1 row i
Re: [Dbmail] PostgreSQL complains about string missuse on INSERTS related to quotations marks
Josh Berkus wrote: > > However, the reason why we added the warning is that we figured out in > 2006 that \\ escapes are a SQL injection vulnerability (this goes for > MySQL as well). It's possible by version 8.5 (2010) PostgreSQL will > stop supporting them. > Interesting... Can you cite some CVE reference or what have you? Google does not seem to be my friend today. Thanks! ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Imapsync: messages skipped
Giulio Ferro wrote: > Peter Rabbitson wrote: >> Giulio Ferro wrote: >> >>> I'm in the middle of a mailbox migration from vpopmail to dbmail. >>> >>> Everything seems to work reasonably well, but I've noticed that those >>> boxes which have a lot of folders and subfolders aren't integrally >>> copied. >>> >>> At first not even all folders were copied: I had to run imapsync with >>> --justfolders to create them. Then even after executing the program >>> repeatedly there where messages not copied. >>> >>> What's interesting is that the final report showed no error, but only >>> messages and bytes skipped > 0. >>> >>> Why are messages and bytes skipped? Where should I look into? >>> >> >> Please provide your entire command line invoking imapsync. >> > > Sure: > imapsync --host1 --host2 --user1 --user2 > --password1 --password2 --authmech1 PLAIN > --authmech2 LOGIN --regextrans2 s/INBOX/oldmail/g > > imapsync version: 1.264 Basically you will always have _some_ mistamtched headers etc. The first step is to exclude the size comparison. And then exclude headers you are not interested in too. I used the following with 1.219 2 years ago hope it gives you a starting point: ~/imapsync-1.219/imapsync \ --host1 localhost --user1 migrate --password1 abc123 --authmech1 PLAIN --port1 20143 \ --host2 localhost --user2 $user --password2 $pass --authmech2 LOGIN \ --exclude '^\.|^mbox' --delete2 --expunge2 \ --syncinternaldates --subscribe \ --skipheader 'X-DBMail-PhysMessage-ID' --skipsize ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Imapsync: messages skipped
Giulio Ferro wrote: > I'm in the middle of a mailbox migration from vpopmail to dbmail. > > Everything seems to work reasonably well, but I've noticed that those > boxes which have a lot of folders and subfolders aren't integrally copied. > > At first not even all folders were copied: I had to run imapsync with > --justfolders to create them. Then even after executing the program > repeatedly there where messages not copied. > > What's interesting is that the final report showed no error, but only > messages and bytes skipped > 0. > > Why are messages and bytes skipped? Where should I look into? Please provide your entire command line invoking imapsync. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] db_load_module error
Prashanth wrote: > Hi paul, > > On Wed, Oct 8, 2008 at 3:55 PM, Paul J Stevens <[EMAIL PROTECTED]> wrote: >> Most likely you specified a database driver in dbmail.conf but forgot to >> run ./configure --with-DRIVER, or vice versa. >> >> the driver in dbmail.conf must be included in the configure run. >> >> So if you want to use postgres: >> >> ./configure --with-pgsql >> >> and in dbmail.conf specify: >> driver = pgsql >> >> same for mysql and sqlite >> > > > I have recheck and reinstalled but same issue. > > dbmail/imap4d[11930]: FATAL:[db] dbmodule.c,db_load_driver(+80): > could not load db module - turn up debug level for details > dbmail/lmtpd[11931]: FATAL:[db] dbmodule.c,db_load_driver(+80): could > not load db module - turn up debug level for details > What is the actual value of driver in your dbmail.conf? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] future 2.2 release
Aaron Stone wrote: > > On Oct 6, 2008, at 3:58 PM, Peter Rabbitson wrote: > >> Aaron Stone wrote: >>> >>> On Oct 6, 2008, at 3:29 PM, Peter Rabbitson wrote: >>> >>>> Aaron Stone wrote: >>>>> >>>>> On Oct 6, 2008, at 11:40 AM, Peter Rabbitson wrote: >>>>> >>>>>> Aaron Stone wrote: >>>>>>> >>>>>>> On Oct 6, 2008, at 4:10 AM, Peter Rabbitson wrote: >>>>>>> >>>>>>>> >>>>>>>> If something matches we examine every forward destination. If it >>>>>>>> is an >>>>>>>> email address of the form: >>>>>>>> *[EMAIL PROTECTED] - we just forward >>>>>>>> *[EMAIL PROTECTED] AND there was a YYY above we send to [EMAIL >>>>>>>> PROTECTED] >>>>>>>> (using >>>>>>>> the >>>>>>>> same delimiter as found in the forward in case we recognize >>>>>>>> multiple >>>>>>>> delimiters) >>>>>>> >>>>>>> I'm not sure I get what you mean here... >>>>>>> >>>>>> >>>>>> Suppose we do: >>>>>> >>>>>> dbmail-users -x [EMAIL PROTECTED] -t [EMAIL PROTECTED] >>>>>> >>>>>> Then it would be neat (and logical) for any email to >>>>>> >>>>>> [EMAIL PROTECTED] to be forwarded to [EMAIL PROTECTED] >>>>>> >>>>>> This is just (to me) a logical extension of what is already in place. >>>>>> The original bug complains only about the lack of XXX+YYY@ support. >>>>> >>>>> >>>>> Oh, that's pretty neat. Currently the forward should work but with the >>>>> +mailbox removed. Does that work in practice for you? >>>>> >>>> >>>> I am not sure what your question is here. Can you clarify? >>> >>> Mail addressed to [EMAIL PROTECTED] where a forward exists for [EMAIL >>> PROTECTED] -> >>> [EMAIL PROTECTED] will work, but the original message will go into WWW's >>> Inbox. At >>> least, that's what I think happens; I haven't tried this in a while. >>> >> >> Yes this is what happens. And what I was proposing is expand on this >> schema to allow the sub-addressing to be carried over to the destination >> (unless the destination doesn't _already_ specify a sub-address on its >> own, in which case things have to be left intact) > > I'm pretty sure this is significantly non-trivial, but I'll take a look > and see! > Oh... I thought it will be an additional line or two, as you already have extracted all the relevant information while processing the address against the alias/forward table... If it is indeed a non-trivial feature request, please disregard it. I am happy enough that XXX+YYY@ will get fixed :) Thank you for your time Cheers! ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] future 2.2 release
Aaron Stone wrote: > > On Oct 6, 2008, at 3:29 PM, Peter Rabbitson wrote: > >> Aaron Stone wrote: >>> >>> On Oct 6, 2008, at 11:40 AM, Peter Rabbitson wrote: >>> >>>> Aaron Stone wrote: >>>>> >>>>> On Oct 6, 2008, at 4:10 AM, Peter Rabbitson wrote: >>>>> >>>>>> >>>>>> If something matches we examine every forward destination. If it >>>>>> is an >>>>>> email address of the form: >>>>>> *[EMAIL PROTECTED] - we just forward >>>>>> *[EMAIL PROTECTED] AND there was a YYY above we send to [EMAIL >>>>>> PROTECTED] (using >>>>>> the >>>>>> same delimiter as found in the forward in case we recognize multiple >>>>>> delimiters) >>>>> >>>>> I'm not sure I get what you mean here... >>>>> >>>> >>>> Suppose we do: >>>> >>>> dbmail-users -x [EMAIL PROTECTED] -t [EMAIL PROTECTED] >>>> >>>> Then it would be neat (and logical) for any email to >>>> >>>> [EMAIL PROTECTED] to be forwarded to [EMAIL PROTECTED] >>>> >>>> This is just (to me) a logical extension of what is already in place. >>>> The original bug complains only about the lack of XXX+YYY@ support. >>> >>> >>> Oh, that's pretty neat. Currently the forward should work but with the >>> +mailbox removed. Does that work in practice for you? >>> >> >> I am not sure what your question is here. Can you clarify? > > Mail addressed to [EMAIL PROTECTED] where a forward exists for [EMAIL > PROTECTED] -> > [EMAIL PROTECTED] will work, but the original message will go into WWW's > Inbox. At > least, that's what I think happens; I haven't tried this in a while. > Yes this is what happens. And what I was proposing is expand on this schema to allow the sub-addressing to be carried over to the destination (unless the destination doesn't _already_ specify a sub-address on its own, in which case things have to be left intact) Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] future 2.2 release
Aaron Stone wrote: > > On Oct 6, 2008, at 11:40 AM, Peter Rabbitson wrote: > >> Aaron Stone wrote: >>> >>> On Oct 6, 2008, at 4:10 AM, Peter Rabbitson wrote: >>> >>>> >>>> If something matches we examine every forward destination. If it is an >>>> email address of the form: >>>> *[EMAIL PROTECTED] - we just forward >>>> *[EMAIL PROTECTED] AND there was a YYY above we send to [EMAIL >>>> PROTECTED] (using >>>> the >>>> same delimiter as found in the forward in case we recognize multiple >>>> delimiters) >>> >>> I'm not sure I get what you mean here... >>> >> >> Suppose we do: >> >> dbmail-users -x [EMAIL PROTECTED] -t [EMAIL PROTECTED] >> >> Then it would be neat (and logical) for any email to >> >> [EMAIL PROTECTED] to be forwarded to [EMAIL PROTECTED] >> >> This is just (to me) a logical extension of what is already in place. >> The original bug complains only about the lack of XXX+YYY@ support. > > > Oh, that's pretty neat. Currently the forward should work but with the > +mailbox removed. Does that work in practice for you? > I am not sure what your question is here. Can you clarify? Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] future 2.2 release
Aaron Stone wrote: > > On Oct 6, 2008, at 4:10 AM, Peter Rabbitson wrote: > >> >> If something matches we examine every forward destination. If it is an >> email address of the form: >> *[EMAIL PROTECTED] - we just forward >> *[EMAIL PROTECTED] AND there was a YYY above we send to [EMAIL >> PROTECTED] (using the >> same delimiter as found in the forward in case we recognize multiple >> delimiters) > > I'm not sure I get what you mean here... > Suppose we do: dbmail-users -x [EMAIL PROTECTED] -t [EMAIL PROTECTED] Then it would be neat (and logical) for any email to [EMAIL PROTECTED] to be forwarded to [EMAIL PROTECTED] This is just (to me) a logical extension of what is already in place. The original bug complains only about the lack of XXX+YYY@ support. Cheers ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] future 2.2 release
Paul J Stevens wrote: > Casper, > > No worries. I'll do a rc2 later today or early tomorrow. Debian packages > will follow immediately after that. > > If this rc2 release doesn't introduce any regressions - I depend on you > guys to verify this - I'll release 2.2.11 one week after the rc. > As promised I filed bug 731, please consider for inclusion before releasing. Also if Aaron is reading this, I would really love and appreciate to have bug 672 fixed. Here is how I envision it should work: Assuming that the delimiter of the day is '+' (can be set to whatever), we receive a mail addressed to: [EMAIL PROTECTED] Then: The forward table is checked in succession for these forwards (just like the aliases): * [EMAIL PROTECTED] * [EMAIL PROTECTED] * XXX+YYY@ * XXX@ * @DDD If something matches we examine every forward destination. If it is an email address of the form: * [EMAIL PROTECTED] - we just forward * [EMAIL PROTECTED] AND there was a YYY above we send to [EMAIL PROTECTED] (using the same delimiter as found in the forward in case we recognize multiple delimiters) 90% of this functionality is already in place as far as I can tell, it just needs some TLC. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] disable (or increase) the 'long DB request' warning
Paul J Stevens wrote: > Ah those are missing from the default config and the manpage: > > [DBMAIL] > query_time_info=10 > query_time_message=20 > query_time_warning=30 > Sweet. Should I open a bug for this so it doesn't fall off the radar, or did someone already patch the docs on the spot? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] disable (or increase) the 'long DB request' warning
When I run my dbmail-util -ay weekly, I get a result laced with warnings about the DB taking too long as seen below. Is there a way to disable this, or increase the amount of seconds at which the warning is triggered? DBMail is 2.10 Thanks == Repairing DBMAIL messageblocks integrity... Ok. Found [0] unconnected messageblks. Repairing DBMAIL physmessage integrity... Ok. Found [0] unconnected physmessages. Repairing DBMAIL message integrity... Ok. Found [0] unconnected messages. Repairing DBMAIL mailbox integrity... Ok. Found [0] unconnected mailboxes. Repairing DBMAIL for NULL messages... Ok. Found [0] NULL messages. Checking DBMAIL for NULL physmessages... Sep 24 04:03:53 Arzamas dbmail-util[7779]: Message:[db] dbmodule.c,db_query(+149): slow query [SELECT pm.id FROM dbmail_physmessage pm LEFT JOIN dbmail_messageblks mbk ON pm.id = mbk.physmessage_id WHERE mbk.physmessage_id is NULL] took [22] seconds Ok. Found [0] physmessages without messageblocks. Deleting messages with DELETE status... Ok. [3591] messages deleted. Repairing DBMAIL for rfcsize field... Ok. Found [0] missing rfcsize values. Repairing DBMAIL for incorrect is_header flags... Sep 24 04:16:29 Arzamas dbmail-util[7779]: Message:[db] dbmodule.c,db_query(+149): slow query [SELECT MIN(messageblk_idnr),MAX(is_header) FROM dbmail_messageblks GROUP BY physmessage_id HAVING MAX(is_header)=0] took [654] seconds Sep 24 04:16:29 Arzamas dbmail-util[7779]: Warning:[db] dbmodule.c,db_query(+151): slow query [SELECT MIN(messageblk_idnr),MAX(is_header) FROM dbmail_messageblks GROUP BY physmessage_id HAVING MAX(is_header)=0] took [654] seconds Ok. Found [0] incorrect is_header flags. Repairing DBMAIL for cached envelopes... Ok. Found [0] missing envelope values. Repairing DBMAIL for cached header values... Sep 24 04:16:58 Arzamas dbmail-util[7779]: Message:[db] dbmodule.c,db_query(+149): slow query [SELECT p.id FROM dbmail_physmessage p LEFT JOIN dbmail_headervalue h ON p.id = h.physmessage_id WHERE h.physmessage_id IS NULL] took [21] seconds Ok. Found [0] un-cached physmessages. Setting DELETE status for deleted messages... Ok. [6908] messages set for deletion. Re-calculating used quota for all users... Ok. Used quota updated for all users. Removing aliases with nonexistent delivery userid's... Ok. Found [0] dangling aliases. Vacuuming and optimizing database... Ok. Database cleaned up. Maintenance done. No errors found. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] feature for dbmail-util
Uwe Kiewel wrote: > > My conclusion for this topic after our short discussion is to let > dbmail-util maintain the content and > to maintain the db itself by a dba. Cool :) > Hint: I'm still using the 2.2 schema. Same here. 2.3 is way too new for my taste. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] feature for dbmail-util
Uwe Kiewel wrote: > Hi there, > > I know, Paul is on holiday. > > But in relation to Jorges "db growing size" thread: > > Is it an option to include the "OPTIMIZE TABLE" parts into the code of > dbmail-util? Let's discuss about advantages and disadvantages! > Placing this functionality in dbmail-util would suggest that it is ok to run this daily/weekly/what have you, and I'd say this is an extremely bad idea. As someone said earlier in Jorge's thread this is a task for the DBA, not for the dbmail admin. My dataset size is 25GB, and an OPTIMIZE on the messageblocks takes about 2 hours on a dedicated server. The entire table is under an exclusive lock during this time[1], which renders the entire suite (both insertion and IMAP/POP access) stall with timeouts. Besides, specifically for Innodb running with file_per_table=1 (which is what everyone should do anyway), the OPTIMIZE TABLE statement doesn't do much. The first (and only) time I ran it was 10 months after deploying my setup, and it saved me a total of 250M, out of roughly 28G. The total downtime was about 4 hours (I did this at night). Clearly nothing to gain here. My 2c Peter [1] http://dev.mysql.com/doc/refman/5.0/en/optimize-table.html ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Db growing size
Jorge Bastos wrote: > Yes Aaron but just notice, with 2.2x it didn't happen. > I mean, the used space by the innodb tables, wasn't this huge like now. > Jorge, With all due respect it would be much more constructive, if you were to quit whining, and attempt the actions suggested to you numerous times in this thread. You are the only one having a problem with this, so the "burden of proof" is entirely yours. Please let us know how did the OPTIMIZE and/or mysqldump went. Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Db growing size
Jorge Bastos wrote: > Hum on messageblks i gained about 800MB less. > On headervalue gained 900MB less. > But the problem is dbmail_mimeparts, that has the complete email. > It has 45GB, that doesn’t corresponde to the email that is stored. > Can I consider a bug here? > I didn't run optimize on this table... I'm scared to do it :) > What do you say paul? Even without optimize this should contain the correct > size regarding the email that is stored? > Jorge, Take a mysqldump of your data, and pipe it to another test database (simply create a new database, and let the mysql load recreate the tables there). This way you are not going to lock your tables for 48h optimization, and you will get a precise idea of what you would gain. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Db growing size
Jorge Bastos wrote: > Hi Paul, > > > > In 2.3x version, with the new database design, the size that the > database has, it’s not the real size. > > I have now 52GB of DB Size, but comparing with the email that is stored > in this very moment: > > “SELECT round((sum(curmail_size)/1024)/1024,2) FROM dbmail_users ;” > > Result: > > 8648.34 > > Wich mean that I have currently about 8,5GB of email. > > > > Is there something to tune this? > > Is there a special reason for this huge size? > In case you are using the standard innodb pool - yes. The pool never shrinks, and unless you "defragment" it, it will keep growing. In order to avoid this situation it is very advisable to use http://dev.mysql.com/doc/refman/5.0/en/multiple-tablespaces.html ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] ANNOUNCE: dbmail-2.2.11 release candidate 1
Hi Paul, Any chance for bug#672 to be addressed? Thanks ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail postgresql
Curtis Maurand wrote: Is there an advantage in using PostgreSQL vs. MySQL (no flame wars please.)? I'm just looking for an relatively objective answer. I'm sure there are passions on both sides. I've been using MySQL for years without any difficulty except that the innodb file just gets larger and larger even though the mail store doesn't. I've been thinking about experimenting with the pbxt storage engine (Also ACID compliant). If your only issue is the innodb file - simply switch to a file-per-table storage. The "migration" is extremely simple - you stop all database access, dump your database (mysqldump), set innodb_file_per_table = 1, drop/recreate the database, and load the dump back in. HTH Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Explanation of dbmail-util -ay run
Peter Rabbitson wrote: I have never seen this happen before, and there were no dbmail upgrades either. What could have caused this and what did I possibly lose? Repairing DBMAIL physmessage integrity... Ok. Found [22409] unconnected physmessages Ok. Orphaned physmessages deleted. ... After asking some more around, it appears that during this particular week (I run dbmail-util weekly instead of daily) someone deleted a user like this: dbmail-users -d The program ran for about 3 - 4 minutes and returned to the prompt. Can the above messages about orphans be connected to such an action, and is this normal? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-lmtpd cannot find localhost
erik pepermans wrote: May 31 15:33:05 relay dbmail/lmtpd[5644]: Error:[misc] misc.c,drop_privileges(+76): could not find group nogroup Group nobody and user nobody exist ! On your system you have noBODY and dbmail is clearly looking for noGROUP. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Follow up on Error:[imap] imap4.c, IMAPClientHandler(+303): command return with error [idle]
About half a year ago it was acknowledged that dbmail's behavior in this regard is incorrect: http://www.mail-archive.com/dbmail@dbmail.org/msg13638.html. Currently I am running 2.2.10 and the issue is still present. Did it just fall off the radar or did I miss some further discussion about it? Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Explanation of dbmail-util -ay run
I have never seen this happen before, and there were no dbmail upgrades either. What could have caused this and what did I possibly lose? Repairing DBMAIL physmessage integrity... Ok. Found [22409] unconnected physmessages Ok. Orphaned physmessages deleted. ... ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] upgrade failure
Paul J Stevens wrote: Curtis, It is quite safe to delete and re-create all the cache tables as created by the 2.0_2.2 upgrade script, and re-run dbmail-util -by after that. No messages will be lost. the whole list of caching tables: | dbmail_ccfield| | dbmail_datefield | | dbmail_envelope | | dbmail_fromfield | | dbmail_headername | | dbmail_headervalue| | dbmail_referencesfield| | dbmail_replycache | | dbmail_replytofield | | dbmail_subjectfield | | dbmail_tofield| I can confirm this works: http://www.mail-archive.com/dbmail@dbmail.org/msg14505.html ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Exporting users from DBMail to Postfix lookup table
Aleksander Kamenik wrote: As for the postfix configuration, it looks like this for me, I'm pretty sure I found it in the dbmail README or wiki. mail:/etc/postfix # grep local_recipient_maps main.cf local_recipient_maps = mysql:/etc/postfix/sql-recipients.cf Just FYI - make sure to add proxy: to all your mysql lookups just like Josh Marshall did here http://www.mail-archive.com/dbmail@dbmail.org/msg14781.html Without this you are risking a DoS attack on your sql server (too many concurrent connections). ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] duplicate key on dbmail_headername_1
Paul J Stevens wrote: Peter Rabbitson wrote: Paul J Stevens wrote: Messages without cached headers can still be retrieved through IMAP or POP3, though in the case of IMAP they will look funny in the list-pane, and look just fine in the preview pane. Isn't this a potential mail-loss scenario? A user opens his email client, sees weird subjects and deletes them without looking inside. Is it possible to add a layer of translucency to the cache table? So if the cached header for a message is not present, the real physmessage is parsed instead? That would defeat the purpose of the cache tables. Imagine where an IMAP client requests a list of headers: x UID FETCH 1:* (BODY.PEEK[HEADER.FIELDS (References X-Ref X-Priority X-MSMail-Priority X-MSOESRec Newsgroups)]) (this is from the outlook fetch command). It is perfectly valid for an imap client to request headers that may not be there. So if you have a mailbox with messages, none of which have for example the X-MSOESRec header, dbmail would have to retrieve each of those messages in full to do a double check for the non-existence of a header that is (correctly) not in the caching tables. Good point. However I still think that caching (and subsequently losing) the subjet is inherently bad. Probably the Subject needs preferential treatment, just like From/To as they seem to not be affected by cache presence? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] duplicate key on dbmail_headername_1
Paul J Stevens wrote: Messages without cached headers can still be retrieved through IMAP or POP3, though in the case of IMAP they will look funny in the list-pane, and look just fine in the preview pane. Isn't this a potential mail-loss scenario? A user opens his email client, sees weird subjects and deletes them without looking inside. Is it possible to add a layer of translucency to the cache table? So if the cached header for a message is not present, the real physmessage is parsed instead? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Feature request: do-as-if-alive for POP and IMAP
Michael Monnerie wrote: On Mittwoch, 26. März 2008 Paul J Stevens wrote: Wouldnt it be easier *not* to take your database offline? Yes of course, but you know, that's a dream sometimes. Last week we moved the DB from a RAID6 to RAID10 after a hardware upgrade, this caused a (short) downtime. Yes we want to have a replicated DB on another server, but that doesn't exist ATM. And even if: the option to have a maintenance mode to fool clients is good to have, for example for kernel or shared lib updates that require a reboot. The right thing to do is not to be lazy, and do all this at 4am of our local time zone. I bet you will not receive _any_ complaints. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] (patch) Another case of duplicate keys (via dbmail-smtp)
Hi Paul, I see that the fix made it to the repository. Do you have an ETA for a 2.10 debian package? The 2.9 version has rolled into official debian testing and I need to figure out how to proceed with my upgrade plans (i.e. to mark the packages as held, or just wait a couple of days if a new release comes soon). Either way is fine, I just wanted to stay informed :) Thanks ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: AW: AW: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
John Fawcett wrote: in squirrelmail I see: - ---start--- Received: by mail.gufonero.com (Postfix, from userid 0) id 8DB13AFB94; Tue, 4 Mar 2008 03:39:04 -0500 (EST) Received: by .gufonero.com (Postfix, from userid 0) id 8DB13AFB94; Tue, 4 Mar 2008 03:39:03 -0500 (EST) To: [EMAIL PROTECTED] Subject: Test Message-Id: <[EMAIL PROTECTED]> Date: Tue, 4 Mar 2008 03:39:04 -0500 (EST) From: [EMAIL PROTECTED] Return-Path: [EMAIL PROTECTED] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Is it possible for you to verify how the same message looks like when seen from OE? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: AW: AW: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
John Fawcett wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Peter Rabbitson wrote: John Fawcett wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Peter Rabbitson wrote: Ah, I just wrote about this in another message. So to confirm - when displaying in a MUA a message retrieved from dbmail, I see no return path header (unless I add it at my MTA). Peter that's exactly what I get. However, if I do a select on dbmail_headervalues table, I find the Return-PAth value stored there. I'm not sure where MUAs are expecting to get this, from the message header or from the imap server directly, but I think it should go into the message header too. So what is the verdict on this? Should I be adding a RP just before handing email to dbmail, or should dbmail append it both in the headers and physmessage, or does it matter at all? with the patch applied I could not find issues whether you add RP in the MTA or not. My question was was about the fact that currently dbmail does not record the RP in the body of the message. I reviewed the patchset and don't see this addressed. The bug indicated that OE discards information from the IMAP server and reads the message body to get whatever it needs. So my question is more like: will dbmail be adjusted in future version to insert RP in the message body as well, or is this something in violation of the protocols? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: AW: AW: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
John Fawcett wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Peter Rabbitson wrote: Ah, I just wrote about this in another message. So to confirm - when displaying in a MUA a message retrieved from dbmail, I see no return path header (unless I add it at my MTA). Peter that's exactly what I get. However, if I do a select on dbmail_headervalues table, I find the Return-PAth value stored there. I'm not sure where MUAs are expecting to get this, from the message header or from the imap server directly, but I think it should go into the message header too. So what is the verdict on this? Should I be adding a RP just before handing email to dbmail, or should dbmail append it both in the headers and physmessage, or does it matter at all? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] (patch) Another case of duplicate keys (via dbmail-smtp)
John Fawcett wrote: We can spend time trying to get this to work, but I'm just wondering whether we need to use g_mime_message_set_header at all. Seeing as in the end we must have only one Return-Path header in the delivered message and we are trying to update an existing header, maybe it would be easier to ignore existing return-path headers and just add our own: that way we also avoid duplicate Return-Path headers in the in-memory structure which is the fundamental issue. I don't know much about C if anything, but just to chime in. As far as I understand this code deals with general header handling. Isn't it perfectly legal to have several X-headers with absolutely the same values in a message? What happens then? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
Paul J Stevens wrote: Peter Rabbitson wrote: Paul J Stevens wrote: please try the attached patch to see if this fixes it. I will not be able to take the system offline during business hours EST (and I have only 13 minutes left). If by the end of today Simon has not tested it yet I certainly will. Paul - is there anything that can be done about the messages with missing headers? I have about 2000 of them (1.5 days worth). Is it safe to continue running dbmail-utils? I can provide you with any sql queries you might want to see. You have to make sure all cache entries for the physmessages involved are cleaned out from the relevant tables: | dbmail_ccfield| | dbmail_datefield | | dbmail_envelope | | dbmail_fromfield | | dbmail_headervalue| | dbmail_referencesfield| | dbmail_replytofield | | dbmail_subjectfield | | dbmail_tofield| If you clean out above tables for those messages, you can safely run dbmail-util -by and the cache will be rebuild for them. What would be the query to find the ids of affected messages? Should they be missing a dbmail_hedervalue with an id corresponding to the dbmail_headername entry of return-path? I am not quite sure how the subject gets lost... ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
Paul J Stevens wrote: please try the attached patch to see if this fixes it. I will not be able to take the system offline during business hours EST (and I have only 13 minutes left). If by the end of today Simon has not tested it yet I certainly will. Paul - is there anything that can be done about the messages with missing headers? I have about 2000 of them (1.5 days worth). Is it safe to continue running dbmail-utils? I can provide you with any sql queries you might want to see. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: AW: AW: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
John Fawcett wrote: I'm not sure where MUAs are expecting to get this, from the message header or from the imap server directly, but I think it should go into the message header too. Considering that OE ignores what the server returns (i.e. no subject) and just parses out the subject from within the email body, I would think that it behaves the same for other headers including return-path. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: AW: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
Simon Lange wrote: First thing I also noticed when running dbmail util after 2.2.6rc1 -> 2.2.9 was that dbmail killed 730 unconnected physmessages.. I'd disable the autorun of dbmail-util untill we hear from the developers. The database might be inconsistent, but it seems to behave correctly as far as users are concerned. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: AW: AW: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
John Fawcett wrote: Paul just to pick up on this. You saw from the log insert that there was only one Return-Path header. However from my limited experience the Return-Path header added by dbmail is only stored in dbmail_headervalues and never added to the phymessage. Up to now I was using cyrus and all my migrated messages have Return-Path in phymessage. New messages received with dbmail do not have Return-Path in phymessage, but there is always a value in dbmail_headervalues for Return-Path. Ah, I just wrote about this in another message. So to confirm - when displaying in a MUA a message retrieved from dbmail, I see no return path header (unless I add it at my MTA). ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
John Fawcett wrote: There should only be one Return-Path header and that should be written by the final delivery agent, so there isn't much alternative here. Return-Path headers should only be used for final delivery, so should not be added to forwarded messages. However MTAs should remove existing Return-Path headers, so I doubt this will be a problem. I tried this - disabled addition of return-path in postfix (removed the R flag). This solves the problem with 2.9, but then my emails end up with no return-path at all (both under 2.7 and 2.9). I am currently back to 2.7, with return-path append enabled in the MTA (it considers dbmail-smtp the final delivery). ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
John Fawcett wrote: I think that it would be a good idea to add a guard to dbmail to remove existing Return-Path headers so that this problem does not occur. E probably not add anything if a duplicate is detected. I definitely do not want my headers tampered with unless necessary. In the meantime it is probably best to configure the MTA not to add the Return-Path header. Email often passes through dbmail twice, so the MTA will not solve anything. I have a number of dbmail forwards like: [EMAIL PROTECTED]: [EMAIL PROTECTED] so the email loops through the MTA and comes back to the correct mailbox. This is to work against stupid forms/systems that do not support '+' ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] HMM got lmtp dupicate entries HOW and WHY?
Simon Lange wrote: I started getting the same errors that Simon describes right after upgrading to 2.9. Worst part is that this error actually has some very nasty consequences. Please look at bug #685. Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Re: DBMail 2.3.0 released
James Cloos wrote: "Jani" == Jani Partanen <[EMAIL PROTECTED]> writes: Jani> Every time when you hash something what is bigger than your Jani> returned hash, there can be collision. You still talking about one chance in on the order of 1. Is this what you will tell to the CEO when his quarterly report turns out to be a goatse pic? :)) The question is not about how small is the chance, but that there is in fact a chance of this happening. The developers seem to be aware of this, and are indicating that it will be ironed out before 2.4 is out, so things seem to be OK. OTOH I will remain a loyal 2.2 user if 2.4 does not have a bit-by-bit comparison of a just-received attachment and an entry in the database with the same hash. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] enormous DB Traffic using imapsync
Paul J Stevens wrote: Michael Monnerie wrote: I make an imapsync to my mailserver, which connects to the PostgreSQL db, and I get these numbers: mailsrv -> db 15.482.309 db -> mailsrv 648.119.851 client -> mailsrv 1.771.458 mailsrv -> client 15.956.176 So with ~18MB IMAP traffic I produce ~663MB database traffic? Can somebody explain me this? Interesting figures. Can you produce a base-line? Like say: traffic numbers for syncing a single account, containing a single folder with a single message of a known size, blah blah. Log the query patterns triggered, and the traffic numbers involved. Out of curiosity. Perhaps the explanation will jump out if the logs. One hypothesis a priori: we don't cache as much as we could. In fact, dbmail caches *very* little and will at times happily call the same query a thousand times. And it doesn't even use prepared statements (yet). I also had a hard time migrating 20GB of mbox email to dbmail, it actually took about 12 days, 3 imapsync runs with wrappers alerting me when things get ugly (granted I had the emails evenly distributed between accounts, so I could arrange a staged migration). The figures you are talking about can be easily derived using my torture script which I wrote to track down the 2.2 memory leaks some months ago. You guys even included it in the test-scripts section. On a quick glance it will need the following modifications: * in init_db() function, so that all resulting emails will be identical (now the subject is an incrementing number) * in torture() imapsync is supplied a skipheader that may no longer be relevant * at the end of the script the torture runs twice with and without G_SLICE=always-malloc which is irrelevant * instead of t_memcheck() the dbmail-imapd daemon should be started by t_normal() Cheers! ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] set library parameter
Jorge Bastos wrote: Dec 9 18:34:01 lira dbmail/imap4d[4206]: Debug:[db] dbmodule.c,db_load_driver(+53): no value for library_directory, using default [/usr/lib/dbmail] RTFManpage dbmail.conf(5) library_directory=/usr/lib/dbmail # directory for locating libraries ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Message visible in SquirrelMail, invisible in OutlookExpress
Jorge Bastos wrote: I had this, maybe it's my post you're talking about. There was one situation that is, the message exists in outlook express, but sinse it has a crazy date, it appears at the first. Can you do a search by the subject of that message to really confirm it? This is the Inbox of a sales account. All messages are moved to different folders once being read. At the time of writing there are 5 messages in the Inbox. There is no way I can miss it. It is a spam message, so I guess some very weird message inconsistency. Still it doesn't seem quite right. I will investigate this more during the night, taking a snapshot of the database so I don't disrupt the message causing it to all of a sudden appear. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Message visible in SquirrelMail, invisible in OutlookExpress
I am almost 100% sure there was a report like this about half a year ago, but I can not find it to see what is the resolution. Thus the likely duplicate question - what might be causing the behavior described in the subject? Running 2.2.7. Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Offtopic (sorry)
Paul J Stevens wrote: Josh Marshall wrote: Hi, I tested with imapsync and found that it caused duplicates. The reason being that DBMail puts the message id in the header - then the emails don't match and next time you sync it copies across again. Dbmail doesn't do that anymore. Still it is probably the case for messages inserted in the db with an older version of DBMail, no? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Offtopic (sorry)
Josh Marshall wrote: Hi, I tested with imapsync and found that it caused duplicates. The reason being that DBMail puts the message id in the header - then the emails don't match and next time you sync it copies across again. Read the man page - there are clear instructions on how to ignore specific headers. There is the script[1] that was used to catch most of the memory leaks present in dbmail 2.2 - it heavily relies on imapsync in torture(). There is also a FAQ[2] for impasync which has some outdated information specific to dbmail, but you should get the idea nevertheless. [1] http://svn.ic-s.nl/websvn/filedetails.php?repname=DBMail&path=%2Ftrunk%2Fdbmail%2Ftest-scripts%2Ftorture.sh&rev=0&sc=0 [2] http://www.linux-france.org/prj/imapsync/FAQ ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Mailling lists
Paul J Stevens wrote:
Aaron Stone wrote:
On Mon, Nov 5, 2007, Jorge Bastos <[EMAIL PROTECTED]> said:
I need to setup a mailling list, how can I do that? Is this supported?
I put this on the wiki two years ago:
http://dbmail.org/dokuwiki/doku.php?id=mailman
However, I remember seeing a much better way to do this on the mailing
list more recently. Something about mailman domains. I don't recall the
details. Let's update the wiki with whatever the best-practice is.
That was me talking about the mailman transport in postfix. All documented in
the mailman mta integration docs.
/etc/postfix/transport:
lists.mydomain.com mailman:
/etc/postfix/master.cf:
mailman unix - n n - - pipe flags=FR user=list
argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${mailbox}
The mentioned python script (postfix-to-mailman.py) contains the above setup
documentation as well.
After setting this up, all you need to do is create lists (newlist listname),
and you're all set. No need to manage aliases.
The approach above makes it cumbersome to use anything other than the
list addresses and the administrative addresses postfix-to-mailman.py
knows about (currently postmaster@, abuse@, mailer-daemon@). In my case
I wanted to add admin@ and webmaster@, for which I'd have to edit said
python script which seemed messy. Then I realized that mailman has
automatic alias management for quite some time now. The relevant option
in the config is
MTA = 'Postfix'
which will result in the file /var/lib/mailman/data/aliases populated
with pipes to the correct binaries, automatically kept up-to-date by
internal mailman hooks, and a postalias-ed result in aliases.db. Then
all you have to do is populate the postfix recipient list as you would
do otherwise:
# use system-level aliases + automatically generated alias lists
alias_maps = hash:/etc/postfix_promo/admin_aliases
hash:/var/lib/mailman/data/aliases
# there is nothing that can be updated by newaliases
alias_database =
# we only accept mail to existing lists and admin addresses
local_recipient_maps = $alias_maps
No transport trickery and very straightforward.
On a side note mailman scales extremely bad when you reach 50k+
subscribers. I am planning to migrate to sympa[1], but have not had the
time to make the necessary research. Anyone using this software?
[1] http://www.sympa.org/
___
DBmail mailing list
DBmail@dbmail.org
https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Totally OT
Hi, I would like to know what software was used to draw/generate this: http://www.dbmail.org/dokuwiki/lib/exe/detail.php?id=er-model&cache=cache&media=dbmail22-er-model-transparent.png. Thanks in advance. Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Difference in option case in dbmail.conf
Hi, Is there a reason why most of the options in dbmail.conf are in UPPERCASE whereas some are lower case? Is there some significance like mainstream/experimental option or it is just a mess that has not been cleared yet? Thanks ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail 2.2.7 candidate 4
Paul J Stevens wrote: Hi all, Let's roll another one. changes since rc3: - HARD_MAX_CHILDREN was increased from 200 to 300. Not as significant as I would like, but increasing it much beyond caused new problems I rather not have to deal with right now. - build on OSX with gcc should work now as well (bug #651) - all known problems with rfc2047 (utf7) encoding of header lines have been fixed (bug #649). - some usage of private sqlite3 call was removed that caused errors in ubuntu/gutsy's version of sqlite3. - allow IDLE after authentication, rather than only after selecting a mailbox. That should make IDLE especially more Outlook friendly (bug #648). Hi. After the RC1 troubles I am a bit nervous before upgrading, but otoh I need the IDLE fix. So a little survey - after a week in the wild did anyone experience any problems with RC4 yet? Thanks ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] CPU LOAD
Paul J Stevens wrote: Mmm, the rfc is less than clear imo. I'll take Arnts word for it. I'll fix this asap. Hi Paul, Just saw the rc3 announcement. Does it include the stuff quoted above, or it is left for rc4? Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Extensions of catch-all addresses
Aaron Stone wrote: Please post a level 5 trace so that we can see if the lookup code is running or not. On Wed, Aug 8, 2007, Peter Rabbitson <[EMAIL PROTECTED]> said: Hi, I was wondering what happened with the issue discussed here: http://article.gmane.org/gmane.mail.imap.dbmail/8068/match=extension+send+mail+sub+folder I have an alias user@, with no other occurrences of [EMAIL PROTECTED] Currently (2.2.6rc1) any mail to user+stuff gets a 550 Recipient FAIL. Thanks Hi Aaron, Did you have a chance to look at this yet? Should I file a bug or it is something very stupid on my end? Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Use an smtp transport instead of sendmail pipes
Hi, Is it possible to have dbmail send all external forwards directly to an smtp host/port instead of invoking the local sendmail? The rationale is to still being able to scan the local Postfix maildrop (/usr/sbin/sendmail) with a global content_filter setting, but to bypass this scan for all mail coming from dbmail, since it was scanned once when it got into dbmail already. Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Extensions or catch-all addresses
Aaron Stone wrote:
Please post a level 5 trace so that we can see if the lookup code is
running or not.
Here is the entire setup:
[EMAIL PROTECTED]:~# dbmail-users -l
-- users --
anyone:x:2:0:0.00:0.00:
tester:x:4:0:0.00:0.00:test@
[EMAIL PROTECTED]@__:x:1:0:0.00:0.00:
__public__:x:3:0:0.00:0.00:
-- forwards --
[EMAIL PROTECTED]:~#
This is a problematic delivery:
[EMAIL PROTECTED]:/# echo | dbmail-smtp -d [EMAIL PROTECTED]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_load_driver(+53): no value for library_directory, using
default [/usr/lib/dbmail]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_load_driver(+69): looking for mysql as
/usr/lib/dbmail/libmysql.so
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Warning:[sql]
dbmysql.c,db_connect(+165): MySQL host is set to localhost, but no
mysql_socket has been set. Use sqlsocket=... in dbmail.conf. Connecting
will be attempted using the default socket.
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_query(+290): query [SET NAMES utf8]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_query(+145): last query took [0] seconds
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_query(+290): query [SHOW VARIABLES LIKE 'collation_%']
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_query(+145): last query took [0] seconds
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_mysql_check_collations(+120): does
[collation_database:utf8_general_ci] match
[collation_connection:utf8_general_ci]?
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[auth]
authmodule.c,auth_load_driver(+48): no value for library_directory,
using default [/usr/lib/dbmail]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[auth]
authmodule.c,auth_load_driver(+64): looking for auth_sql as
/usr/lib/dbmail/libauth_sql.so
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_query(+290): query [SELECT 1=1 FROM dbmail_physmessage
LIMIT 1 OFFSET 0]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_query(+145): last query took [0] seconds
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_query(+290): query [SELECT 1=1 FROM dbmail_headervalue
LIMIT 1 OFFSET 0]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_query(+145): last query took [0] seconds
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_query(+290): query [SELECT 1=1 FROM dbmail_envelope LIMIT 1
OFFSET 0]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_query(+145): last query took [0] seconds
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[message]
dbmail-message.c,_set_content_from_stream(+426): parse message
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[smtp]
main.c,main(+289): no return path found
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Info:[dsn]
dsn.c,dsnuser_resolve(+578): checking if [EMAIL PROTECTED] is a
valid username, alias, or catchall.
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[auth]
authsql.c,auth_check_user_ext(+222): checking user
[EMAIL PROTECTED] in alias table
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[auth]
authsql.c,auth_check_user_ext(+239): checks [0]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_query(+290): query [SELECT deliver_to FROM dbmail_aliases
WHERE lower(alias) = lower('[EMAIL PROTECTED]') AND lower(alias) <>
lower(deliver_to)]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_query(+145): last query took [0] seconds
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[auth]
authsql.c,auth_check_user_ext(+266): user [EMAIL PROTECTED] not in
aliases table
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[dsn]
dsn.c,address_has_alias(+251): user [EMAIL PROTECTED] found total
of [0] aliases
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[auth]
authsql.c,auth_check_user_ext(+222): checking user [EMAIL PROTECTED] in
alias table
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[auth]
authsql.c,auth_check_user_ext(+239): checks [0]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_query(+290): query [SELECT deliver_to FROM dbmail_aliases
WHERE lower(alias) = lower('[EMAIL PROTECTED]') AND lower(alias) <>
lower(deliver_to)]
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[db]
dbmodule.c,db_query(+145): last query took [0] seconds
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[auth]
authsql.c,auth_check_user_ext(+266): user [EMAIL PROTECTED] not in
aliases table
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[dsn]
dsn.c,address_has_alias_mailbox(+278): user [EMAIL PROTECTED] found
total of [0] aliases
Aug 09 11:00:28 Thesaurus dbmail-smtp[2467]: Debug:[sql]
dbmysql.c,db_query(+290): query [SELECT user_idnr FROM dbmail_users
WHERE lower(userid) = lower('[EMAIL PROTECTED]')]
Aug 0
[Dbmail] Extensions or catch-all addresses
Hi, I was wondering what happened with the issue discussed here: http://article.gmane.org/gmane.mail.imap.dbmail/8068/match=extension+send+mail+sub+folder I have an alias user@, with no other occurrences of [EMAIL PROTECTED] Currently (2.2.6rc1) any mail to user+stuff gets a 550 Recipient FAIL. Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: R: [Dbmail] Advantages of innodb_file_per_table (was: Table dbmail_messages is full)
Andrea Brancatelli wrote: Summarizing everything: - If you have a single DB server: use a shared InnoDB tablespace preallocating the space and disabling the autoextend. Using the optimize table will give you a better optimization of the tables, and you'll have no problem with the space as it's already allocated up to a fixed size - If you have a machine with various tasks going on, like a mail server, web server, db server and whatever, use the innodb_file_per_table. Usigon the optimize table you'll reclaim your space back whenever you delete anything or whenever any table will significantly decrease in size. Doubt? Question? Fear? Panic? Excellent writeup! So basically it helps avoid juggling around 50GB+ files. One question though - does it make any difference in performance? You said that you have not done any benchmarking - did anyone else on the list? When one has 15GB of email, performance counts. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Advantages of innodb_file_per_table (was: Table dbmail_messages is full)
Hi, Sorry for hijacking the thread. Can someone clarify what is the advantage of using innodb_file_per_table versus one infinitely growing tablespace? Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Backups
Daniel Kasak wrote: Hi all. I've been using mysqldump ( and the binary logs ) to backup our databases, including dbmail. This works, but restores are *incredibly* slow. We've got almost 20GB of email. Have you looked at LVM snapshots? The idea is basically to have your entire Database on a LVM volume (it doesn't have to be a multidisk aggregation, just the LVM layer is what matters). Then you can use the mysql client to issue a flushing lock, create a snapshot block device, and release the lock. After this you have an atomic snapshot of the latest database, which you can simply tar away. Restore from a backup then is as simple as stopping mysql for 5 seconds, copying(renaming) the directory containing your data, tarting mysql. If you find this promising and can not find enough information on the internet, let me know and I will put together a more detailed explanation. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Advice on tuning the startup of a MySQL server on Debian
Charles Marcus wrote: This is not strictly a DBMail question, but it is close. Lately I noticed that any time the SQL server needs to be restarted for some reason, DBMail starts having problems with inserting messages. Since the error message was changed to 450 from 503, this is not a big issue, as mail is not lost. Nevertheless I want to know if someone else had such problems and what steps if any were taken to solve them. What about having an option to fallback to a temp-deliver-to-filesystem mode when the DB is temporarily unavailable (and then move messages from the filesystem to the DB when DB connectivity is restored)? You could then have an option that if the DB is unavailable for longer than x minutes, *then* start temp failing delivery. Just a thought... Err... Isn't this exactly what the MTA in front of dbmail is for? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Advice on tuning the startup of a MySQL server on Debian
Paul J Stevens wrote: Peter Rabbitson wrote: Hello all, This is not strictly a DBMail question, but it is close. Lately I noticed that any time the SQL server needs to be restarted for some reason, DBMail starts having problems with inserting messages. Since the error message was changed to 450 from 503, this is not a big issue, as mail is not lost. Nevertheless I want to know if someone else had such problems and what steps if any were taken to solve them. The situation is as follows. Under Debian the mysql init script runs the following upon the start of the daemon: /usr/bin/mysqlcheck --defaults-file=/etc/mysql/debian.cnf --all-databases --fast --silent This process is pretty resource intensive and causes the following to happen in DBMail from time to time: Jul 20 09:29:23 Arzamas dbmail-lmtpd[2954]: Error:[sql] dbmysql.c,db_query(+293): [Lock wait timeout exceeded; try restarting transaction] [INSERT INTO dbmail_messageblks (is_header, messageblk,blocksize, physmessage_id) VALUES (_omitted_)] I got this every time I restarted the mysql server on a dbmail host. You can tune the timeout involved /etc/mysql/my.cnf: innodb_lock_wait_timeout But my choice would be to disable the check run. Considering your experience with MySQL servers, were there any situations in which the check was actually worth it? Or is it largely useless with the current state of the InnoDB engine? Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Advice on tuning the startup of a MySQL server on Debian
Hello all, This is not strictly a DBMail question, but it is close. Lately I noticed that any time the SQL server needs to be restarted for some reason, DBMail starts having problems with inserting messages. Since the error message was changed to 450 from 503, this is not a big issue, as mail is not lost. Nevertheless I want to know if someone else had such problems and what steps if any were taken to solve them. The situation is as follows. Under Debian the mysql init script runs the following upon the start of the daemon: /usr/bin/mysqlcheck --defaults-file=/etc/mysql/debian.cnf --all-databases --fast --silent This process is pretty resource intensive and causes the following to happen in DBMail from time to time: Jul 20 09:29:23 Arzamas dbmail-lmtpd[2954]: Error:[sql] dbmysql.c,db_query(+293): [Lock wait timeout exceeded; try restarting transaction] [INSERT INTO dbmail_messageblks (is_header, messageblk,blocksize, physmessage_id) VALUES (_omitted_)] My question is - should I try to fix this at all? Is the mysqlcheck process useful at all or should I disable it on startup? Or can I tune some timeouts within dbmail? Any input would be appreciated. Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] DBmail webmail app
Paul J Stevens wrote: Tom Allison wrote: My point also. Doing sha1 on the whole file seems to make the most sense. People who want to do bigger files faster can simply add more lmtpd hosts with bigger cpus and more ram. Why is the hashing performance penalty considered such an issue? All of the hashing algorithms are streaming, hence all daemons capable of accepting a message (smtp/lmtp/imap) can call a utility function on the data that is unloaded from the socket. Or no? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-util extension, was: Just a question
Jesse Norell wrote: And if anyone is into dbmail-util, lets not forget bug id 305 .. I think it's managed to slip by unaddressed for well over a full year now (acknowledged, high priority and several sql solutions listed in the report). Neither this issue: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=421018. I filed it under the debian bug tracker because I did not know better at the time. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] DBmail webmail app
Peter Welzien wrote: Hi. I'm writing a webmail application that works directly with the SQL-server. Actually a question to the authors: Paul, Aaron, are you as religious about the DBMail internals as Wietse Venema is about the postfix queue? :) Joking aside - is the internal format considered stable enough that such applications are safe to use, and do not risk destroying some inter-table dependencies? If so (it is stable) is it documented somewhere/are there plans on documenting it? Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak
Hi Aaron, Here are the new test results for revision 2574: http://rabbit.us/pool/misc/dbmail_bug584_memchek_svn2574.tar.bz2. Let me know if I can provide anything else. Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak
Aaron Stone wrote: > Looks like we were not correctly freeing the lists returned by > g_list_slices. I've updated dbmail-imapsession.c, dbmail-mailbox.c, and > db.c with a fix. Hopefully imapsync users will feel less pain. Please > test out SVN and let me know! > > Aaron > It does not seem like there is any difference at all :( I am currently running another valgrind check, will send you the details shortly. Just to make sure that we are on the same page: I am compiling: [EMAIL PROTECTED]:~/devel/dbmail$ svn info Path: . URL: https://svn.ic-s.nl/svn/dbmail/branches/dbmail_2_2_branch Repository Root: https://svn.ic-s.nl/svn/dbmail Repository UUID: 7b491191-dbf0-0310-aff6-d879d4d69008 Revision: 2574 Node Kind: directory Schedule: normal Last Changed Author: aaron Last Changed Rev: 2574 Last Changed Date: 2007-05-15 22:55:38 +0200 (Tue, 15 May 2007) The dbmail-imapd version is: [EMAIL PROTECTED]:~# which dbmail-imapd /usr/local/sbin/dbmail-imapd [EMAIL PROTECTED]:~# [EMAIL PROTECTED]:~# /usr/local/sbin/dbmail-imapd -V This is DBMail version 2.2.5-rc2-svn Copyright (C) 1999-2004 IC & S, [EMAIL PROTECTED] ... ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak
Aaron Stone wrote: > Ah, sorry, forgot to mention that it requires an autoreconf. How do I do that? I installed the autoconf and automake debian packages (sid) and I get this: [EMAIL PROTECTED]:~/devel/dbmail$ autoreconf /usr/share/aclocal/smpeg.m4:13: warning: underquoted definition of AM_PATH_SMPEG /usr/share/aclocal/smpeg.m4:13: run info '(automake)Extending aclocal' /usr/share/aclocal/smpeg.m4:13: or see http://sources.redhat.com/automake/automake.html#Extending-aclocal configure.in:40: warning: macro `AM_ENABLE_SHARED' not found in library configure.in:41: warning: macro `AM_ENABLE_STATIC' not found in library configure.in:40: error: possibly undefined macro: AM_ENABLE_SHARED If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. configure.in:41: error: possibly undefined macro: AM_ENABLE_STATIC configure.in:116: error: possibly undefined macro: AC_PROG_LIBTOOL autoreconf: /usr/bin/autoconf failed with exit status: 1 [EMAIL PROTECTED]:~/devel/dbmail$ If I just try to make I get the same error as Jorge (about DBMAIL_VERSION). Please help :) ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak
Leif Jackson wrote: > Peter Rabbitson wrote: >> Aaron Stone wrote: >> >>> First let's look at some debug logs and see what's going on. Maybe the >>> program is caught in a tight loop somewhere? If there's a particular >>> IMAP command string that immediately triggers the problem, that'd be >>> sort of the ideal situation, but not necessarily likely. >>> >>> Since we use Glib, these instructions for profiling GNOME applications >>> apply to us as well: >>> http://developer.gnome.org/doc/guides/optimisation/Massif.html >>> >>> Please ask any and all questions along the way, we're more than happy to >>> bring you up to speed on how to debug! >>> >>> Aaron >>> >>> >> >> Hi Aaron >> >> I played around with the latest rc2 build and my problem is still there >> (bug 584). I spent more time on it and did a fully logged run of >> valgrind as described in the url you gave me. Problem is I do not know >> what to do about the output I get. You can find a tar of all the >> information I could scramble at: >> http://rabbit.us/pool/misc/dbmail_bug584.tar.bz2 >> >> The file includes: >> dbmail_test - the script that made this happen >> dbmail.conf - my config file >> netstat.log - a netstat snapshot showing active connections/pids >> mem.log - a `ps aux` showing the fat process >> dbmail.err - a level 5 trace of the entire process >> the valgrind/massif output for all dbmail processes >> >> I understand that this is way too much information, but hopefully you >> can just glance at it and tell me right away what I should do next. >> Thanks for the help! >> >> >> > > Peter, > > If you run your tests with valgrind in memcheck mode using > --suppressions=dbmail.supp in the contrib directory I created a while > back. Use options > > valgrind --suppressions=contrib/dbmail.supp --tool=memcheck > --leak-check=full --show- > reachable=yes --trace-children=yes --log-file=/tmp/valtrace > the logfile directory can be any you choose it will make a file per pid > and log the leaks that arn't suppressed by the valgrind suppession file. > That should help us track this down futher. As arron said the options > that you specified may also be needed but the tool should be memcheck. > Thanks for helping track this down. > Here it is: http://rabbit.us/pool/misc/dbmail_bug584_memcheck.tar.bz2 The tar contains the same information as the previous one. None of the options I used before work - I guess they are specific to massif. Let me know what to do next. Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak
Aaron Stone wrote: > First let's look at some debug logs and see what's going on. Maybe the > program is caught in a tight loop somewhere? If there's a particular > IMAP command string that immediately triggers the problem, that'd be > sort of the ideal situation, but not necessarily likely. > > Since we use Glib, these instructions for profiling GNOME applications > apply to us as well: > http://developer.gnome.org/doc/guides/optimisation/Massif.html > > Please ask any and all questions along the way, we're more than happy to > bring you up to speed on how to debug! > > Aaron > Hi Aaron I played around with the latest rc2 build and my problem is still there (bug 584). I spent more time on it and did a fully logged run of valgrind as described in the url you gave me. Problem is I do not know what to do about the output I get. You can find a tar of all the information I could scramble at: http://rabbit.us/pool/misc/dbmail_bug584.tar.bz2 The file includes: dbmail_test - the script that made this happen dbmail.conf - my config file netstat.log - a netstat snapshot showing active connections/pids mem.log - a `ps aux` showing the fat process dbmail.err - a level 5 trace of the entire process the valgrind/massif output for all dbmail processes I understand that this is way too much information, but hopefully you can just glance at it and tell me right away what I should do next. Thanks for the help! Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] recipient enconding in TB sent folder
Paul J Stevens wrote: > Aaron Stone wrote: >> We issue only a "SET NAMES 'encoding'" without the COLLATION argument, >> so at the moment we fall onto whatever default the database has, but... >> >> Unless Paul changed any queries with the major utf8 cleanup today, the >> database collation doesn't matter because we don't ask the database to >> do any sorting on utf8 columns; all of the sorting takes place on the >> DBMail side in concert with GMime. > > No queries were changed. The cleanup wasn't that major actually. But the > collation does matter because in the IMAP SORT code we do rely on the database > for collation and sorting. It seems to me that's a good thing actually, > because > it allows us to let the database handle this issue. > > We should probably change the table definitions to use UTF8 encoding and > utf8_unicode_ci. DBAs can ofcourse change collation per table or per column as > desired. Dbmail doesn't really care at the moment. Erm... It does. It detects collation mismatches just as encoding mismatches. See the thread titled 'mysql collation for dbmail'. Or it might be that I am using an antiquated dbmail version? (the latest 2.2.5-0rc1 debian package) ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Defining tables
Niki Guldbrand wrote: > On Sun, 13 May 2007 22:11:56 +0200 > Peter Rabbitson <[EMAIL PROTECTED]> wrote: > >>> I was wondering if it possible to define the names of the tables used by >>> dbmail, like you can define the table prefix ? >> The configuration parameter 'table_prefix' in the '[DBMAIL]' section of >> the config file (/etc/dbmail/dbmail.conf) does just that. > > Err... Are you sure ? > So if i wanted to have a user table be called users, and at the same time > have eg. the acl table called dbmail_acl, i should use the table_prefix > setting ? > Whops, I read this as "... like _can you_ define the table prefix ?". Excuse the noise. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] Defining tables
Niki Guldbrand wrote: > Hi. > > I was wondering if it possible to define the names of the tables used by > dbmail, like you can define the table prefix ? > The configuration parameter 'table_prefix' in the '[DBMAIL]' section of the config file (/etc/dbmail/dbmail.conf) does just that. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] Correct name of latin encoding
Excuse the n00b question, but what is the correct name for the latin-1 encoding (want to set it for default_msg_encoding)? 'latin1' or 'latin-1' or are they aliases? Thanks ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] mysql collation for dbmail
Aaron Stone wrote: > On Sat, 2007-05-12 at 12:44 +0200, Peter Rabbitson wrote: >> Aaron Stone wrote: >>> On Sat, 2007-05-12 at 11:15 +0200, Peter Rabbitson wrote: >>>> Peter Rabbitson wrote: >>>>> In case that encoding=utf8, what collation does dbmail expect in the >>>>> mysql database? Does it matter if it is any of the case insensitive ones >>>>> or should it be the bin? Thanks >>>>> >>>> To answer my own question: I tried with utf8_bin, and got >>>> dbmysql.c,db_mysql_check_collations(+116): does >>>> [collation_database:utf8_bin] match [collation_connection:utf8_general_ci]? >>>> >>>> I guess the only supported collation is general_ci, since there is no >>>> corresponding setting in dbmail.conf. >>>> >>>> Excuse the noise :) >>> The corresponding setting in dbmail.conf is also utf8_bin ;-) >>> We just pass this value along to MySQL, it is not used internally. >>> >> Erm... so how is the dbmail.conf setting called then? There is >> 'encoding' for database encoding and for collation there is...? > > The MySQL query we issue is "SET NAMES " > > """ > Setting character_set_connection to x also sets collation_connection to > the default collation for x. It is not necessary to set that collation > explicitly. To specify a particular collation for the character sets, > use the optional COLLATE clause: > """ > > http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html > > I tried setting dbmail.conf: "encoding=utf8_bin" on my dev machine, and > MySQL reported that this was not a valid character set. Of course - it's > only a collation, not a character set. So it has to go in with the query > "SET NAMES 'utf8' COLLATE 'utf8_bin'". > > We'd need a new dbmail.conf entry to enable this. It's very simple to > add this, but I need to know, do we actually need it? > If the server side sorting relies on the RDBMS' `order by` then yes. Otherwise - not really. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] mysql collation for dbmail
Aaron Stone wrote: > On Sat, 2007-05-12 at 11:15 +0200, Peter Rabbitson wrote: >> Peter Rabbitson wrote: >>> In case that encoding=utf8, what collation does dbmail expect in the >>> mysql database? Does it matter if it is any of the case insensitive ones >>> or should it be the bin? Thanks >>> >> To answer my own question: I tried with utf8_bin, and got >> dbmysql.c,db_mysql_check_collations(+116): does >> [collation_database:utf8_bin] match [collation_connection:utf8_general_ci]? >> >> I guess the only supported collation is general_ci, since there is no >> corresponding setting in dbmail.conf. >> >> Excuse the noise :) > > The corresponding setting in dbmail.conf is also utf8_bin ;-) > We just pass this value along to MySQL, it is not used internally. > Erm... so how is the dbmail.conf setting called then? There is 'encoding' for database encoding and for collation there is...? ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] mysql collation for dbmail
Peter Rabbitson wrote: > In case that encoding=utf8, what collation does dbmail expect in the > mysql database? Does it matter if it is any of the case insensitive ones > or should it be the bin? Thanks > To answer my own question: I tried with utf8_bin, and got dbmysql.c,db_mysql_check_collations(+116): does [collation_database:utf8_bin] match [collation_connection:utf8_general_ci]? I guess the only supported collation is general_ci, since there is no corresponding setting in dbmail.conf. Excuse the noise :) ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] mysql collation for dbmail
In case that encoding=utf8, what collation does dbmail expect in the mysql database? Does it matter if it is any of the case insensitive ones or should it be the bin? Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak (on list)
Ouch, I forgot the list is too helpful by adjusting the reply-to :) Excuse the noise. ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak (off list)
Daniel Kasak wrote: > Peter Rabbitson wrote: > >> Hi, >> >> There is a raging memory leak in dbmail-imapd that manifests itself >> after about an hour of heavy usage (imapsync on huge mailboxes). Once >> the leak awakes it grows at about 200k per _second_. I have reproduced >> it numerous times during the last three days. What information/debuging >> output shall I provide to help you guys zap this one? >> > > Just a quick 'me too' posting. > > When I migrated from courier-imap to dbmail ( via imapsync ) I had what > could only be described as a major catastrophe when dbmail-imampd > consumed all the server's memory and the kernel started killing > processes, starting with mysqld! > > I have a cron job that restarts dbmail-imapd every 2 hours ( a little on > the paranoid side, but occasionally people do move massive amounts of > email into / out of shared folders ). You have to be *very* careful when > doing large amounts of mail handling, for example imapsync. > Hi, Sorry for contacting you off list, this is irrelevant to the thread. How did you work around this for the _sync_ part? I can not get all my mail imported since I am hitting the 2gb process limit :( ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak
Aaron Stone wrote: > First let's look at some debug logs and see what's going on. Maybe the > program is caught in a tight loop somewhere? If there's a particular > IMAP command string that immediately triggers the problem, that'd be > sort of the ideal situation, but not necessarily likely. > Hi Aaron, I filed bug 584 with a simple script that reproduces my case 100%. If it is not possible for you to run the script yourself, let me know and I will try my best to provide you with debug info obtained locally. This problem is a real show stopper for me, as the migration is a must. Thanks for any help/pointers. Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
Re: [Dbmail] dbmail-imapd memory leak
Aaron Stone wrote: > First let's look at some debug logs and see what's going on. Maybe the > program is caught in a tight loop somewhere? If there's a particular > IMAP command string that immediately triggers the problem, that'd be > sort of the ideal situation, but not necessarily likely. Excellent. I wanted to make sure I have someone to work with before I started digging into it. Later today or tomorrow I will hack together something using Mail::IMAPClient, so I can reproduce the excercise. > Since we use Glib, these instructions for profiling GNOME applications > apply to us as well: > http://developer.gnome.org/doc/guides/optimisation/Massif.html I see. I will give it a shot, will let you know if I can't get the hang of it. Thanks for the pointers! Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
[Dbmail] dbmail-imapd memory leak
Hi, There is a raging memory leak in dbmail-imapd that manifests itself after about an hour of heavy usage (imapsync on huge mailboxes). Once the leak awakes it grows at about 200k per _second_. I have reproduced it numerous times during the last three days. What information/debuging output shall I provide to help you guys zap this one? Thanks Peter ___ DBmail mailing list DBmail@dbmail.org https://mailman.fastxs.nl/mailman/listinfo/dbmail
