Re: Deleting unused or dormant mail accounts.
> "BG" == Bron Gondwana writes:
BG> yes, it should work with 2.4.
Good news.
Since folks may not know much about this, here's some detail.
I've been using this little Perl script to talk to the fud server for so
long that I've no recollection of where it came from. It's only 53
lines, so I'll just include it. And of course you have to add the
following to cyrus.conf:
fud cmd="fud" listen="fud" proto="udp" prefork=1 maxchild=10
and give the proper (weird) permission so that fud actually works:
sam user/* anonymous 0
The script follows. Output looks like this:
> fud-client tibbs
|tibbs|Inbox|0|1315866483|1315866228|
user: tibbs
mbox: Inbox
Number of Recent: 0
Last read: Mon Sep 12 17:28:03 2011
Last arrived: Mon Sep 12 17:23:48 2011
but of course you could hack it up to do whatever you wanted.
- J<
#!/usr/bin/perl -w
# perl "port" of contrib/fud-client.c that comes with the Cyrus IMAP
# distribution (http://asg.web.cmu.edu/)
# Written by Jason Englander
#
# Released under the GNU GPL. See http://www.gnu.org/copyleft/gpl.html
# for more information.
#
# 0.1 - July 3, 2002
# [jme] - Initial release
use Socket;
use strict;
my ($hisiaddr, $hispaddr, $iaddr, $paddr, $port, $proto, $resp);
my $server = 'your.imap.server.here';
my $user = $ARGV[0];
my $mailbox = $ARGV[1] || 'Inbox';
die "\nUsage: $0 user mailbox\n\n" if ! $server || ! $user || ! $mailbox;
die "\nUsage: $0 server user mailbox\n\n" if ! $server || ! $user || ! $mailbox;
$proto = getprotobyname('udp');
$port = getservbyname('fud', 'udp');
socket(SOCKET, PF_INET, SOCK_DGRAM, $proto) || die "socket: $!";
$| = 1;
$hisiaddr = inet_aton($server) || die "unknown host";
$hispaddr = sockaddr_in($port, $hisiaddr);
defined(send(SOCKET, "$user|$mailbox", 0, $hispaddr)) || die "send $server: $!";
recv(SOCKET, $resp, 512, 0) || die "recv: $!";
close(SOCKET);
my $resp_str = unpack('A*',$resp);
print "|$resp_str|\n";
if ("$resp_str" eq "UNKNOWN") {
printf "Server did not recognize mailbox %s\n", $mailbox;
}
elsif ("$resp_str" eq "PERMDENY") {
printf "Permission denied attempting get mailbox info for %s\n",
$mailbox;
}
else {
my ($uname,$mbox,$numrecent,$lread,$lappend) = split('\|',$resp_str);
printf "user: %s\nmbox: %s\nNumber of Recent: %d\n", $uname, $mbox,
$numrecent;
$lread = localtime($lread);
printf "Last read: %s\n", $lread;
$lappend = localtime($lappend);
printf "Last arrived: %s\n", $lappend;
}
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: expunge_days & cyr_expire best practice
On Mon, Sep 12, 2011 at 04:29:47PM +0200, Wolfgang Breyha wrote: > Hi! > > I'm currently preparing our "murder" migration from 2.3 to 2.4. Since we > already use delayed expunge and delete I try to fully understand the impact of > expunge_days: setting. Currently we use -X 2 with cyr_expire. Fair enough. > Since QRESYNC needs at least "expunge_days: 7" as stated in "man imapd.conf" I > tried to figure out how to set it up right. Nah, it doesn't really. It can cope fine with instant expunge. It's just a bit less efficient if it can't guarantee that it knows about all the expunges since the last request, so it falls back to telling about EVERY gap in the UID sequence, rather than only telling about known removals. If you're comfortable reading the code (which it looks like you are!) you can see the algorithm working with highestmodseq and deletedmoseq. Deletedmodseq is the HIGHEST modseq of any record that has ever been removed from the index. If you request changes since a modseq after that, you can guarantee that all the required expunge information is available, so you can give efficient responses. This also applies to replication, if you're using it. Replication can be much more efficient if expunge data is available, because it can send a full list of changes rather than having to sync the whole mailbox. > If I read the source right only mailbox.c:mailbox_close() and cyr_expire do > cleanup jobs on cyrus cache and unlink files. Yes, that's correct. expunge_days is a nasty workaround for the fact that otherwise mailboxes would just grow unbounded if people didn't run a cyr_expire. The default is now expunge_mode: default, which deletes the message, but keeps the index metadata. > mailbox_close() waits until expunge_days+8 after the index record was expired. > What are these 8 days for? Is this the minimum needed for QRESYNC mentioned in > "man imapd.conf"? The basic idea is that if you run cyr_expire regularly enough, then you never get a random other process having to do the expensive expunge task. On the other hand, if you FAIL to run cyr_expire, your mess still gets cleaned up eventually. Again, bogus heuristic. I can only claim I wasn't thinking super clearly when I wrote that section of the logic. It's more "clever" than "good". > cyr_expire doesn't care about expunge_days at all. So I guess -X should be at > least >= expunge_days, right? No, -X should be the same or less than expunge_days, then you get to control when you expunge. > Is > expunge_days: 7 > and > cyr_expire -X 7 > ok so far? That's what we run, and it keeps us nice and happy. We also only expunge on Saturdays, which keeps the IO load down during the week. Bron ( and we run checkreplication on Sundays - so the weekends are actually _higher_ IO load than the weeks... ) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Deleting unused or dormant mail accounts.
On Sun, Sep 11, 2011 at 11:57:00PM -0300, Andres Tarallo wrote: > Hi !!! > > I'm administering a mail server with a few thousand mail accounts. Due > to the way this organization works we have in this server some > accounts that aren't checked. > > In order to get a list of mailboxes to be removed we're looking the > date the file username.seen was last modified. Do we have a better > strategy? Just a warning, that definitely won't work reliably in 2.4.x, because we don't use the .seen file anywhere near as much. We update a "last logged in" timestamp in our database through a custom saslauthd, but of course that may not work for you. What would work though, is just tailing your syslog file for IMAP or POP login events and tracking every username that logs in for a few weeks, or whatever period you want to track, and then start deleting accounts. Assuming you don't have to do it RIGHT NOW. Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: deliver.db to big size.
Ok. thx for the aid. I`m waiting is the last questión over deliver.db, and sorry for my poor english. When finish the cyr_expire command, how has long the file size? I suppose the size of the file is delimited by the flag -E. If the -E flags is the number of day recording, then the size file is determinate by the traffic on the days stored and the numbre of this days stored? Thx for the aid. 2011/9/12 Adam Tauno Williams > On Mon, 2011-09-12 at 12:40 +0200, Manuel Vazquez wrote: > > Thx for the information. This solution is similar to: ctl_deliver -d > > > flat_text_file? > > Sure, ctl_deliver is just a command specifically for the delivery > database. > > > Is not possible do query with other tools? > > Depends on the format of the file. Is the delivery database Berkley, > flat, or skiplist? [the "file" command can probably identify the type if > it isn't explicitly declared in imapd.conf). > > On the other hand - what are you trying to do? There isn't really > anything all that interesting in delivery.db. > > > Other question i execute the next command: cyr_expire -E 3 -X 2 -D > > 180. The documentation say this command deleted all the registrer more > > old of 180 days. But this deleted the file stored to delete in the > > trash of mailbox? > > Doesn't have anything to do with "trash"; trash is a [IMNSHO] *very* > misguided MUA (Mail User Agent / Client) "feature". cyr_expire deletes > from the server messages and/or mailboxes that have been > expunged/deleted (for messages) or deleted (for mailboxes). > > Only -E has to do with delivery.db > > > Prune the duplicate database of entries older than expire-days. This > value is only used for entries which do not have a > corresponding /vendor/cmu/cyrus-imapd/expire mailbox annotation.. > > > -X and -D only mean anything if you have delayed delete enabled (which > has nothing to do with delivery.db). > > You can add the "-v" option if you want to see more info about what > cyr_expire is doing. > > > > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > -- Manuel Vázquez. Técnico Informático. manuel.vazquez.gonza...@hotmail.com titocer...@gmail.com Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
expunge_days & cyr_expire best practice
Hi! I'm currently preparing our "murder" migration from 2.3 to 2.4. Since we already use delayed expunge and delete I try to fully understand the impact of expunge_days: setting. Currently we use -X 2 with cyr_expire. Since QRESYNC needs at least "expunge_days: 7" as stated in "man imapd.conf" I tried to figure out how to set it up right. If I read the source right only mailbox.c:mailbox_close() and cyr_expire do cleanup jobs on cyrus cache and unlink files. mailbox_close() waits until expunge_days+8 after the index record was expired. What are these 8 days for? Is this the minimum needed for QRESYNC mentioned in "man imapd.conf"? cyr_expire doesn't care about expunge_days at all. So I guess -X should be at least >= expunge_days, right? Is expunge_days: 7 and cyr_expire -X 7 ok so far? Greetings, Wolfgang -- Wolfgang Breyha | http://www.blafasel.at/ Vienna University Computer Center | Austria Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Deleting unused or dormant mail accounts.
Andres Tarallo wrote: > In order to get a list of mailboxes to be removed we're looking the > date the file username.seen was last modified. Do we have a better > strategy? Judging by the last delivery date is compromised by spam. Getting mail unfortunately does not mean the account is being used. Judging by seen state is much better provided you can assume IMAP. I don't think POP updates seen. But POP devices tend to check in regularly, so you could cover obvious cases by checking the last couple of weeks of system log. It won't be perfect. Joseph Brennan Columbia University Information Technology Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: deliver.db to big size.
On Mon, 2011-09-12 at 12:40 +0200, Manuel Vazquez wrote: > Thx for the information. This solution is similar to: ctl_deliver -d > > flat_text_file? Sure, ctl_deliver is just a command specifically for the delivery database. > Is not possible do query with other tools? Depends on the format of the file. Is the delivery database Berkley, flat, or skiplist? [the "file" command can probably identify the type if it isn't explicitly declared in imapd.conf). On the other hand - what are you trying to do? There isn't really anything all that interesting in delivery.db. > Other question i execute the next command: cyr_expire -E 3 -X 2 -D > 180. The documentation say this command deleted all the registrer more > old of 180 days. But this deleted the file stored to delete in the > trash of mailbox? Doesn't have anything to do with "trash"; trash is a [IMNSHO] *very* misguided MUA (Mail User Agent / Client) "feature". cyr_expire deletes from the server messages and/or mailboxes that have been expunged/deleted (for messages) or deleted (for mailboxes). Only -E has to do with delivery.db Prune the duplicate database of entries older than expire-days. This value is only used for entries which do not have a corresponding /vendor/cmu/cyrus-imapd/expire mailbox annotation.. -X and -D only mean anything if you have delayed delete enabled (which has nothing to do with delivery.db). You can add the "-v" option if you want to see more info about what cyr_expire is doing. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: deliver.db to big size.
Sorry i want say:
But this delete the files stored in the trash subfolder of the mailbox?
Thx again.
2011/9/12 Manuel Vazquez
> Thx for the information. This solution is similar to: ctl_deliver -d >
> flat_text_file? Is not possible do query with other tools?
>
> I detect, when say in other mail before, the size of the file is too big,
> but i don`t know the register more old stored in this file.
>
> Other question i execute the next command: cyr_expire -E 3 -X 2 -D 180. The
> documentation say this command deleted all the registrer more old of 180
> days. But this deleted the file stored to delete in the trash of mailbox?
>
> Thx for all the aid.
>
> 2011/9/12 Adam Tauno Williams
>
>> On Mon, 2011-09-12 at 12:09 +0200, Manuel Vazquez wrote:
>> > Thx for aid me to solved my question. But i am need query the
>> > information stored in the deliver.db. How do this? With cyradm? O
>> > exists other tool more for query the information saved in deliver.db?
>>
>> You can convert the database to a text file with cvt_cyrusdb -
>>
>> cvy_cyrusdb {souce file} {source format} {destination file} {destination
>> format}.
>>
>> /usr/lib/cyrus-imapd/cvt_cyrusdb /var/lib/imap/deliver.db
>> skiplist /tmp/deliver.txt flat
>>
>> There is also a tool named cyr_dbtool, but documentation/examples are
>> thin and you really need to know the structure of the database in
>> question.
>>
>>
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>>
>
>
>
> --
> Manuel Vázquez.
> Técnico Informático.
> manuel.vazquez.gonza...@hotmail.com
> titocer...@gmail.com
>
>
--
Manuel Vázquez.
Técnico Informático.
manuel.vazquez.gonza...@hotmail.com
titocer...@gmail.com
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: deliver.db to big size.
Thx for the information. This solution is similar to: ctl_deliver -d >
flat_text_file? Is not possible do query with other tools?
I detect, when say in other mail before, the size of the file is too big,
but i don`t know the register more old stored in this file.
Other question i execute the next command: cyr_expire -E 3 -X 2 -D 180. The
documentation say this command deleted all the registrer more old of 180
days. But this deleted the file stored to delete in the trash of mailbox?
Thx for all the aid.
2011/9/12 Adam Tauno Williams
> On Mon, 2011-09-12 at 12:09 +0200, Manuel Vazquez wrote:
> > Thx for aid me to solved my question. But i am need query the
> > information stored in the deliver.db. How do this? With cyradm? O
> > exists other tool more for query the information saved in deliver.db?
>
> You can convert the database to a text file with cvt_cyrusdb -
>
> cvy_cyrusdb {souce file} {source format} {destination file} {destination
> format}.
>
> /usr/lib/cyrus-imapd/cvt_cyrusdb /var/lib/imap/deliver.db
> skiplist /tmp/deliver.txt flat
>
> There is also a tool named cyr_dbtool, but documentation/examples are
> thin and you really need to know the structure of the database in
> question.
>
>
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>
--
Manuel Vázquez.
Técnico Informático.
manuel.vazquez.gonza...@hotmail.com
titocer...@gmail.com
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: deliver.db to big size.
On Mon, 2011-09-12 at 12:09 +0200, Manuel Vazquez wrote:
> Thx for aid me to solved my question. But i am need query the
> information stored in the deliver.db. How do this? With cyradm? O
> exists other tool more for query the information saved in deliver.db?
You can convert the database to a text file with cvt_cyrusdb -
cvy_cyrusdb {souce file} {source format} {destination file} {destination
format}.
/usr/lib/cyrus-imapd/cvt_cyrusdb /var/lib/imap/deliver.db
skiplist /tmp/deliver.txt flat
There is also a tool named cyr_dbtool, but documentation/examples are
thin and you really need to know the structure of the database in
question.
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: deliver.db to big size.
Thx for aid me to solved my question. But i am need query the information
stored in the deliver.db. How do this? With cyradm? O exists other tool more
for query the information saved in deliver.db?
Thanks for your aid.
2011/9/11 Bron Gondwana
> On Sun, Sep 11, 2011 at 01:34:12PM +0200, Manuel Vazquez wrote:
> > Hi, i'm newbie in the cyrus world. I am admin junior of 1 system of
> > corporate mail. In 1 instance of de cyrus imapd i detect the deliver.db
> file
> > have to big size (more de 5GB). I two question over this file.
> > 1. This file have the traffic info of mailbox user?
>
> Are you running cyr_expire daily, like this? (it needs to
> be in cyrus.conf)
>
> EVENTS {
>checkpoint cmd="ctl_cyrusdb -c" period=30
>delprune cmd="cyr_expire -E 3" at=0400
>tlsprune cmd="tls_prune" at=0400
> }
>
> If you need to change your config to add cyr_expire, it is safe
> to shutdown Cyrus and delete the deliver.db, then start it up
> again. We delete deliver.db on our systems every time we shut
> down Cyrus.
>
> > 2. I'm problem with the space on the filesystem on cyrus imapd run. Could
> i
> > move the file deliver.db to other filesystem and soft link to this for
> not
> > lost the information?
>
> No, sorry. If you are running Cyrus 2.4.9 or later you can move
> the deliver.db by setting the config variable duplicate_db_path.
> Otherwise softlinks won't work, so you have no option with earlier
> Cyrus versions.
>
> Also, skiplist files are limited to 4Gb in size. There is an
> open bug for this.
>
> > Thx, and sorry for my poor english.
>
> I assume you speak Spanish? Hopefully there are other Spanish
> speakers on the list who can help you as well!
>
> Regards,
>
> Bron.
>
--
Manuel Vázquez.
Técnico Informático.
manuel.vazquez.gonza...@hotmail.com
titocer...@gmail.com
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
