Hello all, it would be better if I write more info about my project.
Postfix is great mail server software with clean design and lot of configuration options. But with its standard features, it is not suitable for envireonments with many different users (for example webhosting company). There are no features/standard tools of commercials smtp software. For example: - adaptive throttling (users), - warming up of new ip address, - user segmentation (good, spammers, sametime spammers...), - delivery reporting all this features is "must have" when you want (need) to maintain reputation for your IP space. All this features you can find on for example PowerMTA (port25.com) or MailChannels (which segments users based on DSN of emails - take a quick look@ http://learn.mailchannels.com/responseanalytics-briefing/). Wietse in last email ask me "How much delay can you tolerate?". The answer is: no delay :) Reason is, that only single email to tarpit address can get IP address to blacklist. Only single email can hurt our custommer service with tons of custommer complains about deliveriability. My colleague Richard, until today implement the milter and dovecot auth module for collecting statistics based on user login and type of transactions. For every user we currently have following info: number of good logins, list of uniq IP addresses from which each user login, complete login log (good, wrong attemps), number of transactions (smtp transaction is from first MAIL FROM to last dot after DATA) number of internal and external recipients (this is for throttling like https://support.google.com/a/answer/166852?hl=en) list of uniq recipients all this data is in 15m/60m/24h periods with some history in redis DB. Based on this data, we can block user for example when: - we see, user send lots of emails (more than median for last two weeks), - we see, that user logs from IP address from "too many countries", - Richard also implements some behavioral heuristics, he evaluates user's habits (this user does not send email after 8PM for example), - we have also mobile app with tracks user location, this is technology uses also Mastercard - if you have installed Mastercard APP in your phone, Mastercard knows your location. Then when you use your credit card on POS which is on the other side of the globe ... Mastercard denies payment. We have similar logic in our platform. Last but not least, we want to make some response analytics. Which I hope, will helps a bit to maintain our IP addresses clean and we can block infected user and/or spammer at soon as possible. This is why we want to track result codes from sent emails. Ok, and now back to source code. Thank you Wietse for pointing me to MSG_OUTPUT(3). I have found may be better place. Please: - we well track each SMTP transaction in milter, so we can later find based on MESSAGE QUEUE ID following info: - SASL user name (which identifies our user), - number of recipients, - list of recipients Later, we can hook log_adhoc(3): void log_adhoc(const char *id, MSG_STATS *stats, RECIPIENT *recipient, const char *relay, DSN *dsn, const char *status) we have all info we need, msg id, recipient and status (with dsn). The only problem is, when we hook this function, we can not get "final status" of delivery. My be I'm wrong, but this seems to be better, because we do not need to parse "log entry", we have struct. On Tue, Jan 27, 2015 at 6:54 PM, Wietse Venema <wie...@porcupine.org> wrote: > Peter Hodur: > > On Tuesday, January 27, 2015, Wietse Venema <wie...@porcupine.org> > wrote: > > > > > > > > > This is typically done with "enable_long_queue_ids = yes", a program > > > that watches the maillog file, and a collection of regular expressions > > > that extract information. > > > > > > > > Thank you for tour reply. I know this option and i use it. But i do not > > want to parse log file. This will introduce some delay in processing and > > decision based on logged data. (For example, i want to monitor 5xx > response > > How much delay can you tolerate? > > > ratio on individual sasl user names and block users in realtime - many > 5xx > > errors is obviously indicator, that user send spam messages, so i can > route > > it to another smtp client process with dedicate ip and/or block it) > > > > Instead i want to hook some function in postfix (may be function which > > flushes records to log file?) and add couple of lines of code that > connect > > to redis database and puts there some statistics. > > This still requires a collection of regexps to extract information, > so all you gain is a few milliseconds. Is that worth it? > > > So i ask here, if someone with knowledge of postfix internals can tell > me, > > what is the right place to add this code. What "function" to hook... I > have > > read som source code of postfix and im not sure im able to identify right > > place to add my logging code) > > You can register a logging function with msg_output(). > > Wietse > > MSG_OUTPUT(3) > MSG_OUTPUT(3) > > NAME > msg_output - diagnostics output management > > SYNOPSIS > #include <msg_output.h> > > typedef void (*MSG_OUTPUT_FN)(int level, char *text) > > void msg_output(output_fn) > MSG_OUTPUT_FN output_fn; > ... > DESCRIPTION > This module implements low-level output management for the msg(3) > diag- > nostics interface. > > msg_output() registers an output handler for the diagnostics > interface. > An application can register multiple output handlers. Output > handlers > are called in the specified order. An output handler takes as > argu- > ments a severity level (MSG_INFO, MSG_WARN, MSG_ERROR, > MSG_FATAL, > MSG_PANIC, monotonically increasing integer values ranging from > 0 to > MSG_LAST) and pre-formatted, sanitized, text in the form of a > null-ter- > minated string. > > > Wietse >
