Version 2.6.4 of amavisd-new has been released, it is available at: http://www.ijs.si/software/amavisd/amavisd-new-2.6.4.tar.gz
release notes: http://www.ijs.si/software/amavisd/release-notes.txt The 2.6.4 is mainly a maintenance release over 2.6.3, but also brings a couple of new features. ================ NOTE: When upgrading Perl to version 5.10 or planning to do so, please do not forget to add a missing /m flag to regular expressions in your existing AV entries (if you haven't already done so with a 2.6.3 upgrade), as suggested in an example file amavisd.conf in a package. Perl 5.8 does not mind missing /m flags, but with perl 5.10 the results from a virus scanner may no longer be properly recognized. See the BUG FIXES section in 2.6.3 release notes. ================ COMPATIBILITY WITH 2.6.3 The output of amavisd-agent and contents of a database snmp.db has changed according to the now published MIB. Several new SNMP counters were added, a few retired, and some renamed. If you are parsing the output of amavisd-agent or accessing snmp.db directly, please review AMAVIS-MIB.txt, and perhaps switch to using the new amavisd-snmp-subagent. BUG FIXES - amavisd failed to start when spam scanning was disabled either by @bypass_spam_checks_maps=(1) or by @spam_scanners=(), giving: Can't locate object method "new" via package "Amavis::SpamControl" As a workaround one could use a @spam_scanners=(undef) to disable spam scanning; reported by Steve; - several decoders failed to propagate "Exceeded storage quota" exception, so the protection of AV scanners against mail bombs was ineffective; reported by Jorgen Lundman; - milter usage (AM.PDP): verbatim header edits inserted a header body of "1" instead of the correct string, for example: "Authentication-Results: 1"; - updated AV entry for BitDefender's bdscan to recognize tabs around a colon in its output; contributed by Steve; - fix parsing of a combined result from DSPAM (option --classify), as earlier versions of DSPAM did not include a signature with a combined result line; problem reported by Marijan Vidmar; NEW FEATURES SUMMARY - provide a true SNMP agent and a MIB, facilitating monitoring the health of a content filtering system, its performance and mail characteristics; - a new AV interface to SMTP-based antivirus scanners; - allow customizing SMTP-status response reason text for blocked messages; - prevent inserting fake copies of certain important mail header fields without breaking a DKIM signature; NEW FEATURES - newly supplied with the package is a program amavisd-snmp-subagent, acting as a SNMP AgentX, exporting amavisd statistical counters database (snmp.db) as well as a child process status database (nanny.db) to a SNMP daemon supporting the AgentX protocol (RFC 2741), such a NET-SNMP. It is similar to combined existing utility programs amavisd-agent and amavisd-nanny, but instead of writing results as text to stdout, it exports data to a SNMP server running on a host (same or remote), making them available to SNMP clients (such a Cacti or mrtg) for monitoring or alerting purposes. The amavisd program does not have any additional requirements, but to run amavisd-snmp-subagent the following Perl modules are required: NetSNMP::OID, NetSNMP::ASN, NetSNMP::agent, NetSNMP::default_store. All of these come with a Net-SNMP package (previously known as "ucd-snmp"), home at http://net-snmp.sourceforge.net/, FreeBSD ports: net-mgmt/net-snmp. Also, a snmpd daemon must be running on a host. It can be a snmpd from a Net-SNMP package or some other SNMP server supporting AgentX protocol. When using snmpd from Net-SNMP, just add the following to its snmpd.conf: master agentx agentXSocket tcp:127.0.0.1:705 so that amavisd-snmp-subagent will be allowed to connect to it. The setup was tested with Net-SNMP versions 5.4.2.1 and 5.3.2.3. If you experience wild numbers served in Counter64 variables on a 64-bit platform, the following patch (at the server side) solves the problem: http://www.mail-archive.com/ net-snmp-us...@lists.sourceforge.net/msg19502.html The patch seems to already be incorporated into version 5.3.3 of Net-SNMP, but not yet in 5.4.2. A MIB module (SNMP Management information base) is provided in a file AMAVIS-MIB.txt. It is not necessary to make it available to a SNMP server, and not even necessary for SNMP clients, but making it available to clients allows them to display data with names of variables, not just their OIDs. A query example with no MIB modules: snmpbulkwalk -v2c -c xxx host.example.com .1.3.6.1.4.1.15312.2.1 A query example when a file AMAVIS-MIB.txt is in a subdirectory ./mibs/ : snmpbulkwalk -m+AMAVIS-MIB -M-mibs -OQ -v2c -c xxx host.example.com amavis The amavisd-snmp-subagent can be started at any time, either before or after amavisd, and either before or after snmpd. It can also be restarted at any time. Also, amavisd can be restarted without having to restart amavisd-snmp-subagent, as it will automatically notice a database change and connect to a new database. Similarly, a snmpd daemon can be restarted at any time and amavisd-snmp-subagent will reconnect to it if necessary. A natural starting order is: snmpd first, then amavisd and then amavisd-snmp-subagent. Restarting amavisd will reset its counters. A SNMP client typically interprets a decremented value of a counter variable as a wraparound, which results in a large spike when graphing data. There are two common solutions to the problem: a reasonable upper limit can be provided to a client, so that a spike will be treated as invalid data and ignored, or else a AMAVIS-MIB::sysUpTime variable can be monitored, and if its value is smaller than on a previous reading, this indicates that counters were reset (i.e. amavisd was restarted) and values of counters should not be treated as wrapped on maxint. Consult your SNMP client documentation. The amavisd-snmp-subagent should have access to databases snmp,db and nanny.db in a $db_home directory (environment variable AMAVISD_DB_HOME, defaults to /var/amavis/db) and have rights to connect to a snmpd daemon. It is safe to run it as root, although perhaps not necessary. For testing purposes start amavisd-snmp-subagent from a command line using a command line option -f to let it stay in foreground, and optionally increase debug level, e.g: amavisd-snmp-subagent -f -d 5 If everything goes well, start it without -f and let it daemonize. Supplying a filename with an option -P tells a daemonized agent to write its PID to that file, and remove the file on shutdown (on receiving a signal TERM or INT): amavisd-snmp-subagent -P /var/run/amavisd-snmp-subagent.pid Some suggested sets of OIDs making up interesting diagrams (e.g. for displaying by Cacti): counters: * inMsgsStatusRelayed, inMsgsStatusDiscarded, inMsgsStatusNoBounce, inMsgsStatusBounced, inMsgsStatusRejected * inMsgs, inMsgsOriginating * inMsgsSize, inMsgsSizeOriginating * inMsgsSize, outMsgsSizeSubmitQuar, outMsgsSizeRelay * inMsgs, outMsgsRelay, outMsgsSubmitQuar, outMsgsSubmitDsn, outMsgsSubmitNotif * contentCleanMsgs, contentCleanMsgsOriginating * inMsgs, contentSpamMsgs, contentBannedMsgs, contentVirusMsgs * contentSpamMsgsOriginating, contentBannedMsgsOriginating, contentVirusMsgsOriginating * timeElapsedTotal, timeElapsedDecoding, timeElapsedVirusCheck, timeElapsedSpamCheck * procGone gauges: * procBusy, procAll * procBusy, procBusyTransfer, procBusyDecode, procBusyVirus, procBusySpam * procBusy0, procBusy1s, procBusy2s, procBusy4s, procBusy8s * procBusy15s, procBusy30s, procBusy1m, procBusy2m, procBusy4m * mtaQueueEntriesIncoming, mtaQueueEntriesActive, mtaQueueEntriesDeferred Note that even frequent or extensive SNMP queries do not burden amavisd processes. The amavisd-snmp-subagent process keeps a cache of current variable values. It queries one or the other berkeley database as needed, i.e. when cached data is stale and there was an actual SNMP query for a variable in one or the other database. When a berkeley database needs to be accessed, all its data is fetched in one quick sweep by using a database cursor with a read lock, so that data is consistent. No more than one database sweep in 4 seconds is performed, and less often when queries are less frequent and preferably batched in groups. If some time has passed since the last SNMP query (more than 4 seconds currently), resulting values are always fresh as collected from a database at the time of a SNMP query. There is one additional experimental feature - experimental in a sense that it may change or be dropped in future versions. If running Postfix on the same host as amavisd-snmp-subagent, a count of files (mail messages) in each of the Postfix queue directories is provided as Gauge32 variables in the MIB under .1.3.6.1.4.1.15312.2.1.3, i.e. under amavisMta subtree. The following SNMP variables are available: mtaQueueEntriesMaildrop, mtaQueueEntriesIncoming, mtaQueueEntriesActive, mtaQueueEntriesDeferred. Although semantically outside the scope of amavisd, it provides a quick insight into health of a MTA, and indirectly into health of amavisd. Data is made available only if a command 'postconf -h queue_directory' is successful at amavisd-snmp-subagent startup time and provides a sensible result. Like with the other two real databases, MTA directories are only scanned if and when actually queried by a SNMP client (again, subject to caching). As a safety measure for times when MTA queue grows huge, there is a time limit for scanning each directory subtree (currently 5 seconds, which is about how much a typical SNMP client is willing to wait for a response). Also, a long scan time automatically increases cache validity time (time-to-live) of that measurement. - a new experimental interface to SMTP-based antivirus scanners is provided; an @av_scanners entry may look like the following: ['av_smtp', \&ask_av_smtp, ['{}', 'smtp:[127.0.0.1]:5525', 'du...@localhost'], qr/^2/, qr/^5/, qr/FOUND:\s*(.*?)\s*$/m ], The ask_av_smtp mechanism connects to a virus scanner using the specified protocol (typically SMTP or LMTP) on a given IP address and a port number, considering the virus scanner as an ordinary MTA. The full original message is then fed to the scanner (currently ignoring the "{}" argument), using the original envelope sender address and a given address as a single recipient (defaults to 'du...@localhost'). It is expected that a virus scanner will accept a clean message (2xx) and reject an infected message (status 5xx). A SMTP response is parsed as usual for any output from a virus scanner, typically considering a response starting with 2 as clean, a response starting by 5 as infected, and anything else as a scanner failure. The SMTP-based virus scanner should be configured not to deliver a message. This may be achieved by feeding its SMTP output to a dummy SMTP listener, such as smtp-sink as supplied by a Postfix package. It is not a particularly efficient interfacing mechanism, but some virus scanners do not provide a choice. Prompted by Kevin M. Myer; - a new configuration variable %smtp_reason_by_ccat allows customizing SMTP-status response reason text. The reason strings are subject to macro expansion, so built-in macros are available (README.customize). Multi-line texts are allowed and produce a valid multi-line SMTP response, but use it sparingly, as some nonstandard mailers may not like it. Currently %smtp_reason_by_ccat is only consulted for blocked messages, a passed clean message still uses a hardwired reason text; suggested by Ralf Hildebrandt, based on a patch by Noah Baker; - support inclusion of null header field names in an 'h' tag of a DKIM signature generated by amavisd for specified header field names, thus preventing third parties from prepending additional occurrences of these header fields without breaking a signature; useful for example for protecting a recipient's mail reader or a filter from being tricked by supplying a duplicate From or Subject header fields; the protection is requested by specifying a value larger than 1 in %signed_header_fields, e.g.: $signed_header_fields{'from'} = 2; $signed_header_fields{'subject'} = 2; $signed_header_fields{'message-id'} = 2; $signed_header_fields{'content-type'} = 2; Please restrict values used in %signed_header_fields to 0, 1, or 2, consider other values reserved for future compatibility. By default the following header fields are protected from duplicates by a DKIM signature generated by amavisd: From, Date, Subject, Content-Type. To revert to a classical behaviour, set their value in %signed_header_fields to 1, e.g.: $signed_header_fields{lc($_)} = 1 for qw(From Date Subject Content-Type); - add a config variable @spam_notifyadmin_cutoff_level_maps, which allows suppressing of spam administrator notifications when spam score exceeds a level resulting from a lookup into this list of lookup tables; suggested by Rudy Gevaert; - new configuration variables: $snmp_contact, $snmp_location, empty strings by default; these end up in a MIB as exported by the new amavis SNMP agent; OTHER - failure of a file(1) utility is now only logged (at syslog level LOG_ERR) and no longer treated as a fatal error; suggested by Matija Grabnar; - require a minimal version 2.017 of Compress::Raw::Zlib when unpacking a zip archive (by Archive::Zip) to avoid an archive (e.g. in an infected mail) from causing amavisd process to hang; thanks to Alexander 'Leo' Bergolth for troubleshooting the memory allocation issue in a zip library; - amavisd-nanny: write notes about lost processes to STDERR instead of to STDOUT, making it easier to use it non-interactively, e.g. from cron; suppress printing of "exited" when not interrupted; based on a patch provided by Thomas Gelf; - amavisd-agent: suppress printing of "exited" when not interrupted; - amavisd-agent: updated according to changes in MIB; - reduce a log level of a virus scanner failure from -2 to -1, unless all virus scanners have failed; - added to @virus_name_to_spam_score_maps : /^Structured\.(SSN|CreditCardNumber)\b/ /^Sanesecurity.TestSig_/ /^Email\.Spammail\b/ /^winnow\.(phish|spam)\./ /^INetMsg\.SpamDomain/ - internal: added user_policy_id and user_policy_id to recipient data in object Amavis::In::Message::PerRecip, facilitating storing users.policy_id into msgrcpt.sql_policy_id; suggested by Stefan Palme; - amavisd.conf-sample: updated comment explaining the use of %a and %k expansions in $sql_select_policy; thanks to Max-Julian Pogner; - documentation: change all "author signature" to "author domain signature" according to draft-ietf-dkim-ssp-10; Mark ------------------------------------------------------------------------------ _______________________________________________ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
