Hello, Murray, > My understanding is that the general practice of an FBL is > that they are not typically established in an automated > fashion. That may change a little as we do the > reporting discovery work, but at the moment I don't think > there are any parties that just start sending or accepting > FBLs and assuming they're all valid. Thus, there's > already an out-of-band component to establishing an FBL, and > it makes sense to prearrange a list of authorized IP > addresses in that process rather than in the base protocol. > ... > You cited RFC3834 in previous email as your basis for the > "RFCs" claim above. I disagree that it applies to > FBLs, which are typically activated by humans and not by > software to report spam, and thus aren't automated > messages.
I see the autoresponders fully involved here - MARF is actually an ideal tool to be used by spam filters, virus scanners and the like, long term they'll certainly take advantage of the possibilities of MARF. Of course, MARF also has its use in Outlook or similiar Mail Software where a user can push a button to mark a mail spam, or not spam, and the sending ISP (and/or filter provider) automatically gets informed upon that user intervention. A fully manual MARF report would seem over the top however. We certainly do spam and malware reporting fully automated. A brief look into history: In 2002, at the height of the first large virus outbreaks, we decided our mailservers would all get virus scanners (long before any of the large ISPs even began to think about it), and we wrote our first own spamfilter, which would also decode the mails (the virusscanners then couldn't cope with the mail format) and send the decoded parts through the virusscanner. As a result, only for virusses then (we didn't trust our own spamfiltering), we parsed the whois entries for the originating IP addresses of infected mails and sent fully automated malware reports to the relevant abuse mail address. This was quite effective, as quite a lot of malware infected machines got cleaned as result and the load on our mailservers reduced noticeable. We have kept that concept, in the meantime the spam filtering has gotten sufficiently reliable too, and hence our tools now do automated reporting on both issues. With a very few exceptions amongst the ISPs spam bots amongst their customers get identified and cleaned very quickly as a result. When Yakov presented his I think second draft on the abuse feedback format, I touched base with him and we discussed one or two details. As result I changed our spam reporting to ARF (but left the malware reporting in our original free text reporting). In the meantime all large ISPs started to introduce their virus filtering, spam filtering became the norm, and the load overall reduced, especially malware seemed to be a thing of the past. After the shut down of the Russian Business Network our filter software did not catch a single malware for about 2, maybe 3 years. However, we are again seeing malware mails increasing in numbers and our filter probably catches a dozen or two every day again. So, malware reporting is still necessary. In the last few weeks a few of our free text malware reports bounced, one with a large ISP, who said in its bounce message, that MARF is expected for abuse reports. That's basically the trigger of why I checked the current definitions and started implementation of MARF feedback-type virus into our utility until I realized the issues (which resulted in the initial mail to you). Hence, to word a request to the MARF group: design the MARF drafts not only with manual, but also with automated responders in mind. I am convinced, that automated responders will create most of the feedback messages in not too distant future. In my opinion, the currently released documents do not create any problem with the spam reporting side. In the malware reporting side however, there is a serious issue I already pointed out. > I believe someone (you or someone else) claimed that nobody > has implemented an FBL that reports viruses so far. If > that's the case, we could consider updating the registry to > mark it as historic. Those are the places where > RFC3834 might apply. Well, if that claim of no-one ever reporting malware automatically ever existed, our implementation is the living example to the contrary. > If people concur that this is an issue, I would suggest > updating the base document to provide a requirement that a > virus report include the name of the reporting software and > the virus name it used, and restrict the third MIME part to > be text/rfc822-headers. I obviously support that motion! > There may be FBLs (such as one run by a malware analysis > company) that want to get every variant of a virus. > Reporting only the virus name and detection software would > mean the details of a variant are lost. I don't know > if this is an issue or not. Malware analysis companies have established their own channels of how to receive malware samples. That's not a consideration for reporting mail abuse, in my opinion. Mail abuse reports, as I see it, should trigger an action at the offender's end to stop identified abuse. Therefore it is irrelevant to see what variants exist - if the virusscanner at the complainer's side has detected a malware, there'll be a report that unambiguously identifies the malware. If the virus scanner did not recognize the threat, there will obviously be no report anyway and the malware go through - and may cause damage at the recipient's end, which then will trigger the malware being submitted to malware analysers. You see therefore, these are two very distinct steps, where malware analysers or abuse reports come into play. So we should remain focussed on the reporting side of identified abuse, which as far as I understand it is the purpose of MARF (mail abuse reporting format). To hammer the point in: It doesn't matter whether an abuse has been identified by the human pushing a button in his mailer, or whether the abuse has been identified by a spamfilter on established criteria or the abuse has been identified by a virus scanner based on its database and detection criteria. In every case, the abuse has been identified somehow and may lead to a MARF report. > They might wish to switch to FBLs as a standard way of > doing this. Perhaps we should ask them. No need to, that's a waste of resources as I lined out above, malware analysers mainly get involved on unidentified threats that already caused damage. They have their own ways of getting those samples. > This, I believe, is a restatement of the out-of-band issue; > we know (or at least assume) that reports from random > sources can't be trusted. Roger, so we are clear on this point. When I mentioned advertising the IP address of the potential MARF report transmitter similiar to SPF, I overlooked an important security question (that's always the danger of a single brain thinking). SPF and MARF have a decisive difference in the use for the bad boys. SPF is hardly exploitable even if it gets under full control of a malicious user. However, MARF could become a dangerous tool in the hands of a malicious user misleading ISPs/regulating bodies or launching DoS attacks against abuse departments. Which leads me to think on a second thought that the MARF senders should be identified through ARIN and their delegated authorities, too, just like the abuse mail addresses. That approach immediately puts the out of band topic to rest, too. Now, I guess, this would hamper introduction of MARF, especially before it gets widely accepted, hence there should be a compromise which allows MARF senders without such identification. I'd propose to look at the implementation of SPF, and instead of the service selector SPF use MARF, otherwise just import the SPF RFC. Implementations of MARF receiving processors checking the name servers thus could easily use the SPF library with some minor modifications to the source code. The syntax and format, apart from the service, could and should remain identical as the purpose is the same: SPF identifies IP addresses to transmit mails with the specific sender domain, MARF DNS identifies IP addresses permitted to transmit mail abuse reports for a particular entity. The imported SPF draft will need two additions: S.1: for reverse DNS resolving (IP address translated into domain name) the relevant MARF entry should be looked up at the full URL, if no entry is found there, drop the most inner part of the URL (e.g. in ip-x-y-z.xyzdomain.co.uk drop the ip-x-y-z) and retry (now on xyzdomain.co.uk) until all parts of the URL are used up (the root domains like com, co.uk, ... might want to publish a MARF entry too, who knows?). Stop at the first discovery of a MARF entry and check originating IP address against those results. (Reason: reverse resolving usually remains under control of the ISP due to name server granularity issues). S.2: for DNS name resolving (URL as input) resolve the IP address, then determine the MARF entry using S.1 (via reverse resolving). (Reason: malicious domain registrations still need some ISP to host their stuff, and we reach the ISP, not the malicious user that way). The draft should still nail down that the name server(s) being checked may be under control of a not trustworthy source, and any such identified MARF sender may still be needed to be treated with caution. If an ARIN entry exists identifying the permitted MARF sender, that entry overrules whatever is stated in the name server's MARF entry. So, in order to summarize the current standing of the discussion and proposals: A) feedback-type: virus I propose to change the keyword virus to malware. To facilitate the malware feedback type, the evidence section (section 3) of MARF should be changed from message/RFC822 to text/RFC822-Headers and contain only and only the complete header of the offending message. In the report section following fields become mandatory: Source-IP: reports the IP address the reporting server received the offending mail directly from Reported-Malware: the name of the malware as identified by the reporting malware scanner Reporting-Utitlity: the name of the reporting malware scanner, ideally also identifying the database in use An optional field: Reported-CVE-ID: the CVE ID numbers, separated by commas, identifying the threats, along http://cve.mitre.org B) MARF Discovery (_report subdomain) B.1 Rethink the whole draft in view of automated responders, malicious abuse and ongoing ARIN database extensions, I'd think the current proposal could be dropped B.2 Require ARIN/delegated authorities published abuse addresses to unconditionally accept MARF reports (they automatically do as it stands due to the humanly readable first part and third part containing the relevant evidence) B.3 Introduce a MARF sender identification (via IP address(es)) with ARIN/IP assignment authorities similiar to abuse address registration B.4. As long as ARIN/IP assignment authorities are not featuring such MARF database entries import the SPF RFC replacing selector SPF with MARF for identifying authorised MARF transmitters. A sample MARF report: ===================== From: "Complainer" <[email protected]> To: <[email protected]> Subject: MARF report Date: Thu, 28 Jul 2011 21:08:47 +0200 MIME-Version: 1.0 Content-Type: multipart/report; report-type=feedback-report; boundary="----=_NextPart_000_004A_01CC4D6A.8AA038D0" This is a multi-part message in MIME format. This is a MIME-formatted message. Portions of this message may be unreadable without a MIME-capable mail program. ------=_NextPart_000_004A_01CC4D6A.8AA038D0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable This is an example MARF compliant complaint: we received malware "W32/Mytob.AT@mm" from your IP address 192.168.0.45 at 21:08:45 +0200 Find the full header of the message below. We ask to have this machine checked and cleaned or temporarily disconnected until it has been cleaned. ------=_NextPart_000_004A_01CC4D6A.8AA038D0 Content-Type: message/feedback-report Feedback-Type: malware User-Agent: sample MARF generator Version: 1 Source-IP: 192.168.0.45 Reported-Malware: W32/Mytob.AT@mm Reporting-Utility: Sample Virus Scanner, database 2011-07-28-5142 Reported-CVE-ID: CVE-1999-0067, CVE-1999-0068 ------=_NextPart_000_004A_01CC4D6A.8AA038D0 Content-Type: text/RFC822-Headers Return-Path: <[email protected]> Received: from ([192.168.0.45]) by mail.complainer.com with ESMTP id p6SJ82qN000353 for <[email protected]>; Thu, 28 Jul 2011 19:08:45 GMT Date: Thu, 28 Jul 2011 19:08:45 GMT To: [email protected] From: [email protected] Subject: Your documents attached ------=_NextPart_000_004A_01CC4D6A.8AA038D0-- A sample DNS entry: =================== marf.sample.com. 1D IN TXT "v=marf1 ip4:192.168.0.48 -all" @ 1D IN TXT "v=marf1 redirect=marf.sample.com" _______________________________________________ marf mailing list [email protected] https://www.ietf.org/mailman/listinfo/marf
