>Are you sure??? If I run a DNS server that handles the REVERSE lookups for a specific block of IP's,
Yes, I'm 100% sure. YOU (and everyone else) will NOT be able to create a PTR record for a not owned domain and a NOT associated IP. PTR's are controlled by ISP's, which got IP-ranges for public distribution from and under terms of ARIN,RIPE,LACNIC,AFRINIC,KRNIC,APNIC ... - observed by the rules of IANA. What a (100%) senseless discussion! Repeating anything stupid, unwise or wrong over and over. doesn't make it anyhow better nor will it become true. Your assumtion, that anyone can easily get or hold its own public reverse lookup zone is currently the same way far away from reality, like the flight to the center of the milky way. >What if a yahoo user doesn't send through yahoo.?? block the mail - valid yahoo users will never able to send valid mails without using their yahoo-ISP-authentication DKIM will fail. SPF will fail if yahoo.com is in strictSPFRe and/or blockstrictSPFRe "v=spf1 ptr:yahoo.com ptr:yahoo.net ?all" is NOT strict enougth - "v=spf1 ptr:yahoo.com ptr:yahoo.net -all" should be used (strictSPFRe, blockstrictSPFRe ) SenderBase and WHOIS-IP may also help in such cases (not only yahoo). > You don't think that reporting to the carrier is a waste of time? The global players care about their reputation. But if no one reports abuse - nothing will change. BTW: I've got not a single malicious email from any global players SMTP-host for a very long time. Yes - from abused/faked addresses - but not from their registered hosts. Before you report - think about what an abuse is! It is not a mail you don't want (others may want them) - abuse is related to malicious code or links. Advertising mails are not an abuse outsite the EU! >So can you think of another way to insure that any ip that reverses to a yahoo domain isn't ever added to a pbwhite or pbblack? No. Collecting IP's of global players like yahoo, google...... for pbwhite or pbblack does not make any sense to me. They are DKIM signing every mail - and they never send mails from outsite their SPF ranges. >Most US ISP's let you authenticate with the ISP credentials, but send as anything once that's done. This is commonly done everywhere and nothing bad. SPFoverride may help. the 'elsif (/ptr:/' part of the script will not work in 99% - e.g. for all PTR definitions with domain definitions (not hosts) Thomas Von: "K Post" <nntp.p...@gmail.com> An: "ASSP development mailing list" <assp-test@lists.sourceforge.net> Datum: 09.09.2019 03:46 Betreff: Re: [Assp-test] noPB and NoPBWhite based on reverse dns Thanks as always for entertaining my questions Thomas. My comments below inline. On Thu, Aug 29, 2019 at 3:17 AM Thomas Eckardt <thomas.ecka...@thockar.com > wrote: >Is there a way to have an entry like *.yahoo.com in noPB or noPBWhite? No - hostnames in IP lists are forward lookedup, not reverse >I've got a little script that takes IP's from SPF records for major providers. How should such a script work for yahoo? - "v=spf1 ptr:yahoo.com ptr: yahoo.net ?all" You would need a list of all defined yahoo related PTR's. Yep, that's what my script does, but they're using PTR records unlike the others I use (like google, outlook.com, etc) >However, 66.163.184.147 doesn't match their SPF it matches yahoo's SFP record - the IP resolves to sonic309-21.consmr.mail.ne1.yahoo.com the SPF record is "v=spf1 ptr:yahoo.com ptr:yahoo.net ?all" True, as long as the IP reverses to a yahoo name, they say it's valid. That's why PTR isn't recommended / is depreciated in SPF, more below. yahoo.com should be in strictSPFRe and/or blockstrictSPFRe I don't know about that. Lots of silly users send yahoo mail through home ISP connections and their home ISP's smtp servers. I hate it, but it's the reality. Most US ISP's let you authenticate with the ISP credentials, but send as anything once that's done. Shameful. Think about the logic - if a mail is valid DKIM signed by yahoo.com, it is impossible that it was sent from an invalid SPF IP. If SPAM are sent from valid yahoo.com accounts and you expect to receive also HAM from there - only the personal black list and/or content base checks will help. If you get attacked with malicious mails from valid yahoo accounts, report the abuse to yahoo (or any other major provider). We get a TON of spam mails from actual yahoo accounts, sent through yahoo servers. You don't think that reporting to the carrier is a waste of time? >I'm aware that a spammer could easily have their ip reverse to a yahoo hostname No this should never be possible (even not in the US). To create a custom PTR-record - you need to create the related A or AAAA record first (you have to be the domain owner). Are you sure??? If I run a DNS server that handles the REVERSE lookups for a specific block of IP's, I could have 1.2.3.4 (provided my dns is the reverse for the 1.2.3.0/24 block), reverse to mail.yahoo.com. I'm not saying that mail.yahoo.com will then become 1.2.3.4 but if I did a reverse lookup of 1.2.3.4, it would show mail.yahoo.com. ASSP would be angry, because the IP of the mail.yahoo.com A record obviously wouldn't match. I feel like I could pass SPF for yahoo if I sent from an IP that I control the reverse DNS for. It would definitely fail DKIM. So can you think of another way to insure that any ip that reverses to a yahoo domain isn't ever added to a pbwhite or pbblack? I want to do all the other processing, including scoring (but not blocking for the previously explained reasons) SPF, DKIM, etc. If we could magically also use the PTR record for the sending IP to either match a single hostname, or wildcard like *.yahoo.com in noPBWhite and/or noPBBlack, the original issue goes away. It's still imperfect, but at least it would allow me to stop heavily used Yahoo ip's which are generally sending good mail from getting on the pbwhite and then decreasing the score which allows hmm only span through. Do you think it's a good idea to negate the pbwhite status by then assigning a negating score to anything from an @yahoo.com sender (net zero sum)? What if a yahoo user doesn't send through yahoo.?? Like I said, my nopb method has been working great with gmail and outlook.com, by periodically parsing all of the big provider's SPF records to find out which IP's to never PBwhite or PBblack. I works really well. There's just too much mail from these providers to whilelist or blacklist by IP. Making them good will have too much spam slip through. making them black will block way too much legitimate mail. Spammers will always abuse these free providers, but the services are too prevalent to penalize! It's a bit of a catch 22 that the method I use fixes - but not for yahoo because of their stupid SPF record using PTR's. I'm hoping with all my might that I've not been doing something incredibly stupid all along, but I know you'll tell me if that's the case! Might it be that I've got a system in place that could be rolled into ASSP as something that's universally useful? The script writes IP blocks to the files. Then in the group config, I'll do something like this: [GROUP-GOOGLE-IPS] # include IP-Lists/IPS-google.com.cfg >From there, I can add the group definition that I want to to noPBWhite and noPBBlack Here's the script: #!/usr/bin/perl -- # GetDomainIPSfromSPF v0.2 # Output all IP4 addresses to a file, one per line, from a hostname's SPF record(s) # does NOT consider PTR records # Copyright (C) 2015 Ken Post under the terms of GPL v3 # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License (http://www.gnu.org/licenses/) for more details. use strict; use warnings; use Mail::SPF::Query; use Net::Nslookup; my @names = ( "ccsend.com", "salesforce.com", "force.com", "mandrillapp.com", "outlook.com", "hotmail.com", "google.com", "amazon.com", "facebook.com", "facebookmail.com", "verticalresponse.com", "mailchimp.com", "bluestatedigital.com", "yahoo.com", "pphosted.com" ); my $hostname = ""; my $ipcomment = ""; foreach my $i (0 .. $#names) { $hostname = $names[$i]; $ipcomment = ""; open(my $fh, '>', 'IP-Lists\\IPs-' . $hostname . '.cfg'); print $fh "# \n"; print $fh "# Generated by WriteFile-GetDomainIPSFromSPF.pl \n"; print $fh "# \n"; RecurseSPF($hostname,'','FROMSPF:' . $ipcomment, $fh); close $fh; } sub RecurseSPF { my ($hostname,$ipcomment,$originalipcomment, $fh) = @_ ; my @SplitSPFLines; print "Working on " . $hostname . "\n"; # get SPF record for the hostname. Using Mail::SPF::Query out of convenience, # bogus IP and helo sent my $query = eval { new Mail::SPF::Query ( ip => '1.1.1.1', sender => 'someone@' . $hostname, helo => 'helo' )}; # spf_record gets populated with the SPF record my ($result, $smtp_comment, $header_comment, $spf_record, $detail) = $query->result(); if (defined $spf_record) { # split into an array of words based on spaces @SplitSPFLines = split /\s+/, $spf_record; } foreach (@SplitSPFLines) { # if the word starts include: or redirect: run RecurseSPF recursively again, # pulling up the SPF record for the referenced hostname if (/(include|redirect):/) { # strip off include:/redirect: s/(include|redirect)://; # run it recursively #print "# Include SPF for $_\n"; RecurseSPF($_,$_,$originalipcomment,$fh); #if we've found and IP4 record, print that IP address or range to stdout } elsif (/ip4:/) { s/ip4://; print $fh $_." $originalipcomment $ipcomment\n"; } elsif (/ptr:/) { s/ptr://; my @addrs = nslookup(type => "A", domain => $_); my $ThePTR = $_; foreach (@addrs) { print $fh $_." $originalipcomment from ptr $ThePTR - $ipcomment\n"; } } } } An: "ASSP development mailing list" < assp-test@lists.sourceforge.net> Datum: 29.08.2019 02:34 Betreff: [Assp-test] noPB and NoPBWhite based on reverse dns Is there a way to have an entry like *.yahoo.com in noPB or noPBWhite? I know we can put something like sonic309-21.consmr.mail.ne1.yahoo.com but what if I never want any IP that reversed to any yahoo.com name to be penalized? I'm aware that a spammer could easily have their ip reverse to a yahoo hostname, but I'd hope to catch using other methods. I've got a little script that takes IP's from SPF records for major providers. (I've posted it here before). Those IP's get added to group definitions and can be used from there. One thing I've done for a long time is having the IP's from gmail's and yahoo's SPF records in noPB and noPBWhite. This way, these email providers are never penalized nor pbWhite. Too many spammers send mail through real yahoo and gmail accounts, but we can't negatively score because about 20% of our legit inbound mail comes from these 2 providers. We also don't want to pbWhite the IP's or bayesian/hmm spam will get 15 points removed and pass. This has worked great for a long long time. However, with yahoo, I'm noticing now that there's inbound mail coming from non-SPF matching IP addresses. For example: Aug-24-19 12:27:31 61051-11848 66.163.184.147 <sen...@yahoo.com> to: ouru...@domain.org [scoring] DKIM signature verified-OK - header-passed - identity is: @yahoo.com - sender policy is: neutral - author policy is: neutral Aug-24-19 12:27:32 61051-11848 66.163.184.147 <sen...@yahoo.com> to: ouru...@domain.org Message-Score: added -15 (pbwValencePB) for In Penalty White Box, total score for this message is now -15 That message DKIM verified. It really came through yahoo. However, 66.163.184.147 doesn't match their SPF, so it wasn't excluded from my IP whitelist. It's in the pbWhite. Even though the message gets 50 for bayesian, it starts at -15, so passes. Any other suggestions are very welcome!! thanks _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test DISCLAIMER: ******************************************************* This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! ******************************************************* _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test DISCLAIMER: ******************************************************* This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! *******************************************************
_______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
