Re: Please evaluate my understanding wrt access files
On 10/31/2009 12:32 AM, Stan Hoeppner wrote: All these cidr maps are a great example of separate tables that should be combined into a single table with a Makefile. Interesting. I may take a look into Makefile some time. I was unaware of it until today. Just out of curiosity, how does putting it all in one file make the resulting single file smaller than the sum of the individual files' contents? Does Makefile compress the contents in some way, such as postmap indexes hash files? I ask as the only overlapping data in my CIDR files is the REJECT + custom error within each CIDR file. Each lookup table requires overhead. 30 separate tables requires considerably more overhead than one table. The size of the dataset doesn't change, it's the overhead that gets smaller. The more concurrent smtpd processes running, the more it matters. It probably doesn't matter in your environment, but posting examples suitable for only a very small environment without labeled as such doesn't help the wider user base. And postmap doesn't compress files, but the index allows postfix to find a particular key quickly -- lookup time and system overhead for hash: and cdb: is essentially independent of the size of the table, so it makes sense to use a single table as big as necessary. In particular, cdb: scales well up to millions of entries, hash: tables easily to hundreds of thousands. -- Noel Jones
filter option for sasl authenticated users
Hi there, I am running Postfix 2.6.5 with Amavisd-new 2.6.4 I have a global content_filter statement and I override it on all mynetworks providing through amavisd-new dkim signing capability. Is there any way to do something similar with the sasl_authenticated users; Thanks in advance Dimitrios Karapiperis
Re: filter option for sasl authenticated users
Δημήτριος Καραπιπέρης a écrit : Hi there, I am running Postfix 2.6.5 with Amavisd-new 2.6.4 I have a global content_filter statement and I override it on all mynetworks providing through amavisd-new dkim signing capability. Is there any way to do something similar with the sasl_authenticated users; smtpd_sender_restrictions = check_client_access pcre:/etc/postfix/filter_trusted.pcre permit_mynetworks permit_sasl_authenticated check_client_access pcre:/etc/postfix/filter_default.pcre == filter_trusted.pcre /./ FILTER scan:[127.0.0.1]:10586 == filter_default.pcre /./ FILTER scan:[127.0.0.1]:10024 assuming, amavisd-new listens on 10586 for trusted users (mynetworks, sasl) and on 10024 for others. use amavisd policy banks to set different filtering configs.
Re: Reverse DNS Rejection Problem
Ansgar Wiechers a écrit : On 2009-10-29 Phillip Smith wrote: Tell the admin of the remote domain to fix their PTR records and/or MX helo configuration because in the meantime, you're going to have to implement a dirty hack to make their server work. But the PTR needs no fix. The IP resolves to a hostname perfectly fine , only that the hostname does not resolve. Then a) it doesn't resolve perfectly -- it should resolve both ways. And b) any given IP address should only have *one* corresponding PTR record, not multiple PTR's. For one, it causes problems like this. It's a perfectly valid and supported DNS feature to have multiple PTR records. If this causes problems, then the respective application is at fault, not DNS. Using multiple PTRs brings nothing but problems. there is nothing bad with a setup like this: 192.0.2.1 PTR uranus.example.com uranus.example.com A 192.0.2.1 www.example.com A 192.0.2.1 ftp.example.com A 192.0.2.1 blog.example.comA 192.0.2.1 wiki.example.comA 192.0.2.1 ...
Re: Reverse DNS Rejection Problem
On 2009-10-31 mouss wrote: Ansgar Wiechers a écrit : On 2009-10-29 Phillip Smith wrote: Then a) it doesn't resolve perfectly -- it should resolve both ways. And b) any given IP address should only have *one* corresponding PTR record, not multiple PTR's. For one, it causes problems like this. It's a perfectly valid and supported DNS feature to have multiple PTR records. If this causes problems, then the respective application is at fault, not DNS. Using multiple PTRs brings nothing but problems. Which part of then the respective application is at fault did you fail to understand? there is nothing bad with a setup like this: 192.0.2.1 PTR uranus.example.com uranus.example.comA 192.0.2.1 www.example.com A 192.0.2.1 ftp.example.com A 192.0.2.1 blog.example.com A 192.0.2.1 wiki.example.com A 192.0.2.1 ... There's also nothing wrong with a setup like this: 192.0.2.1 PTR uranus.example.com. 192.0.2.1 PTR www.example.com. 192.0.2.1 PTR ftp.example.com. 192.0.2.1 PTR blog.example.com. 192.0.2.1 PTR wiki.example.com. uranus.example.com. A 192.0.2.1 www.example.com.A 192.0.2.1 ftp.example.com.A 192.0.2.1 blog.example.com. A 192.0.2.1 wiki.example.com. A 192.0.2.1 Except that b0rken software may choke on it. Duh. And if you want to avoid multiple PTR records, there's also nothing wrong with a setup like this: 192.0.2.1 PTR uranus.example.com. uranus.example.com. A 192.0.2.1 www.example.com.CNAME uranus.example.com. ftp.example.com.CNAME uranus.example.com. blog.example.com. CNAME uranus.example.com. wiki.example.com. CNAME uranus.example.com. However, the OP's problem was not that a PTR record existed, but that a corresponding A record did *not* exist. Which is an entirely different issue. Regards Ansgar Wiechers -- Abstractions save us time working, but they don't save us time learning. --Joel Spolsky
Re: Reverse DNS Rejection Problem
On 10/31/2009 10:36 AM, Ansgar Wiechers wrote: There's also nothing wrong with a setup like this: 192.0.2.1 PTR uranus.example.com. 192.0.2.1 PTR www.example.com. 192.0.2.1 PTR ftp.example.com. 192.0.2.1 PTR blog.example.com. 192.0.2.1 PTR wiki.example.com. uranus.example.com. A 192.0.2.1 www.example.com.A 192.0.2.1 ftp.example.com.A 192.0.2.1 blog.example.com. A 192.0.2.1 wiki.example.com. A 192.0.2.1 Except that b0rken software may choke on it. Duh. ... and DNS returns a pseudo-random response, so you can't control which PTR gets returned first. ... and software that cares about the PTR and doesn't choke won't ever look past the first pseudo-random response. So you really don't gain anything other than getting to show off how you can cram lots of unnecessary stuff into your DNS record. Sometimes this makes the neubs feel better, but it really doesn't bring any benefit. -- Noel Jones
Re: Please evaluate my understanding wrt access files
Stan Hoeppner a écrit :
[snip]
Thanks for the hints Noel. I may need them down the road, although not
at the moment. Though I am curious and may play around with Makefile
just to learn something.
you can use a script if you prefer. the advantage of 'make' is that it
only re-generates files when needed (source change).
for your country maps and assuming gnu make, your Makefile would contain
something like this:
#---
#directpry where we store our cidr maps
CIDR_DIR=/etc/postfix/cidr_files
#file to generate
COUNTRIES_DEST=${CIDR_DIR}/countries
#source files, withput directory nor suffix.
COUNTRIES += china
COUNTRIES += korea
COUNTRIES += russia
...
#non portable commands. gnu make assumed.
CIDR_COUNTRIES=$(addsuffix .cidr, ${COUNTRIES})
COUNTRY_MAPS=$(addprefix ${CIDR_DIR}/, ${CIDR_COUNTRIES})
${COUNTRIES_DEST}: ${COUNTRY_MAPS}
cat $^ $@
...
#-
this will catenate all your china.cidr, korea.cidr, ... into countries
which you can then use with
check_client_access cidr:/etc/postfix/cidr_files/countries
PS. since you use a cidr specific directory, you can get rid of the
.cidr suffix. you could then use (in main.cf):
cidr=cidr:/etc/postfix/cidr_files
smtpd_foo_restrictions =
...
check_client_access ${cidr}/access_foo
...
check_client_access ${cidr}/access_bar
Re: smtpd_recipient_restrictions evaluation question
Mikael Bak a écrit : Larry Stone wrote: On Fri, 30 Oct 2009, Mikael Bak wrote: Simon Morvan wrote: The last time I tried it, Zen included too many legitimate users behind ADSL lines. The Policy behind PBL is a bit too restrictive. Maybe it changed, I'll give it another try. Can you please tell me why an ADSL user would send legitimate email without using the ISP's SMTP server? At ths risk of this moving too far away from Postfix, let me just ask if you're thinking ADSL means dynamic IP address? There are many legitimate mail servers on static IP ADSL lines (including mine) provided by ISPs with servers permitted policies. Typically these are business-class services but not always (my ISP does not distinguish between residential and business but their services are not priced for the mass-market residential user). Why handle the outgoing mail myself? Better control. If there's a problem, it sits on my system where I can see it and deal with it, not on my ISP's server where it's invisible to me. You are of course right. I ment home ADSL, not static IP business ADSL. And yes. We're moving away from postfix :-) Mikael And why shouldn't be able to use my own mail server behind my private residential ADSL line ? Not allowing so is against the net neutrality... but this discussion is already out of the postfix scope :) -- Simon.
Re: Reverse DNS Rejection Problem
On 2009-10-31 Noel Jones wrote: On 10/31/2009 10:36 AM, Ansgar Wiechers wrote: There's also nothing wrong with a setup like this: 192.0.2.1 PTR uranus.example.com. 192.0.2.1 PTR www.example.com. 192.0.2.1 PTR ftp.example.com. 192.0.2.1 PTR blog.example.com. 192.0.2.1 PTR wiki.example.com. uranus.example.com. A 192.0.2.1 www.example.com.A 192.0.2.1 ftp.example.com.A 192.0.2.1 blog.example.com. A 192.0.2.1 wiki.example.com. A 192.0.2.1 Except that b0rken software may choke on it. Duh. ... and DNS returns a pseudo-random response, so you can't control which PTR gets returned first. ... and software that cares about the PTR and doesn't choke won't ever look past the first pseudo-random response. You have a weird way of agreeing with me. So you really don't gain anything other than getting to show off how you can cram lots of unnecessary stuff into your DNS record. Sometimes this makes the neubs feel better, but it really doesn't bring any benefit. I didn't say that there's any actual benefit, but that having multiple PTR records is a valid configuration. Meaning that not any given IP address should only have *one* corresponding PTR record, but any given software should take into account the fact that a reverse lookup may return more than just one record. Besides, this still is unrelated to both Postfix and the OP's problem. Regards Ansgar Wiechers -- Abstractions save us time working, but they don't save us time learning. --Joel Spolsky
smtpd_restrictions sanity check
Hi all, Hopefully I don't have the most frequently asked question, but I'm spinning my wheels and perhaps followed some bad advice. I hoped someone could look over my recipient restrictions to see if I'm making some kind of mistake: smtpd_recipient_restrictions = reject_invalid_hostname, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, check_client_access hash:/etc/postfix/client_checks, check_recipient_access pcre:/etc/postfix/relay_recips_checks, check_helo_access hash:/etc/postfix/helo_checks, check_sender_access hash:/etc/postfix/sender_checks, check_sender_access hash:/etc/postfix/disallow_my_domain, permit_mynetworks, check_recipient_access pcre:/etc/postfix/recipient_checks, reject_unauth_destination, reject_maps_rbl, permit I originally had permit_mynetworks further up, but it seems client_checks was then being ignored, despite the client not being on my network. I'm now trying to provide a mail server that is not part of my networks to my network. I also have a handful of cron scripts that run on this remote network that send mail to my network, but with internal hostnames that aren't resolvable once they reach my network. Do I just add them to my postfix hosts file or is there a way to avoid checking the hostname (sender access?) so they aren't rejected with Sender address rejected: Domain not found? Thanks, Alex
Please evaluate my understanding wrt access files
Noel Jones put forth on 10/31/2009 1:12 AM: Each lookup table requires overhead. 30 separate tables requires considerably more overhead than one table. The size of the dataset doesn't change, it's the overhead that gets smaller. The more concurrent smtpd processes running, the more it matters. I guess I was kinda looking for the technical definition of 'overhead' here. If the overhead per file is a straight percentage, then it shouldn't matter if it's one big file or 30 smaller files. What is the actual memory footprint overhead that's being added by Posfix for each map file? And is the amount of overhead the same for all map types, or different/variable depending on map type? Is it something like 3 bytes of overhead added for every 10 bytes in a map file, for example? I can understand that having everything in one map file, or few map files, compared to many, can increase lookup performance due to indexing efficiency. What I don't understand is how having fewer larger map files decreases memory footprint. It probably doesn't matter in your environment, but posting examples suitable for only a very small environment without labeled as such doesn't help the wider user base. Agreed. That's why I described my environment. It's likely much smaller scale than the average Postfix environment. And postmap doesn't compress files, but the index allows postfix to find a particular key quickly -- lookup time and system overhead for hash: and cdb: is essentially independent of the size of the table, so it makes sense to use a single table as big as necessary. In particular, cdb: scales well up to millions of entries, hash: tables easily to hundreds of thousands. Agreed. Indexed files are always much faster to search than flat text files (unless we're talking really tiny flat files). And it's quicker to search fewer indexes than more indexes. This thread began with the premise being stated that many small map files consume more memory in Postfix than fewer larger map files. Indexes are typically proportional in size to the file being indexed, unless the file contains many duplicate entries. I freely admit I'm no indexing or database expert. However, that said, I don't understand how map file consolidation decreases memory footprint. Maybe Wieste can chime in with a brief explanation. Also, given that cidr files aren't postmap'd, is it safe to assume they aren't indexed? If they aren't indexed, how does putting the contents in a single file speed up searching? If they are indexed when Postfix starts or reloads, then why wouldn't Postfix created a single index _after_ reading in all the data from all the cidr files, creating one internal program table? -- Stan
Please evaluate my understanding wrt access files
mouss put forth on 10/31/2009 11:06 AM:
mouss, you rock.
you can use a script if you prefer. the advantage of 'make' is that it
only re-generates files when needed (source change).
The only likely changes would be adding another country. In this case,
would I just add the file name to the source section below, and run
the make script?
for your country maps and assuming gnu make, your Makefile would contain
something like this:
#---
#directpry where we store our cidr maps
CIDR_DIR=/etc/postfix/cidr_files
#file to generate
COUNTRIES_DEST=${CIDR_DIR}/countries
#source files, withput directory nor suffix.
COUNTRIES += china
COUNTRIES += korea
COUNTRIES += russia
...
#non portable commands. gnu make assumed.
CIDR_COUNTRIES=$(addsuffix .cidr, ${COUNTRIES})
COUNTRY_MAPS=$(addprefix ${CIDR_DIR}/, ${CIDR_COUNTRIES})
${COUNTRIES_DEST}: ${COUNTRY_MAPS}
cat $^ $@
...
#-
this will catenate all your china.cidr, korea.cidr, ... into countries
which you can then use with
check_client_access cidr:/etc/postfix/cidr_files/countries
Neat. I'm using gnu make so this should all work. Still don't
understand how this would affect memory footprint, but at least it would
clean up my main.cf a bit.
PS. since you use a cidr specific directory, you can get rid of the
.cidr suffix. you could then use (in main.cf):
cidr=cidr:/etc/postfix/cidr_files
smtpd_foo_restrictions =
...
check_client_access ${cidr}/access_foo
...
check_client_access ${cidr}/access_bar
Ahh, also sweet. I'd seen folks using variables but I hadn't really
found a need yet to dig into it. That and I was afraid of confusing
myself down the road. ;) Just implemented this. Works fine, main.cf
much cleaner looking, _and_ I'm now practicing good form with
check_client_access in front of each CIDR map. ;)
Lots of good ancillary info coming out of this thread. Thanks guys.
Still wondering about the map file memory overhead details...
--
Stan
Re: smtpd_restrictions sanity check
On Sat, 31 Oct 2009, Alex wrote: smtpd_recipient_restrictions = reject_invalid_hostname, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, check_client_access hash:/etc/postfix/client_checks, check_recipient_access pcre:/etc/postfix/relay_recips_checks, check_helo_access hash:/etc/postfix/helo_checks, check_sender_access hash:/etc/postfix/sender_checks, check_sender_access hash:/etc/postfix/disallow_my_domain, permit_mynetworks, check_recipient_access pcre:/etc/postfix/recipient_checks, reject_unauth_destination, Place this higher so you can sooner reject unauthorized relay attempts. reject_maps_rbl, permit This is unnecessary; remove it. I originally had permit_mynetworks further up, but it seems client_checks was then being ignored, despite the client not being on my network. This is not how Postfix works, so you borked something, somewhere else in your configuration. Instead of showing a snippet of your main.cf, paste the entire output of 'postconf -n' and logging that corresponds to your problem description. I also have a handful of cron scripts that run on this remote network that send mail to my network, but with internal hostnames that aren't resolvable once they reach my network. Do I just add them to my postfix hosts file or is there a way to avoid checking the hostname (sender access?) so they aren't rejected with Sender address rejected: Domain not found? Add them to a whitelist and consult that whitelist (with check_client_access) before rejecting unknown (or non-fqdn) sender domains. -- Sahil Tandon sa...@tandon.net
smtpd_recipient_restrictions evaluation question
Simon Morvan put forth on 10/31/2009 12:30 PM: And why shouldn't be able to use my own mail server behind my private residential ADSL line ? You should be able to. Here's how to implement the outbound mail portion to prevent mass rejections: http://www.hardwarefreak.com/postfix-adsl-relay-config.txt -- Stan
