Re: Sieve: Avoiding duplicates in a folder while keeping in Inbox
ok here are the basics
see example plugins below
plugin {
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename
mail_log_fields = uid, box, msgid, from, subject, size, vsize, flags
push_notification_driver = dlog
sieve = file:~/sieve/sieve;active=~/sieve/.dovecot.sieve
sieve_duplicate_default_period = 1h
sieve_duplicate_max_period = 1d
mail_replica = tcp:10.221.0.7:12345
#mail_replica = tcps:10.221.0.7
#mail_replica = remote:vmail@10.221.0.7
#replication_sync_timeout = 2
fts = lucene
fts_lucene = whitespace_chars=@.
}
&
protocol sieve {
managesieve_implementation_string = Dovecot Pigeonhole
managesieve_max_line_length = 65536
}
basically
sieve_duplicate_default_period = 1h
sieve_duplicate_max_period = 1d
handles the duplicate supression
see :
https://doc.dovecot.org/configuration_manual/sieve/extensions/duplicate/
Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.)
Have A Happy Monday !!!
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 5/13/2024 9:48 AM, Stephan Bosch via dovecot wrote:
Op 13-5-2024 om 14:14 schreef Nick Urbanik via dovecot:
On Thu, 2024-04-25 at 14:36 +1000, Nick Urbanik via dovecot wrote:
On Tue, 2024-04-23 at 09:42 +1000, Nick Urbanik via dovecot wrote:
Dear Folks,
I am in the process of learning Sieve, and want to be able to keep
an
email in Inbox, while also keeping *one* copy in another folder.
However, the same mail is copied from Inbox multiple times when I
run
sieve-filter -Wev .dovecot.sieve Inbox
My attempt to prevent this fails:
...
} elsif header :comparator "i;octet" :contains "From"
"pay...@paypal.com.au" {
if not duplicate {
fileinto "paypal";
}
keep;
}
Please can anyone enlighten me?
If there is a Dovecot Sieve mailing list, please point me to it.
OK, let me rephrase the problem:
I want to keep a copy of an email in Inbox and put a copy into another
folder, but only once, even if I run sieve-filter as above multiple
times. I know there are many who know how to do this and I would really
appreciate your generous suggestions.
The sieve-filter tool was mainly created to mend mailboxes after
failures in sieve execution at delivery time. It currently does not
provide access to the duplicate database and any actions that operate on
it are ignored.
What exactly are you using sieve-filter for? Why are the Dovecot LDA or
the Dovecot LMTP service with the Sieve plugin not fulfilling your needs?
Regards,
Stephan.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Any way to make a shared mailbox(not a shared folder)
In general how i do it is just make an info@ account and login as an imap user this allows multiple info@ users especially in thunderbird to be connected as a seconday account accross any device. aka just treat any shared mailboxes as actual mail accounts, way simplier and easier to manage? any user can then move/delete etc an email that they will be looking after thus updating the info@ at the same time. its very common today to have info@ and accounting@ with multiple users attached to these as different people will be responsible for different emails thunderbird allows drag and drop accross email accounts (only one email at a time), this allows easy management. Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Have A Happy Tuesday !!! Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2024-04-02 7:25 a.m., Maksim Rodin wrote: Hello I wonder if there is a right way to make a shared mailbox? I do not mean "shared folder" but a whole mailbox. E.g. I have a mailbox i...@company.com and I have a user mailbox us...@company.com. I would like that a user which can already authenticate as us...@company.com can setup another account in his Thunderbird as, say, us...@company.com\i...@company.com or something like user1*info, enters his own password and can use the mailbox i...@company.com as his second mailbox. It might be something similar to master user feature but I do not want the us...@company.com to have access to all the mailboxes on the dovecot imap server. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: doveadm fetch complains as Message was expunged
Nope world of multitasking imap fetchs a list of emails at the time of the fetch and then processes it after the fact with that list i have python scripts running doing similar stuff best suggestion is to make an error trap and rerun 5 mins later or something ?? I would not really consider it a bug in a multi user / multitasking environment, stuff will trip over each other once in a while you just need to accomodate for any errors thrown Have A Happy Sunday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2/10/2024 7:24 AM, Kirill A. Korinsky wrote: Greeetings, I do have a cron script which runs doveadm NOT mailbox Junk SEEN SINCE 30d Everything works well with one exception, if user removes email when it's running, it may lead to an email from cron like: doveadm(...): Error: fetch(hdr) failed for box=virtual.All uid=145266: Message was expunged (for backend mailbox INBOX) I feel that it is a bug, isn't it? ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Geofencing
thanks for the insite, being an ISP I like this kind of info even if it is off topic a bit on the dovecot mail lists, security today is up there with opertional stuff. Have A Happy Thursday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2023-11-16 5:31 p.m., Jochen Bern wrote: On 16.11.23 16:56, Paul Kudla wrote: the ip that triggered all this says it is allocated from NL (Neatherlands) but physicaly exists in Hawii ? As someone working for a LIR, let me clarify a couple things: IPs get assigned to organizations. The registered contacts may well be that organization's main offices on one continent while the hardware actually using those addresses is located someplace different - and the users whose traffic gets its public IP from that hardware could well be in a third. If we were also an upstream provider operating in several nations, we would not be obliged to use separate IP ranges for (the customers in) different nations, or to register such information with the RIR, much less making it public. One of our customers uses the services of ZScaler to access the Internet, and thus a service where we maintain a whitelist of client IPs that may connect. Every now and then, "their" IPs will change from, e.g., a range assigned to "ZScaler Düsseldorf", to one designated "ZScaler Zürich", to "ZScaler Frankfurt", etc., while our actual customer doesn't move more than whatever amount the keycaps on his keyboard need to travel. Having that said, there are people trying to *second guess* the actual location behind an IP address, from Google (ever wondered why, when you open Google Maps, it usually *happens* to show the place you're in?) to https://www.maxmind.com/en/solutions/ip-geolocation-databases-api-services to hobbyists, and there are software frameworks to make services geofenced or location aware (e.g., there are packages "GeoIP" and "plasma-workspace-geolocation" installed on my laptop apparently right off the bat). And yes, there might easily be no info for an IP you look up, or some that's plain wrong. And *then* there are things like Anycast or BGP hijacking or VPN services to obscure one's origin or ... Kind regards, ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Anyone Watching Actvity from this network? Attempting Dovecot Buffer Overflows?
Ok a few things about IP blocks
If they are portable they can move from country to country ??
without any real notice.
the ip that triggered all this says it is allocated from NL
(Neatherlands) but physicaly exists in Hawii ?
No list will ever be 100% acurate
I did find this link that displays by country but then you have to click
the country understanding that some sub nets are split out by class "A"
/ "B" & "C"
A whole class "A" for example can be split into many subclasses thus
point difference ranges to different countries.
https://www.nirsoft.net/countryip/
maybe write a python program to grab and make a table of ip addresses ?
it has a link to download a csv so some kind of loop striping out the
country links would probably be ok and then download the csv file and
create a full csv file.
then use that for your firewall keeping in mind it needs to be updated
regularly.
I did look around as arin net is responsible for all of this but could
not find a list there either.
https://www.arin.net/reference/
Airn Net is mainly responsible for allocating blocks but not really
responsible for where they might get used.
same with other whois databases around the globe.
also note IPV6 is also out there now and adds a whole new layer to all
of this.
Have A Happy Thursday !!!
Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.)
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 11/16/2023 9:31 AM, Brendan Kearney wrote:
On 11/16/23 9:05 AM, Nick Lockheart wrote:
Are there publicly available lists of IP ranges by region?
There's no reason for any IP outside of North America to be contacting
Postfix
on Submission (587) or IMAP, since these are employee only services.
If not for mobile phones, we could really close it off.
On Thu, 2023-11-16 at 08:27 -0500, Paul Kudla wrote:
Good day to all .
Just adding to the conversation with how I had to deal with this
years ago.
Basically hacks to any server are an issue today but it is cat &
mouse
trying to track all of this.
That being said using the reported ip address below, I patched
postfix
to log the ip address in one syslog pass (to id the sasl user
account
+
ip etc)
Along with the above dovecot logging is verbose (dovecot already
does
all access in one line - ie ip address, username (email address)
etc)
combining the two I run my own ip address firewall tracking system
based
on the syslogging in real time.
For Example :
__
# ipinfo 104.156.155.21
IP Status for : 104.156.155.21
IP Status : IPv4
NS Lookup (Forward) : 104.156.155.21
NS Lookup (Reverse) : None
IP Blacklisted Status : Found 104.156.155. for
104.156.155.21
[D] {Asterisk}
Last Program : sshd
Ip Location Info for : 104.156.155.21
No Ip Information Found
(ie ip location lookup failed / does not exist for this ip ?)
__
basically the ip address block was found in my firewall so
something,
someone etc has tried to hack one of my servers
in the case of scom.ca i run an asterisk server and since the
asterisk
is noted someone tried hacking that one as well.
Basically i run a database that tracks and updates all firewall in
real
time.
Running FreeBSD I use PF and asterisk is linux based so i use the
iptables and update every 10 minutes.
Only time now a days I get involved if a customer calls and
complains
they are not getting emails etc ...
That happens a few times a year.
Again just an FYI
This reply was more to indicate all email servers (and anything
attached
to the internet) really need to run some sort of automated ip
firewall
when username password hacks occur, no reverse ip address etc
etc etc
Food for thought.
Have A Happy Thursday !!!
Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.)
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 11/15/2023 5:53 PM, Simon B wrote:
On Wed, 15 Nov 2023, 23:25 Michael Peddemors,
wrote:
There is a network claiming to be a security company,
however the
activity a
Re: Anyone Watching Actvity from this network? Attempting Dovecot Buffer Overflows?
Good day to all .
Just adding to the conversation with how I had to deal with this years ago.
Basically hacks to any server are an issue today but it is cat & mouse
trying to track all of this.
That being said using the reported ip address below, I patched postfix
to log the ip address in one syslog pass (to id the sasl user account +
ip etc)
Along with the above dovecot logging is verbose (dovecot already does
all access in one line - ie ip address, username (email address) etc)
combining the two I run my own ip address firewall tracking system based
on the syslogging in real time.
For Example :
__
# ipinfo 104.156.155.21
IP Status for : 104.156.155.21
IP Status : IPv4
NS Lookup (Forward) : 104.156.155.21
NS Lookup (Reverse) : None
IP Blacklisted Status : Found 104.156.155. for 104.156.155.21
[D] {Asterisk}
Last Program: sshd
Ip Location Info for: 104.156.155.21
No Ip Information Found
(ie ip location lookup failed / does not exist for this ip ?)
__
basically the ip address block was found in my firewall so something,
someone etc has tried to hack one of my servers
in the case of scom.ca i run an asterisk server and since the asterisk
is noted someone tried hacking that one as well.
Basically i run a database that tracks and updates all firewall in real
time.
Running FreeBSD I use PF and asterisk is linux based so i use the
iptables and update every 10 minutes.
Only time now a days I get involved if a customer calls and complains
they are not getting emails etc ...
That happens a few times a year.
Again just an FYI
This reply was more to indicate all email servers (and anything attached
to the internet) really need to run some sort of automated ip firewall
when username password hacks occur, no reverse ip address etc etc etc
Food for thought.
Have A Happy Thursday !!!
Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.)
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 11/15/2023 5:53 PM, Simon B wrote:
On Wed, 15 Nov 2023, 23:25 Michael Peddemors, wrote:
There is a network claiming to be a security company, however the
activity appears to be a little more malicious, and appears to be
attempting buffer overflows against POP-SSL services.. (and other
attacks).
https://www.abuseipdb.com/check/104.156.155.21
Just thought it would be worth mentioning, you might want to keep an
eye
out for traffic from this company...
Might want to make up your own mind, or maybe someone has more
information, but enough of a red flag, that thought it warranted
posting
on the list.
Not sure yet if it is Dovecot, or the SSL libraries they are
attempting
to break, but using a variety of SSL/TLS methods and connections...
They are not interested in dovecot per se. They scan for TLS vulnerabilities,
mostly.
Anyone with more information?
NetRange: 104.156.155.0 - 104.156.155.255
CIDR: 104.156.155.0/24
NetName: ACDRESEARCH
NetHandle: NET-104-156-155-0-1
Parent: NET104 (NET-104-0-0-0-0)
NetType: Direct Allocation
OriginAS:
Organization: Academy of Internet Research Limited Liability
Company
(AIRLL)
RegDate: 2022-01-07
Updated: 2022-01-07
Ref: https://rdap.arin.net/registry/ip/104.156.155.0
OrgName: Academy of Internet Research Limited Liability
Company
OrgId: AIRLL
Address: #A1- 5436
Address: 1110 Nuuanu Ave
City: Honolulu
StateProv: HI
PostalCode: 96817
Country: US
RegDate: 2021-10-15
Updated: 2022-11-06
Ref: https://rdap.arin.net/registry/entity/AIRLL
--
See also shadowserver.org, census.io, stretchoid, etc. All of them allegedly
reputable, all of them supposedly with opt-out mechanisms, and all of them are
blocked for not asking permission.
Ymmv.
Regards
Simon
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
Re: ldap passdb, userdb imap auth sasl and lmtp
Good morning, OK (If I am wrong someone please update this!) Trying to run multiple auth schemes when sasl is avaliable etc is overkill Next trying to auth via AD (this is mainly another mess windows made) is also impractible, sasl was invented as an auth layer in the first place to then provide various auth mech's to a backend (ldap, mysql, pgsql, local etc etc etc) if you have sasl running for postfix, use that for dovecot or at the very least setup dovecot to read the database you have running sasl layer directly (what I am doing) if you are running different users & passwords in different setups then you will have to update sasl to have the same auth info in it anyways for postfix to work thus making AD and whatever else not needed ? Again just my opinion without more detail but AD was never designed (to my knowledge) to auth users for user@domain ? AD was maily designed for domains & users across multiple network servers (ie one login to auth multiple servers?) FYI Have A Happy Tuesday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 11/13/2023 4:03 PM, bd730c5053df9efb via dovecot wrote: Hi! I'm trying to setup dovecot 2.3.17 such that it authenticates users against a samba4 ad dc when they connect through imap. I would also need dovecot to authenticate the same users through sasl so that postfix can relay mails based on the user's auth and finally I would also need for an lmtp service that maps email addresses into AD users. I've been tinkering with dovecot-ldap.conf.ext using auth_bind = yes and no, with all sorts of pass_filters, user_filter, pass_attr and user_attr and I just can't figure out how it works. I assume that for the auth part (both imap and sasl) I would rather benefit with using auth_bind = yes and auth_bind_userdn = %u and I seem to be able to authenticate the user but I can't get the passdb to prefetch the userdb attributes. I also assume that for lmtp to be able to fetch the ad information it would be necessary for dovecot to be able to bind to the ad ldap server with a "service account" to be able to query the ldap server and I haven't been able to figure out how to have both kinds of auth schemas. Does anyone know of some documentation that could clarify some of this issues, I have been searching the web for days to no avail. I'm sorry I can't show what I have tried for I have tried so many things with more or less the same lack of success that I wouldn't know where to start. Thanks in advance. Best regards, Dave. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: mail boxes on net mounted filesystem with multiple Dovecots
Ok (My Opinion Only) NFS in general does not work well on active servers, although dovecot allows for various locking mech's they do generally trip over each other. This occurs on NFS mounts using a single server and just goes down hill from there if you have 2 servers talking to the same NFS file mount. Simply put its a crap shoot what will work and when I know this is a touchy subject but this is what replication was used for and works well between 2 or more servers updating email boxes in real time It does require a proper database (MySql or Postgresql) and prefereably a dedicated private network between the two mail servers running dovecot. I tried everything noted in this post and it just does not work. Have A Happy Monday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 11/6/2023 9:54 AM, Aki Tuomi via dovecot wrote: On 06/11/2023 16:48 EET lejeczek via dovecot wrote: Hi guys. I see that with mailboxes stored on a network mount-point and more than one box with Dovecot using such a mailbox, Dovecots step on each others toes. ... lmtp(minem...@lemko.xyz)<2674357>: Error: lmtp-server: conn unix:pid=2600068,uid=89 [2]: rcpt minemail@my.private: Mailbox INBOX: Corrupted transaction log file /VMAIL/my.private/minemail/dovecot.index.log seq 4: ext intro: name_size too large (sync_offset=6368) ... Above happened if the same one user was having mail delivered on two Postfix+Dovecot servers at the same time. I hope experts who know Dovecot's internals better can tell... having such multiple node/server Dovecots "talking" to that same network mount-point but!.. only one Dovecot being active - having Postfix using it and other, however many, Dovecots only "idling" - not having Postifx using it (+ no client connections is a goal too) Would that make such multi-Dovecot setup safe & free from errors as above & any storage related ones? Or perhaps there are other ways to have many Dovecots with the same user-base, using same networked storage simultaneously? many thanks, L. Hi guys. I see that with mailboxes stored on a network mount-point and more than one box with Dovecot using such a mailbox, Dovecots step on each others toes. ... lmtp(minem...@lemko.xyz)<2674357>: Error: lmtp-server: conn unix:pid=2600068,uid=89 [2]: rcpt minemail@my.private: Mailbox INBOX: Corrupted transaction log file /VMAIL/my.private/minemail/dovecot.index.log seq 4: ext intro: name_size too large (sync_offset=6368) ... Above happened if the same one user was having mail delivered on two Postfix+Dovecot servers at the same time. I hope experts who know Dovecot's internals better can tell... having such multiple node/server Dovecots "talking" to that same network mount- point but!.. only one Dovecot being active - having Postfix using it and other, however many, Dovecots only "idling" - not having Postifx using it (+ no client connections is a goal too) Would that make such multi-Dovecot setup safe & free from errors as above & any storage related ones? Or perhaps there are other ways to have many Dovecots with the same user-base, using same networked storage simultaneously? many thanks, L. Hi! See https://doc.dovecot.org/configuration_manual/nfs/ This applies to other shared mountpoints too. Aki ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Minimum configuration for Dovecot SASL only?
ni file basically tabbed spaced cert name & cert pem file location (pem file has whole cert info) # cat sni .scom.ca /etc/ssl/postfix.pem.scom secure.mail.elirpa.com /etc/ssl/postfix.pem.elirpa you then need to run "/usr/local/sbin/postmap -c /usr/home/postfix/config -F /usr/home/postfix/config/sni" which will make a valid sni database mapping Yes this is a lot of work but does work well and independant of dovecot. Running : ## postconf mail_version mail_version = 3.4-20181202 & ## openssl version OpenSSL 3.1.0-dev (Library: OpenSSL 3.1.0-dev ) Have A Happy Monday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 11/5/2023 3:36 PM, jeremy ardley via dovecot wrote: On 6/11/23 03:25, Nick Lockheart wrote: I can't use the real Dovecot IMAP server for auth, because it runs on a separate server, and Postfix does not support TLS connections for SASL. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Minimum configuration for Dovecot SASL only?
i researched MariaDB it seems that it is based from mysql, you will probably need to lookup postfix + mysql auth but the syntax should be pretty close. Have A Happy Saturday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 11/4/2023 8:27 PM, Paul Kudla wrote: Actually postfix can auth with sasl without dovecot I run this myself. I am unfamiliar with MariaDB however postgresql runs just fine. You need to setup postfix + sasl (its an option at compile time) and then setup the main.cf postfix config mappings & outgoing sasl auth accordingly that will map to the database setup. dovecot is not required if all you are doing is sending emails. if you intend to receive emails then dovecot is required so they can be delivered ? Have A Happy Saturday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 11/4/2023 7:32 PM, Michael Peddemors wrote: Why use Dovecot/IMAP at all for the SMTP Authentication, can't you simply go direct to your database? On 2023-11-03 09:55, Nick Lockheart wrote: I have a Dovecot IMAP server and a Postfix server on separate machines. The user information is stored in a MariaDB database that is replicated on both servers. Postfix needs to authenticate outgoing mail against our valid user database. I believe this requires us to install a "dummy" Dovecot on the Postfix server so that Dovecot SASL can provide authentication to Postfix from the database. I think Cyrus had a standalone Cyrus-SASL package, but Dovecot doesn't? If I wanted to setup a Dovecot instance on the Postfix server just for the purposes of SMTP authentication, and not use it to handle any mail, what is the minimum configuration required to make that work? Is the dovecot-common package (Debian) enough? Or do I need the full dovecot-imap package? What protocols go in the protocols directive? Can you just make it "protocols = auth" to disable IMAP connections? ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Minimum configuration for Dovecot SASL only?
some of the main.cf config #Auth Stuff smtpd_sasl_auth_enable = yes #Dovecot #smtpd_sasl_path = private/auth #smtpd_sasl_type = dovecot #Cyrus smtpd_sasl_type = cyrus smtpd_sasl_path = smtpd #SASL Common broken_sasl_auth_clients = yes smtpd_sasl_security_options = noanonymous smtpd_sasl_authenticated_header = yes smtpd_sasl_local_domain = $myhostname /usr/home/postfix/config/sasl points to the smtpd postfix lib lrwxr-xr-x 1 root wheel - 31B Mar 27 2013 smtpd.conf -> /usr/local/lib/sasl2/smtpd.conf example smtpd.conf file (needed for auth) : # cat smtpd.conf #Local Password Database #pwcheck_method: saslauthd #mech_list: login plain #saslauthd_path: /var/run/saslauthd #Postygres pwcheck_method: auxprop mech_list: PLAIN LOGIN auxprop_plugin: sql sql_engine: pgsql sql_hostnames: sql_database: xxx sql_user: xx sql_passwd: x sql_select: SELECT password FROM email_users WHERE username = '%u@%r' and password <> 'alias' and currentcount_bad < 30 and status = True #Logging? log_level: 7 _ note select statement and db connection needs to match however to talk to your db? Have A Happy Saturday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 11/4/2023 7:32 PM, Michael Peddemors wrote: Why use Dovecot/IMAP at all for the SMTP Authentication, can't you simply go direct to your database? On 2023-11-03 09:55, Nick Lockheart wrote: I have a Dovecot IMAP server and a Postfix server on separate machines. The user information is stored in a MariaDB database that is replicated on both servers. Postfix needs to authenticate outgoing mail against our valid user database. I believe this requires us to install a "dummy" Dovecot on the Postfix server so that Dovecot SASL can provide authentication to Postfix from the database. I think Cyrus had a standalone Cyrus-SASL package, but Dovecot doesn't? If I wanted to setup a Dovecot instance on the Postfix server just for the purposes of SMTP authentication, and not use it to handle any mail, what is the minimum configuration required to make that work? Is the dovecot-common package (Debian) enough? Or do I need the full dovecot-imap package? What protocols go in the protocols directive? Can you just make it "protocols = auth" to disable IMAP connections? ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Minimum configuration for Dovecot SASL only?
Actually postfix can auth with sasl without dovecot I run this myself. I am unfamiliar with MariaDB however postgresql runs just fine. You need to setup postfix + sasl (its an option at compile time) and then setup the main.cf postfix config mappings & outgoing sasl auth accordingly that will map to the database setup. dovecot is not required if all you are doing is sending emails. if you intend to receive emails then dovecot is required so they can be delivered ? Have A Happy Saturday !!! Thanks - Paul Kudla (Manager SCOM.CA Internet Services Inc.) Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 11/4/2023 7:32 PM, Michael Peddemors wrote: Why use Dovecot/IMAP at all for the SMTP Authentication, can't you simply go direct to your database? On 2023-11-03 09:55, Nick Lockheart wrote: I have a Dovecot IMAP server and a Postfix server on separate machines. The user information is stored in a MariaDB database that is replicated on both servers. Postfix needs to authenticate outgoing mail against our valid user database. I believe this requires us to install a "dummy" Dovecot on the Postfix server so that Dovecot SASL can provide authentication to Postfix from the database. I think Cyrus had a standalone Cyrus-SASL package, but Dovecot doesn't? If I wanted to setup a Dovecot instance on the Postfix server just for the purposes of SMTP authentication, and not use it to handle any mail, what is the minimum configuration required to make that work? Is the dovecot-common package (Debian) enough? Or do I need the full dovecot-imap package? What protocols go in the protocols directive? Can you just make it "protocols = auth" to disable IMAP connections? ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Replication going away?
I know this might have already been answered Can some one give a link to the paid site that does what dovecot project does now more then happy to keep the lights on ! pls advise link ? Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 7/26/2023 5:12 AM, Noel Butler via dovecot wrote: On 20/07/2023 05:55, Gerald Galster wrote: A dns query for imap.web.de address records (IN A) returns two ip addresses. And I'm betting each IP is a hardware load balancer with crap load of servers behind each :) -- Regards, Noel Butler This Email, including attachments, may contain legally privileged information, therefore at all times remains confidential and subject to copyright protected under international law. You may not disseminate this message without the authors express written authority to do so. If you are not the intended recipient, please notify the sender then delete all copies of this message including attachments immediately. Confidentiality, copyright, and legal privilege are not waived or lost by reason of the mistaken delivery of this message. -- This message has been scanned for viruses and dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is believed to be clean. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Outlook and IMAP Flags
just an fyi microsoft outlook does not support most imap features the main one being imap idle (their equivalent to push messages) outlook just does not work with imap very well, its their push for going onto exchange / outlook365 Happy Saturday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 7/22/2023 8:00 AM, Laura Smith via dovecot wrote: Hi I've tried searching the internet, but the only thing I can find is a post on a MIcrosoft forum where a Microsoft reps claims flags are not supported on IMAP (I thought it was an RFC3501 feature ?). Anyway, I have a user who has Outlook/Windows on desktop and iOS (iPhone/iPad) for remote. On the iOS devices, the user can happily set flags against messages with zero issues. And indeed, when they set these flags, they are shown in Outlook. However if they attempt to set the flag in Outlook, nothing happens. Outlook continues showing the message as if it was unflagged. Any ideas ? Laura ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Windows 11 Outlook 365 and dovecot
In general YES ! Any microsoft email imap client past Outlook 2010 justdoes not handle imap properly Windows mail seems not to be too bad for small accounts. there is no IDLE support (ie auto update changes) You need to adjust the mailbox timing down from the default 30 minutes to like 5 in order for the email to update semi-live imap will work and map the trash folders etc ok however it is basically non-functional for the rest. Basically any inbox above 500 or so messages will slow to a halt. If you have to make it work you can fiddle with the settings and it will work (kind of) you are better off using another email client like thunderbird which fully supports imap / dovecot across several devices ? Other option is to look at an exchange proxy/emulator like sogo but that comes with a wack of setup and other complications. These comments come from years of being an isp (fyi) Happy Sunday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 7/15/2023 7:01 PM, dovecot-boun...@dovecot.org wrote: Could be as simple as not having a trash folder mapped in Outlook. Any error messages? On 2023-07-15 16:06, The Doctor via dovecot wrote: Are there any known Idssues? I help set up a client this morning using Win11 and Dovecot and the client cannot delete e-mail like he did previously. -- This message has been scanned for viruses and dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is believed to be clean. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: dovecot and postfix, authentication issue
just an fyi
postfix runs it's own authenticaion especially with virtual users
please note below is based on pgsql but the concept is the same for
passwd - have not used that for a while
simply put virtual users both with postfix & dovecot works way better
with a pgsql database running the show
-rw-r--r-- 1 root vmailuarch 565B Feb 20 05:30 pgsql-aliases.cf
-rw-r--r-- 1 root vmailuarch 434B Feb 20 05:30 pgsql-canonical.cf
-rw-r--r-- 1 root vmailuarch 457B Feb 20 05:31
pgsql-mydestination.cf
-rw-r--r-- 1 root vmailuarch 568B Feb 20 05:31 pgsql-virtual.cf
# cat pgsql-virtual.cf
# pgsql-virtual.cf ***
#
# pgsql config file for alias lookups on postfix
# comments are ok.
#
# the user name and password to log into the pgsql server
hosts = dovecot-mail18.scom.ca:5433
user = pgsql
password =
# the database name on the servers
dbname = xxx
# the table name
table = email_users
#
#Select source email address alias (ie sales@ etc aliases )
where_field = source
#Select destination email account address (final delivery)
select_field = destination
#Account Status (1=good)
additional_conditions = and status = '1'
in main.cf (postfix)
dovecot_destination_recipient_limit = 1
virtual_transport = dovecot
#Postgres Stuff
mydestination = pgsql:/usr/home/postfix/config/pgsql-mydestination.cf
#virtual_maps=pgsql:/usr/home/postfix/config/pgsql-virtual.cf
alias_maps=pgsql:/usr/home/postfix/config/pgsql-aliases.cf
#sender_canonical_maps = pgsql:/usr/home/postfix/config/pgsql-canonical.cf
dovecot config :
# cat dovecot-pgsql.conf
driver = pgsql
connect = host=dovecot-mail18.scom.ca port=5433 dbname= user=pgsql
password=
default_pass_scheme = PLAIN
password_query = SELECT username as user, password FROM email_users
WHERE username = '%u' and password <> 'alias' and status = True and
destination = '%u'
user_query = SELECT home, uid, gid FROM email_users WHERE username =
'%u' and password <> 'alias' and status = True and destination = '%u'
#iterate_query = SELECT user, password FROM email_users WHERE username =
'%u' and password <> 'alias' and status = True and destination = '%u'
iterate_query = SELECT "username" as user, domain FROM email_users WHERE
status = True and alias_flag = False
I can help further if needbe (more complete info), just trying to point
you in the right direction.
Happy Thursday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 7/5/2023 11:26 PM, jeremy ardley via dovecot wrote:
On 6/7/23 10:17, joe a wrote:
Greetings from a new dovecot user.
Have setup dovecot on openSuse 15.4 with postfix as the MTA. Both are
the latest version in that distribution.
Simple virtual user setup using /etc/dovecot/passwd
Dovecot seems to be working and all the defined users are
authenticating well enough for imapsync to migrate files to the
mailboxes.
However, when attempting to send test mail via postfix, only some
users are authenticated and have mail delivered. Using swaks (smtp
toolkit) the failures are: 550 5.1.1 :
Recipient address rejected: User unknown in local recipient table
I'm puzzled, probably some simple thing overlooked. To avoid clutter,
I won't include postfix items unless asked.
dovecot --version 2.3.20 (80a5ac675d)
dovecot -n
# 2.3.20 (80a5ac675d): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.20 (149edcf2)
# OS: Linux 5.14.21-150400.24.66-default x86_64
# Hostname: flitch
auth_verbose = ob-fuskate
disable_plaintext_auth = no
first_valid_uid = 100
info_log_path = /var/log/dovecot-info.log
log_path = /var/log/dovecot.log
mail_location = maildir:~/Maildir
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
plugin {
sieve = file:~/sieve;active=~/.dovecot.sieve
}
protocols = imap lmtp
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
ssl = no
ssl_cipher_list =
ALL:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:
Re: imap syncing issues
ok just a suggestion Dovecot supports idle to inform clients of message updates this works well with most clients however microsoft outlook & other microsoft email clients do not seem to support idle and have not since outlook2010 Microsoft want everyone using outlook / exchange 365 ?? I have not tried windows mail (sorry no need for using that) I do however find that thunderbird on windows is the best client out there. If you need to use microsoft email clients then you will need to set the fetch email intervals to like 5 minutes or whatever is good for you, most microsoft is set for 30 minute sync's also note this is also an issue with iphone as email servers need to send push notifications to the apple in order to notify the end user of a new/changed email box. this has been the case since ios 10. again you need to set fetch intervals for stuff to be updated. fyi Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2023-06-14 12:58 p.m., dovecot-boun...@dovecot.org wrote: The issue seems to be with the clients (an update maybe?), not the server. do you have console/root access to dovecot and files? On 6/14/23 11:17, Michael Grant via dovecot wrote: I’m having synchronization issues in imap. I am accessing my mail from several different imap clients: K9 on Android, Windows 11 and 10 mail client, and Android Gmail app. Both desktop and laptop, tablet, phone. I know I have more than the usual number of imap connections... Often when I delete a message in one place, it doesn’t get deleted in another. For example, if I delete a message on K9 then open my laptop, it’s still there in W11 Mail. But just now, I deleted some messages on my laptop and swiped down on K9 and the message disappeared on K9. But K9 shows other messages which have been deleted in Windows 11 Mail. I’ve not yet been able to figure out a pattern. It’s annoying me. I have to delete messages in several different places. Messages not coming back, they’re just not being deleted in one place and that delete operation is not syncing to the others. Messages seem to be being marked as read properly across devices. This seems to be an issue with delete only, so far as I’ve noticed. I’ve long been using multiple imap clients, this syncing issue started maybe 6 to 8 months ago. Is there a good way to get debugging info out of dovecot as to what the clients are doing? Or does anyone have any advise which might help resolve this without resorting to me digging into the imap protocol? Michael Grant ___ dovecot mailing list --dovecot@dovecot.org To unsubscribe send an email todovecot-le...@dovecot.org -- This message has been scanned for viruses and dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is believed to be clean. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Migration dovecot server with local users to dovecot with virtual mailboxes
ok i went through this and the best solution although it is a bit of work
is imapsync
imapsync --host1 --user1 --password1 \
--host2 --user2 --password2 \
--regextrans2 "s,\.,_,g"
see : https://github.com/imapsync/imapsync
note the reg parameter above deal with dot in mailboxes etc
virtual mail requires a database of user data, i made a python script to
step through all the account to sync to the new servers.
It basically deals with the compatibility issues between servers.
Happy Friday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 2023-06-09 6:51 a.m., t Seeger wrote:
Thank you for the link, unfortunately I am missing the crucial clue. I
get the directories backed up, but I don't get them imported
properly all mails are displayed as unseen. i tried with simple
pasting and the doveadm import command. Is there a trick to preserve the
flags? Maybe it is because the old dovecot server uses short username
without the @ and the domain name and the new one uses the mail address
to identify the user.
On Thu, Jun 8, 2023 at 3:26 PM aki.tuomi via dovecot
mailto:dovecot@dovecot.org>> wrote:
Please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/
<https://doc.dovecot.org/admin_manual/migrating_mailboxes/>
Aki
Original message
From: tseeger...@gmail.com <mailto:tseeger...@gmail.com>
Date: 6/8/23 16:10 (GMT+02:00)
To: dovecot@dovecot.org <mailto:dovecot@dovecot.org>
Subject: Migration dovecot server with local users to dovecot with
virtual mailboxes
Hello,
we are using a dovecot server with NIS, we want to transfer it to a
new setup. As backend a mysql server will be used in the future.
My problem is, if I just copy the maildir or use "doveadm import"
all mails are flagged as "unseen". How can i preserve the flags?
This is how I have tried it so far.
root@mail / $ doveadm import -s -u testuser@testdomain.local
maildir:/tmp/testuser-maildir-230608/Maildir/:INDEX=MEMORY "" "All"
As a "workaround" I tried to change the flags.
root@mail / $ doveadm -v flags add -u testuser@testdomain.local
"Seen" "ALL"
This led partially to the desired success, the mails are shown as
"seen", but the folders (e.g. Roundcube) still show unread mails.
dovecot -n
# 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.13 (cdd19fe3)
# OS: Linux 5.10.0-23-cloud-amd64 x86_64 Debian 11.7
# Hostname: mail.testdomain.local
auth_mechanisms = plain login
mail_fsync = always
mail_gid = vmail
mail_home = /var/vmail/mailboxes/%d/%n
mail_location = maildir:~/mail:LAYOUT=fs:INDEX=MEMORY
mail_privileged_group = vmail
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext imapsieve vnd.dovecot.imapsieve
mmap_disable = yes
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Spam {
auto = subscribe
special_use = \Junk
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
plugin {
imapsieve_mailbox1_before =
file:/var/vmail/sieve/global/learn-spam.sieve
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_name = Spam
imapsieve_mailbox2_before =
file:/var/vmail/sieve/global/learn-ham.sieve
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_from = Spam
imapsieve_mailbox2_name = *
quota = maildir:User quota
quota_exceeded_message = User %u has exhausted allowed storage space.
sieve =
file:/var/vmail/sieve/%d/%n/scripts;active=/var/vmail/sieve/%d/%n/active-script.sieve
sieve_before = /var/vmail/sieve/global/spam-global.sieve
sieve_global_extensions = +vnd.dovecot.pipe
sieve_pipe_bin_dir = /usr/bin
sieve_plugins = sieve_imapsieve sieve_extprograms
}
protocols = imap lmtp sieve
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
unix_
Re: replicator service
replication needs to run on both servers you need to assume that a mailbox can change on either server replication keeps them synced in real time Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2023-05-14 5:19 p.m., Przemysław Kwiatkowski via dovecot wrote: I'm confused. Replicator service is bidirectional. Does it mean I should start it only on *one* end of a replication pair? Or on both, working simultaneously? ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Can't seem to setup remote access to doveadmI'm using
Updated : %s' %(self.username) #Send update email send_from = 'moni...@scom.ca' send_files = [] send_to = ['moni...@scom.ca'] send_text = '\n\n'+ send_subject + '\n' sendmail(send_from,send_to,send_subject,send_text,send_files) #Send the warning email if send_settings != '' : #Send Email Setup to this address servername = 'mail.%s' %domain send_from = 'i...@scom.ca' send_files = [] send_to = ['%s' %send_settings,] send_subject = 'Email Setup Instructions for : %s %s' %(emailaddress,send_settings) #Assemblt the sxend text with the info _ Happy Saturday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 3/17/2023 10:22 PM, dovecot-boun...@dovecot.org wrote: I’m running version 2.1.7 under Raspbian Wheezy (and have been for a number of years). I want to allow one of my other computers to remotely issue doveadm commands to my server but can’t seem to find the right way to add an inet listener to permit this. Using ‘doveconf -a’ I’ve found the default service definition for doveadm-server. So I copied that and added it to the dovecot configuration with an inet listener section added to it. But that was rejected as a duplicate service definition when I restarted dovecot. I then tried specifying a “host:port” value for the doveadm_socket_path value but that didn’t work as no listening socket at that port appeared when I restarted dovecot. I can’t seem to find in any of the dovecot documentation a way to do this and there seems to be nothing like a “Here’s how to setup doveadm remote access” section in the documentation (which would be most helpful). How can I get doveadm-server to listen not only locally (as it’s already doing) but also open an inet port for remote access?
Re: creating a mailbox via imap
= data[1]
data = data.split('@')
print 'Deleting Email Account : user/%s@%s' % (
str(data[0]), str(data[1]) )
message = 'BAD'
conn.send(message) # echo
print message
conn.close()
s.close()
#Go Back Around
_
the code is not finished but does create the mbox and waits for it to be
completed before returning ?
it is accessed with this code (see python sockets)
_
imap_test = Dovecot_Command ('INFO',self.username) #do i have
this account ?
if 'BAD' in imap_test.answer :
try : #Try to Create the account, note that the db must be
updated properly before it will work
imap_create = Dovecot_Command ('CM',self.username)
if 'OK' in imap_create.answer :
send_subject = 'Email Account Created : %s'
%(str(self.username) )
except :
send_subject = 'Error Account : %s' %(str(self.username) )
pass
else :
send_subject = 'Email Account Updated : %s' %(self.username)
___
and
___
class Dovecot_Command :
def __init__(self,command,username) :
self.command = command
self.username = username
self.answer = ''
import socket
TCP_IP = '10.220.0.18'
TCP_PORT = 8444
BUFFER_SIZE = 1024
MESSAGE = '%s %s' %(self.command,self.username)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
s.send(MESSAGE)
self.answer = s.recv(BUFFER_SIZE)
s.close()
this is crude code but does get the job done.
I went to this extent to eventually create, get info on the account,
delete etc back into my django admin project (like mbox size, last
accessed etc)
basically everything you need to handle accounts on the serer side.
I also run replication and i think that is what lead to this being a
little more complex. (ie a simple cm imap command was insufficent?)
Happy Thursday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 2023-02-23 12:56 p.m., dovecot-boun...@dovecot.org wrote:
is there any way with Dovecot to open an admin imap connection and
create a brand new mailbox?
With Cyrus imapd I can do this by connecting as the Cyrus admin user
and then create a folder "user/newu...@domain.tld".
Wouldn't that be dependent on how Dovecot auth worker verifies a user
exist and is valid? Such as for one method, database queries. How would
Dovecot know what query to run to add another user to your database?
Plus update any other related DB tables needed for your custom setup? I
would imagine too many edge cases for Dovecot to worry about for
creating accounts.
Re: Redundant Database, Pgsql ?
yes that seems to be the approach i setup a dns entry and pointed to 3 servers it does work round robin (ie from main, secondary etc) but that is ok at least it is working when i take the main server offline for maintenance ! Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2023-02-19 12:56 p.m., Benny Pedersen wrote: Paul Kudla skrev den 2023-02-19 16:01: May I please get some guidance on what to add to talk to another postgresql server (i have 6 replicated servers so would probably want a couple worst case issue) change host=localhost to host=some-other-hostname-with-multiple-ips :) then dovecot with timeout and test next server ip there might be more to it, but i think this is how to do it
Redundant Database, Pgsql ?
I am aware that there are provising for redundant database connections Basically i was working on my main db server (which is also a mail sever) I current have this in the dovecot-pgsql.conf __ driver = pgsql connect = host=localhost port=5433 dbname=scom_billing user=pgsql password=x default_pass_scheme = PLAIN password_query = SELECT username as user, password FROM email_users WHERE username = '%u' and password <> 'alias' and status = True and destination = '%u' user_query = SELECT home, uid, gid FROM email_users WHERE username = '%u' and password <> 'alias' and status = True and destination = '%u' #iterate_query = SELECT user, password FROM email_users WHERE username = '%u' and password <> 'alias' and status = True and destination = '%u' iterate_query = SELECT "username" as user, domain FROM email_users WHERE status = True and alias_flag = False ___ Basically I have replicaed servers all over the place but mail18 (because dovecot only knows of one db instance) went down as well. May I please get some guidance on what to add to talk to another postgresql server (i have 6 replicated servers so would probably want a couple worst case issue) -- Happy Sunday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca
Re: NFS and performances
Good morning
I apologize in advance as you probably don't want to here this.
I have a replicated system and tried to use NFS to a file share server
with dedicated gigabit links etc and my second replicated system.
I have 300+ accounts and many have 20+ gig of data over 600+ folders,
your setup seems larger.
I spent 2 months trying to make this work reliably with nothing working out.
that being said (and this IS NOT a dovecot thing) NFS simply will not
work reliably especially in the environment that you seem to be using
I went to local SDRAM drives on the second server and have had zero
issues since.
NFS tweaks can be done and dovecot does try to support this but Linux
flavors (i use FreeBSD) all seem to handle NFS slightly differently thus
leading to the issues of timeouts, data not so much being dropped but
delayed between the NFS mount points.
NFS inherently on most systems runs a 30 second cache and file locking
for the mailboxes can usually is an issue.
Just easier to use hdd's on any local server.
NFS is good for tar backups etc though.
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 2023-02-15 9:25 a.m., tomate aceite wrote:
Hello, i have some question about NFS, dovecot director, and imap settings.
I was reading all dovecot documentation ad mail lists, but some aspect
are not clear to me.
I am looking for performance / tunning my infra to work in a more
efficient way because we experiences some issues some days ago.
This is my infra:
I got an infra with 2 dovecot-directors and 3 imap backend.
I got all the emails stored in a common NFS share filer to all the imap
nodes. ( Index are locally stored in each imap node.)
My NFS mount options:
(0)#: nfsstat -m
/data/mail from myipaddress:/export/mail/maildirs
Flags:
rw,nosuid,noexec,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.205,mountvers=3,mountport=20048,mountproto=tcp,local_lock=none,addr=10.0.0.205
*Questions*: ( https://wiki.dovecot.org/PerformanceTuning
<https://wiki.dovecot.org/PerformanceTuning> ,
https://doc.dovecot.org/configuration_manual/nfs/
<https://doc.dovecot.org/configuration_manual/nfs/> >> i am following
these steps )
1) Is my NFS correct setup with the mount options well optimized ? Not
sure if someone is using the same flags that me or got a better
recomendation to used.
2) Set *mmap_disable = yes ??? * >> This must be set to yes if you
store indexes to shared filesystems. In my case i got them locally in
each imap node not in NFS share folder.
I got setup mmap_disable = no , is this correct? I think no is the
correct option here with indexes locally.
because i can read here:
https://wiki1.dovecot.org/NFS <https://wiki1.dovecot.org/NFS>
>> High performance NFS setup with indexes on local disk (see below
for benefits):
mmap_disable = no
3) Set*mail_fsync = always ???*
Documentation: https://wiki.dovecot.org/PerformanceTuning
<https://wiki.dovecot.org/PerformanceTuning>
always
Use fsync after all disk writes.
Recommended for NFS to make sure there aren’t any delayed write()s.
3.a) where i can setup this option *mail_fsync = always , *because i
run doveconf -n in director, and imap nodes, and they are not showing
nothing.
3.b) *In which node ? *Do i need to add the setting in dovecot.conf in
*director node or in imap node or in both ?* Not sure if this is the
correct way:
This is an attemp of setup, not sure if is correct?
0)#: doveconf -n
# 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.13 (cdd19fe3)
# OS: Linux 5.10.0-20-amd64 x86_64 Debian 11.6
mail_debug = yes
mail_fsync = always
mail_max_userip_connections = 20
mail_nfs_storage = yes
mail_plugins = " notify mail_log"
mail_privileged_group = mail
protocol lmtp {
mail_fsync = always
mail_plugins = " notify mail_log sieve mail_lua push_notification
push_notification_lua"
plugin {
...
}
4) Do not set *mail_nfs_index *or *mail_nfs_storage* (i.e. keep them as
no) ?
First option make sense but the second one not.
https://doc.dovecot.org/settings/core/#core_setting-mail_nfs_storage
<https://doc.dovecot.org/settings/core/#core_setting-mail_nfs_storage>
mail_nfs_storage
Default: no
Values: Boolean
Flush NFS caches whenever it is necessary to do so.
This setting should only be enabled if you are using multiple
servers on NFS.
So should be possible to enable this option *mail_nfs_stora
Re: [SOLVED] Pigeonhole Sieve Vacation Reply-To peculiarity with inbound AWS-SES
Happy Saturday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2/11/2023 8:12 AM, Dr. Rolf Jansen wrote: Spare your breath. I have solved my issue AWS-SES, and it behaves well now with Pigeonhole Sieve Vacation (s. patch attached) Many thanks for all your thoughts. I will leave the list now. Best regards Rolf Am 11.02.2023 um 09:01 schrieb Paul Kudla : Ok again just trying to help ___ The question on why I use AWS-SES as my outbound mail relays can be simply answered with the attribute „superior reputation“. ___ that being said, again an experience thing that most people do not know about ! opensrs (i use them for my domain registration thus i had a wholesale account setup and could interact with tech support on other issues, this being an example of one.) that being said reputations are mostly purchased now a days, people do not block server's based on reputation that in most cases is actually paid for. For example years ago I had a customer receive an email from a supplier in china Suppliers MUST have a bank transfer etc before they will ship My customer lost 15000.00 us in a bogus transfer because opensrs's email servers were on a spf whitelist? What can i say experience, spf is designed to prevent spam emails but more so verify that they came from an authorized server. Believe it or not, the supplier got hacked, the hacker setup a duplicate email with the same email address on an opensrs server. SPF would have caught it except opensrs's email server are whitelisted ! Customer lost the money, unable to recover and opensrs denied any responsibilty for paying to be whitelisted. My SPF system is now patched to skip any whitelist via SPF as it functions as it should now. Microsoft, Google etc are also other culprites on bypassing things in the name of saving some bandwidth. Anything within there systems are generally automatically whitelisted, Again another customer, they are on Outlook 365, I received an email that said our domains were suspended etc, nothing new there get those all the time, the worrisom part was someone setup an email server, then proxied through microsoft in a way that was very clever, had an spf record and everything setup, but they were using microsoft as a proxy to a microsoft account so the mail got delivered when again it should have bounced back as invalid sender. I understand this is not directly related but reputations are paid for and relays will never fully work upstream as it is dependant on what the upstream provider changes from time to time Its a cat and mouse game that will never end. Again just trying to help. Happy Saturday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2/10/2023 9:27 AM, Dr. Rolf Jansen wrote: As stated elsewhere, the severe problem of incomprehensible OoO notice comes not because I relay MY outbound mails via Amazon’s SES but because some of MY PEERS (senders of the original messages and receivers of OoO notices) do or perhaps other relays which do funny manipulations of envelope sender and some headers in the message body as well. That said, my usage of AWS-SES may probably raise similar problems to the receivers of our mails wanting to return OoO notices to our users. The question on why I use AWS-SES as my outbound mail relays can be simply answered with the attribute „superior reputation“. My experience is that SES is blocked nowhere, except perhaps in North Korea, I didn’t try yet. For professional emails this is mission critical, and you cannot even get close to this if you setup somewhere, somehow your best practice own relay. This reputation has of course to do with SES controlling bounces. SES does control outgoing rate. SES does control the domain of the sender's address (envelop and From:) has been registered with the service. They do everything that SES ist not being compromised by any criminals. For me this is important, and then I need to live with the peculiarities and annoyances and perhaps find workarounds. Best regards Rolf Am 10.02.2023 um 10:30 schrieb Paul Kudla : Good morning, I have been following this post for a bit and would like to share experience please and thanks. This is not meant to give a solution but save some massive frustration with other system as i have gone through the same issues overall. In general I found found over the past few years all the big boys are forcing all the private systems into standards that are not really defined and get implemented willy nilly. Just because microsoft starts a standard, then google picks up on it then AWS and then yahoo etc etc
Re: Pigeonhole Sieve Vacation Reply-To peculiarity with inbound AWS-SES
Ok again just trying to help ___ The question on why I use AWS-SES as my outbound mail relays can be simply answered with the attribute „superior reputation“. ___ that being said, again an experience thing that most people do not know about ! opensrs (i use them for my domain registration thus i had a wholesale account setup and could interact with tech support on other issues, this being an example of one.) that being said reputations are mostly purchased now a days, people do not block server's based on reputation that in most cases is actually paid for. For example years ago I had a customer receive an email from a supplier in china Suppliers MUST have a bank transfer etc before they will ship My customer lost 15000.00 us in a bogus transfer because opensrs's email servers were on a spf whitelist? What can i say experience, spf is designed to prevent spam emails but more so verify that they came from an authorized server. Believe it or not, the supplier got hacked, the hacker setup a duplicate email with the same email address on an opensrs server. SPF would have caught it except opensrs's email server are whitelisted ! Customer lost the money, unable to recover and opensrs denied any responsibilty for paying to be whitelisted. My SPF system is now patched to skip any whitelist via SPF as it functions as it should now. Microsoft, Google etc are also other culprites on bypassing things in the name of saving some bandwidth. Anything within there systems are generally automatically whitelisted, Again another customer, they are on Outlook 365, I received an email that said our domains were suspended etc, nothing new there get those all the time, the worrisom part was someone setup an email server, then proxied through microsoft in a way that was very clever, had an spf record and everything setup, but they were using microsoft as a proxy to a microsoft account so the mail got delivered when again it should have bounced back as invalid sender. I understand this is not directly related but reputations are paid for and relays will never fully work upstream as it is dependant on what the upstream provider changes from time to time Its a cat and mouse game that will never end. Again just trying to help. Happy Saturday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2/10/2023 9:27 AM, Dr. Rolf Jansen wrote: As stated elsewhere, the severe problem of incomprehensible OoO notice comes not because I relay MY outbound mails via Amazon’s SES but because some of MY PEERS (senders of the original messages and receivers of OoO notices) do or perhaps other relays which do funny manipulations of envelope sender and some headers in the message body as well. That said, my usage of AWS-SES may probably raise similar problems to the receivers of our mails wanting to return OoO notices to our users. The question on why I use AWS-SES as my outbound mail relays can be simply answered with the attribute „superior reputation“. My experience is that SES is blocked nowhere, except perhaps in North Korea, I didn’t try yet. For professional emails this is mission critical, and you cannot even get close to this if you setup somewhere, somehow your best practice own relay. This reputation has of course to do with SES controlling bounces. SES does control outgoing rate. SES does control the domain of the sender's address (envelop and From:) has been registered with the service. They do everything that SES ist not being compromised by any criminals. For me this is important, and then I need to live with the peculiarities and annoyances and perhaps find workarounds. Best regards Rolf Am 10.02.2023 um 10:30 schrieb Paul Kudla : Good morning, I have been following this post for a bit and would like to share experience please and thanks. This is not meant to give a solution but save some massive frustration with other system as i have gone through the same issues overall. In general I found found over the past few years all the big boys are forcing all the private systems into standards that are not really defined and get implemented willy nilly. Just because microsoft starts a standard, then google picks up on it then AWS and then yahoo etc etc in any order does not mean its a proper approach. That being said is there any reason why you are not sending the emails directly yourself, ie why are you using a proxy. I found (for example) when forwarding an email from @scom.ca to gmail for example all the headers, dkim, spf records are all passed along which resulted in emails never being allowed to be delivered. Although this may be your issue directly or indirectly what i found is to forward to a gmail.com account i had to program the gmail.com account to pop my
Re: Pigeonhole Sieve Vacation Reply-To peculiarity with inbound AWS-SES
Good morning, I have been following this post for a bit and would like to share experience please and thanks. This is not meant to give a solution but save some massive frustration with other system as i have gone through the same issues overall. In general I found found over the past few years all the big boys are forcing all the private systems into standards that are not really defined and get implemented willy nilly. Just because microsoft starts a standard, then google picks up on it then AWS and then yahoo etc etc in any order does not mean its a proper approach. That being said is there any reason why you are not sending the emails directly yourself, ie why are you using a proxy. I found (for example) when forwarding an email from @scom.ca to gmail for example all the headers, dkim, spf records are all passed along which resulted in emails never being allowed to be delivered. Although this may be your issue directly or indirectly what i found is to forward to a gmail.com account i had to program the gmail.com account to pop my server. This does work well but only for gmail.com I have other customers where i try to pop the email from whatever system (which does work) but when i forward to an account on my system postfix rewrite the header from address to the mailxx.scom.ca email server name being used to forward the email which generates the same issues you are having in the headers being rewritten not showing the from address? My server's are setup with custom python programming filters developed over ten years and i can not seem to control anything either? I get you do production stuff (so do my customers) which is why it might be better to send via a postfix instance that you are in control of of couse this does require a static ip etc which i dont know if you have access to or not? but i think this would save a lot of frustration trying to be "COMPATIBLE" with everyone else out there that do not even follow their own standards? Just though i would pass this info along, trying to help ? Happy Friday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2/10/2023 7:18 AM, Dr. Rolf Jansen wrote: Am 08.02.2023 um 20:03 schrieb Michael Peddemors : Dovecot vacation message issues.. Tough for any system to do correctly. The problem here is that inbound mails from third parties utilizing AWS-SES come in with an unpersonalized envelope address and SES takes returns to this as bounce messages and changes the body's From: to „mailer-dae...@xx--1.amazonses.com“, which is not even our MAILER-DAEMON but the one of the receiver of our reply. So the receiver gets no chance to know from the headers the identity of whom replied - he may assume it from the context the actual message, though. We addressed this by NOT returning vacation messages to systems that don't use 'proper' values in the MAIL FROM.. Eg Mailing Lists, Sender Rewrite schemes, and a slurry of other rules. Who is we? Your organization or the Pigeonhole developers? Actually, the question is, whether this is addressed somewhere in Pigeonhole’s code already? But the problem is that if you are using the header From, or Reply-To etc, it's too easy to be sending to forged email addresses. Vacation bombing attacks for instance.. You got a point here, and of course I want to prevent this. Now, there are legitimate cases of the MAIL FROM and header from not aligning, so it is best to send to the MAIL FROM addresses.. IF you don't send it to certain MAIL FROM formats, usually by not responding to anything with mailing list identifiers, auto-suppress headers, and a few others, you only end up with clean MAIL FROM to respond to. From the point of the view of our industrial customers, who are operating processes with our chemicals, this consideration is irrelevant. If they inform a production issue by mail to the responsible service technician, they expect an immediate response, since a production stop is unacceptable. OoO notices play a role here, because we would inform alternative addresses and fone numbers for attending the support case. That said, with Pigeonhole, we are almost there. But if you have an example that is particularly bothering you, and represents your problem, we can walk through that as an example. I send an email from an account of a mail server (Postfix/Dovecot - outbound relay SES) running on an AWS-EC2 instance in São Paulo (Brazil) to another mail address of mine of a mail server (Postfix/Dovecot direct MX) on an AWS-EC2 instance in Frankfurt Germany, and here the Pigeonhole’s vacation reply is activated. In the following I changed my real mail address in Brazil to r...@example.br and the real one in Germany to r...@example.de: The Point of
Re: IMAP tuning for Outlook 365
Ok sorry to NOT have an answer you are looking for but I don't want anyone wasting their time on what is clearly a microsoft issue. Outlook 365 / Outlook since 2010 simply does not support imap. You can tweek it here and there however here are the basic's Outlook IMAP does not support IDLE thus no auto upating of the inbox and other folder changes etc... Outlook (this if from experience) simply does not handle large anything folders well, especially imap - hence the click at the bottom of a folder to see more (ie history) ?? I had a customer spending 700+ / mth to telus because people were leaving emails behind (out of sight out of mind) yes that was per month ! that being said the best you can do with outlook is set to auto update the send/receive settings to every 5 minutes or so A 3G email box is an issue no matter what as the size is the direct issue of the lockups, and/or the quantity of emails in the folder Outlook seems to work ok until about 200 emails (max 1000) in a folder and maybe 1G of actual data. Yes you can move emails to other folders, but the folder will still try to sync in the background and will eventually lock up pending the size. I use thunderbird and although it has its short commings, it handles large email boxes extremely well, handles IMAP IDLE protocol other imap clients (apple ios mail - iphone, windows mail - the free one (suprised me) ) I am running 26G of email, 150+ folders etc etc (some folders aproach 1 emails - archived stuff) and no real issues on a replicated dovecot 2.3.19 server setup. What needs to be kept in mind is that Microsoft (espically 365) is trying to push exchange protcol and has all but dropped support for IMAP. Again open to comments but i recently was forced to use Outlook (Office 365) for a customer and ran into all of these issues yet again, i am weeding them off of Outlook in favor of thunderbird. FYI Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 1/31/2023 10:07 AM, Artur Pydo wrote: Hello, I'm looking for advices on IMAP config tuning for best user experience with Outlook 365. I'm currently using dovecot 2.3.4.1 (f79e8e7e4) provided with Debian 10. One of my users has Outlook 365 and an IMAP mailbox of large size with several folders (more than 3GB). From time to time as his main inbox folder is growing he can see some problems with its Outlook application. Sometimes it freezes during mailbox synchronisation, sometimes he can't see new incoming emails. I wonder if there is some specific workarounds for an up-to-date Outlook in Dovecot or if you can suggests some config setup adjustments (timeouts, idleing ...) to avoid problems in his Outlook. Most of the config are defaults. But I can provide relevant parts of the config if needed. I have this workaround activated for a long time: imap_client_workarounds = delay-newmail By the way, I also have a big IMAP mailbox. Can't see any server specific problem while connecting with Thunderbird to this Dovecot server. -- Best regards, Artur -- This message has been scanned for viruses and dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is believed to be clean.
Re: replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
ok a little bit more info
My servers all run under FreeBSD 12.xx
which was also the base for the apple operating system origionally.
setting default_vsz_limit = 0 i seem to remember trying with no so great
results
setting to zero can cause memory over runs (espically with replication)
etc i found that when i used the config i sent eariler (vsz_limit is
defaulted (not set )) everything worked
I generally find that tweaking the memory alloted to the individual
services a more balanced approach but it does take a lot of trial and error
also note memory in the system is also a factor my mail servers have 32G
dedicated to them which is what the settings were based on that seem to
work pretty good at the moment.
I am running without any setting thus the default
I got this info from :
https://doc.dovecot.org/configuration_manual/service_configuration/
vsz_limit
Limit the process’s address space (both RLIMIT_DATA and RLIMIT_AS if
available). When the space is reached, some memory allocations may start
failing with “Out of memory”, or the kernel may kill the process with
signal 9. This setting is mainly intended to prevent memory leaks from
eating up all of the memory, but there can be also legitimate reasons
why the process reaches this limit. For example a huge mailbox may not
be accessed if this limit is too low. The default value
(18446744073709551615=2^64-1) sets the limit to default_vsz_limit, while
0 disables the limit entirely.
There are 3 types of services that need to be optimized in different ways:
Master services (e.g. auth, anvil, indexer, director, log):
Currently there isn’t any easy way to optimize these. If these
become a bottleneck, typically you need to run another Dovecot server.
In some cases it may be possible to create multiple master processes and
have each one be responsible for only specific users/processes, although
this may also require some extra development.
Services that do disk I/O or other blocking operations (e.g. imap,
pop3, lmtp):
These should have client_limit=1, because any blocking
operation will block all the other clients and cause unnecessary delays
and even timeouts. This means that process_limit specifies the maximum
number of available parallel connections.
Services that have no blocking operations (e.g. imap-login,
pop3-login):
For best performance (but a bit less safety), these should have
process_limit and process_min_avail set to the number of CPU cores, so
each CPU will be busy serving the process but without unnecessary
context switches. Then client_limit needs to be set high enough to be
able to serve all the needed connections (max connections=process_limit
* client_limit). service_count is commonly set to unlimited (0) for
these services. Otherwise when the service_count is beginning to be
reached, the total number of available connections will shrink. With
very bad luck that could mean that all the processes are simply waiting
for the existing connections to die away before the process can die and
a new one can be created. Although this could be made less likely by
setting process_limit higher than process_min_avail, but that’s still
not a guarantee since each process could get a very long running
connection and the process_limit would be eventually reached.
Happy Saturday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 1/6/2023 5:20 PM, Gerben Wierda wrote:
How problematic is it to have
default_vsz_limit = 0
in dovecot.conf? macOS+MacPorts had this as a requirement even.
Gerben
On 6 Jan 2023, at 16:49, Paul Kudla <mailto:p...@scom.ca>> wrote:
i ran into this as well
here is the full config for mine with replication
# cat dovecot.conf
# 2.3.14 (cee3cbc0d): /usr/local/etc/dovecot/dovecot.conf
# OS: FreeBSD 12.1-RELEASE amd64
# Hostname: mail18.scom.ca <http://mail18.scom.ca>
auth_debug = no
auth_debug_passwords = no
default_process_limit = 16384
mail_debug = no
#lock_method = dotlock
#mail_max_lock_timeout = 300s
#mbox_read_locks = dotlock
#mbox_write_locks = dotlock
mmap_disable = yes
dotlock_use_excl = no
mail_fsync = always
mail_nfs_storage = no
mail_nfs_index = no
auth_mechanisms = plain login
auth_verbose = yes
base_dir = /data/dovecot/run/
debug_log_path = syslog
disable_plaintext_auth = no
dsync_features = empty-header-workaround
info_log_path = syslog
login_greeting = SCOM.CA Internet Services Inc. - Dovecot ready
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
mail_location = maildir:~/
mail_plugins = " virtual notify replication fts fts_lucene "
mail_prefetch_count = 20
protocols = imap pop3 lmtp sieve
protocol lmtp {
mail_plugins = $mail_plugins sieve
postmaster_address =
Re: replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
process_limit = 1000
client_limit = 100
vsz_limit = 512m
}
service imap-urlauth-login {
process_limit = 1000
client_limit = 1000
vsz_limit = 1g
}
service imap-login {
process_limit=1000
client_limit = 1000
vsz_limit = 1g
}
protocol sieve {
managesieve_implementation_string = Dovecot Pigeonhole
managesieve_max_line_length = 65536
}
#Addition ssl config
!include sni.conf
with sni cert support (examples)
# cat sni.conf
#sni.conf
ssl = yes
verbose_ssl = yes
ssl_dh =password_query = SELECT username as user, password FROM email_users
WHERE username = '%u' and password <> 'alias' and status = True and
destination = '%u'
user_query = SELECT home, uid, gid FROM email_users WHERE username =
'%u' and password <> 'alias' and status = True and destination = '%u'
#iterate_query = SELECT user, password FROM email_users WHERE username =
'%u' and password <> 'alias' and status = True and destination = '%u'
iterate_query = SELECT "username" as user, domain FROM email_users WHERE
status = True and alias_flag = False
Happy Friday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 1/6/2023 5:32 AM, Gerben Wierda wrote:
On 6 Jan 2023, at 08:53, Aki Tuomi <mailto:aki.tu...@open-xchange.com>> wrote:
On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda
mailto:gerben.wie...@rna.nl>> wrote:
One step further in my quest to create a replacement mail server.
I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new
(2.3.20, Alpine Linux, Docker, apk package). When I turn on
replication it works, but, after a while I see:
Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when
allocating 268435496 bytes
Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child
133 killed with signal 6 (core dumped)
Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning:
replication(sysbh): Sync failure:
Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning:
replication(sysbh): Remote sent invalid input: -
I've removed synchronous operation for now (found a message on the
net suggesting that) but is this known and what does it mean?
Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda
<https://www.linkedin.com/in/gerbenwierda>>)
R IT Strategy <https://ea.rna.nl/ <https://ea.rna.nl/>> (main site)
Book: Chess and the Art of Enterprise Architecture
<https://ea.rna.nl/the-book/ <https://ea.rna.nl/the-book/>>
Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/
<https://ea.rna.nl/the-book-edition-iii/>>
Dovecot default memory limit is 256M. You should probably set
service replicator {
vsz_limit = 2G
}
because replicator might have to use more memory, especially for
larger indexes.
Aki
That is a good tip as well.
I had followed this bit of experience from someone else:
https://marc.info/?l=dovecot=164438199727640
<https://marc.info/?l=dovecot=164438199727640>, haven't seen any err
message since. But that might be because they are in sync now and both
sides are aware. Can I trigger full replication again so I can test?
Gerben
--
This message has been scanned for viruses and
dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
believed to be clean.
Re: Migrating, syncing, maybe load-balancing/failover two dovecot servers?
ok just a few quick things about replication 1. you should upgrade both versions to at least dovecot-2.3.19.1.tar.gz (2.3.18 had issues on larges folder counts - you will probably run into this on smaller servers but just sharing the experience) 2. i found replication worked better without using ssl 3. i went through the sync failures etc as well and found that NOT using NFS etc is the way to go 4. I can provide (or if you look on the mailing lists) my config for SCOM - it took a month of tweeking but finally got a good config that worked. 5. One thing i just remembered that you really should run a pgsql database for user auth, this way the two system will stay up to date automatically everytime an email box is modified. The replicator service selects users from a database to keep the mbox's in sync automatically the above are the basics but i find dovecot runs extremely well vs cyrus that i was running previous Good job to the designers ! Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 1/4/2023 4:24 PM, Gerben Wierda wrote: So, I did set it up. As I am using not real users (but a cram md5 passwd db file with every user uid=dovecot, gid=mail) and my dovecots are owning everything in the mail store I had to synchronise uid/gid of the dovecots on both ends After I did that, I tested the sync. And while it has worked (I now have an equal sized store at both ends), one side (running 2.3.17, the sending 'old server') was throwing up quite a bit of this: Jan 04 20:13:15 doveadm(74435): Error: write() failed: Timed out after 60 seconds Jan 04 20:13:15 doveadm(74435): Panic: file ioloop.c: line 865 (io_loop_destroy): assertion failed: (ioloop == current_ioloop) Jan 04 20:13:15 doveadm(74435): Error: Raw backtrace: 0 libdovecot.0.dylib 0x00010db6d157 backtrace_append + 58 -> 1 libdovecot.0.dylib 0x00010db6d255 backtrace_get + 31 -> 2 libdovecot.0.dylib 0x00010db79ff3 default_fatal_finish + 60 -> 3 libdovecot.0.dylib 0x00010db78afa default_error_handler + 0 -> 4 libdovecot.0.dylib 0x00010db7973b i_internal_error_handler + 0 -> 5 libdovecot.0.dylib 0x00010db78c Jan 04 20:13:15 doveadm(74435): Error: b8 i_fatal + 0 -> 6 libdovecot.0.dylib 0x00010db8fa1f io_loop_destroy + 826 -> 7 doveadm-server 0x00010d3445fc doveadm_print_server_flush + 254 -> 8 doveadm-server 0x00010d33df1e doveadm_print + 44 -> 9 doveadm-server 0x00010d32bd5b cmd_dsync_run + 1618 -> 10 doveadm-server 0x00010d32db67 doveadm_mail_next_user + 479 -> 11 doveadm-server 0x00010 Jan 04 20:13:15 doveadm(74435): Error: d32e8bb doveadm_cmd_ver2_to_mail_cmd_wrapper + 2439 -> 12 doveadm-server 0x00010d33dc0c doveadm_cmd_run_ver2 + 1083 -> 13 doveadm-server 0x00010d34224a client_connection_tcp_input + 1579 -> 14 libdovecot.0.dylib 0x00010db8efe1 io_loop_call_io + 114 -> 15 libdovecot.0.dylib 0x00010db910cf io_loop_handler_run_internal + 314 -> 16 libdovecot.0.dylib 0x00010db8f3fb io_loop_handler_run + Jan 04 20:13:15 doveadm(74435): Error: 212 -> 17 libdovecot.0.dylib 0x00010db8f2e6 io_loop_run + 81 -> 18 libdovecot.0.dylib 0x00010db075e0 master_service_run + 24 -> 19 doveadm-server 0x00010d344c3f main + 292 -> 20 dyld 0x00011c73952e start + 462 Jan 04 20:13:15 doveadm(74435): Fatal: master: service(doveadm): child 74435 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps <https://dovecot.org/bugreport.html#coredumps>) Jan 04 20:16:05 lmtp(pid 74518 user gerben): Warning: replication(gerben): Sync failure: Timeout in 2 secs Jan 04 20:17:05 doveadm(74522): Error: write() failed: Timed out after 60 seconds Jan 04 20:17:05 doveadm(74522): Panic: file ioloop.c: line 865 (io_loop_destroy): assertion failed: (ioloop == current_ioloop) Jan 04 20:17:05 doveadm(74522): Error: Raw backtrace: 0 libdovecot.0.dylib 0x0001050d3157 backtrace_append + 58 -> 1 libdovecot.0.dylib 0x0001050d3255 backtrace_get + 31 -> 2 libdovecot.0.dylib 0x0001050dfff3 default_fatal_finish + 60 -> 3 libdovecot.0.dylib 0x0001050deafa default_error_handler + 0 -> 4 libdovecot.0.dylib
Re: Migrating, syncing, maybe load-balancing/failover two dovecot servers?
maybe look a replicator / replication its designed to do exactly that Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 1/4/2023 7:46 AM, Gerben Wierda wrote: I am in the process of migrating from dovecot on one OS (macOS/darwin) to a new server running dovecot with another OS (Ubuntu Linux 22.4). I have mostly copied/adapted the setup of the old server to the new. I am in the process of finishing that and adding some stuff that still needs to be added/migrated, like rspamd. And the data of course before the new one takes over from the old. I have done a migration before (MacOS X Server dovecot to MacPorts dovecot on macOS), many years ago, I recall that I used dovecot syncing but also rsync and I don't really recall (and anyway, the software has changed since) I have been thinking about keeping them both alive, with one as a failover for the other. They will not share their storage (e.g. NFS), So, I was wondering if I can do something with syncing between instances and dovecot director. I have been looking at the documentation, but a quick scan reveals I cannot locate some sort of tutorial and I am uncertain what will work and what not. If keeping both alive in parallel is too problematic, it is OK to have regular syncing in one direction (old to new) at first and then switch over and have syncing in the other direction (new to old) Can someone enlighten me? Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) R IT Strategy <https://ea.rna.nl/> (main site) Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> -- This message has been scanned for viruses and dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is believed to be clean.
Happy Holidays From SCOM.CA Internet Services Inc.
Wishing you and your Family ... Paul Kudla 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca
Re: OT: Two simultaneous POP3 connections from Outlook
Good morning this is probably not want you are looking for Couple of issues even if you get around the outlook locking issue (unless you are leaving on the server but would be flagged that way anyways) pop3 is first come first serve. Outlook when it comes to imap & pop3 its been my experience that microsoft just does not support these properly anymore as in favor for the exchange protocols. Outlook 2010 was the last version that kinda worked. Also Outlook does not support idle for imap making outlook for outlook and other clients (like thunderbird) more practical. for example in the case of imap outlook on large email boxes will just cycle forever on a folder sync. Completly useless. Just some experience to share with you. Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 2022-12-20 9:53 p.m., dovecot-boun...@dovecot.org wrote: Hello, I recognize this may be somewhat off-topic, but my search-fu has failed to provide useful results, so I ask for pointers to further information. Today, while examining Dovecot logs, I found a case where the Outlook mail client (probably 2016 or later) on a user's computer opened two simultaneous, or at least overlapping, POP3 connections to Dovecot. This did not go well, naturally. Warning: Transaction log file .../mail/.imap/INBOX/dovecot.index.log was locked for 156 seconds (rotating while syncing) If anyone else has found useful information about this peculiar behavior for an email client, I would be pleased to receive a pointer in that direction. Thank you. Ken
Re: "Mailbox isn't a valid MBOX file" Error
question how long is the folder path Thunderbird has a path limit that can kick in around 192 characters and will report different errors? Outlook does not seem to have this issue see : Do not reply to this email. You can add comments to this bug at https://bugzilla.mozilla.org/show_bug.cgi?id=1781789 Thomas D. (:thomas8) changed Bug 1781789 at 2022-07-27 07:09:47 PDT: WhatRemoved Added CC bugzilla2...@duellmann24.net Component Untriaged OS Integration Severity-- S4 Summary folder length too long Problems after renaming a folder when folder name is very long (> 192 characters) Product/Component: Thunderbird :: OS Integration Happy Sunday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 11/6/2022 5:07 AM, Elise wrote: Running currently two local mailclients: Outlook 2021 and Thunderbird 102.4.2 When using TB and remove a message on my main account, I get this error: [CANNOT] Mailbox isn't a valid mailbox On any other TB configured email accounts I can remove messages without any problems. First I thought that Outlook might block removal on messages, but the issue appears as well if I close that mail client. Can you tell what is causing this problem? Best regards, Jos Chrispijn -- This message has been scanned for viruses and dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is believed to be clean.
Re: doveadm backup|sync works for every folder but INBOX
hopefully this is NOT off topic
I too migrated from cyrus imap server, it was to say at least fun
you need to get the dovecot server up and running and testing with
making a new account and get the folder structures working dependant on
your required setups, as what seems to be indicated elsewhere in these
posts (namespace, virtual users, dbpassword etc)
if you intend to run replication then both servers need to be setup and
running fully before migrating. Don't put that off it just adds work
afterwards.
at the end of the day
moving from cyrus to dovecot i used imapsync
which is a perl script to migrate, copy etc
see : https://imapsync.lamiral.info/
the one major note is the seperator, if you used '.' on cyrus you need
to convert it to '-' or something else as those folders will not migrate
into a default dovecot configuration and will get skipped.
ie : folders can not have a '.' in them on default dovecot.
Otherwise imapsync pretty much does the trick.
basically :
imapsync --host1 69.49.101.233 --user1 a...@unitedelevatorltd.com
--password1 Password \
--host2 mail18.scom.ca --user2 a...@unitedelevatorltd.com --password2
Password \
--regextrans2 "s,\.,_,g"
note the regextrans2 expresion above handles the '.' to '-'
there is extensive documentation on the site i have just listed what i
had to use to get an account to work and move over with out any data loss.
basically if you write a script to do the copy's on a users account per
mailbox basis then things should migrate well.
ie :
get user list / passwords from old cyrus
then create mailbox on new dovecot server (ie update password database etc)
then run imapsync like above.
repeat for every user you want to migrate.
I did not have any issues with the INBOX but after a month of testing
found this was the best way to go.
Happy Thursday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 11/3/2022 5:54 AM, Aki Tuomi wrote:
On 03/11/2022 11:46 EET Ralf Becker wrote:
Hi Aki,
Am 03.11.22 um 10:29 schrieb Aki Tuomi:
On 03/11/2022 11:27 EET Ralf Becker wrote:
Hi Aki,
Am 03.11.22 um 09:12 schrieb Aki Tuomi:
On 03/11/2022 10:09 EET Ralf Becker wrote:
Hi Aki,
Am 03.11.22 um 08:50 schrieb Aki Tuomi:
On 03/11/2022 09:46 EET Ralf Becker wrote:
I'm trying to migrate an old Cyrus 2.5 server to Dovecot 2.3.19 using
doveadm backup -R, which works for all folders but the INBOX itself,
which always stays empty.
The Cyrus side uses altnamespace:no and unixhierarchysep:no, it's used
as imapc: remote in doveadm backup -R with imapc_list_prefix=INBOX
Dovecot uses the following namespace to migrate into:
namespace inboxes {
inbox = yes
location =
mailbox Sent {
auto = subscribe
special_use = \Sent
}
### some more folders omitted ###
prefix = INBOX/
separator = /
subscriptions = no
}
Hi!
When syncing mailboxes from other server, you should use migration config file,
which has **no** auto=subscribe or auto=create folders, as these can mess up
with synchronization.
Please see https://doc.dovecot.org/admin_manual/migrating_mailboxes/ for more
details.
Does a migration config file specified with doveadm -c add to and
overwrite the existing Dovecot configuration for the time the command
runs, like the -o options, or do I need to start a separate server with
a full configuration to e.g. have my authentication and mailbox location
available?
Ralf
It does not add/replace/overwrite configuration, you provide a fresh config
file which is used *instead of* the default dovecot.conf.
You don't need to run a separate instance necessarely, although in some larger
migrations this has been used as well.
I created now a separate instance with a modified configuration file
with no auto=subscribe (or create), no replication and an empty storage.
doveadm config -n is attached.
Unfortunately the result is identical to my previous tries:
doveadm -o namespace/subs/location=mbox:/var/dovecot/subs -o
imapc_user='someuser' -o imapc_password='secret' -D backup -n INBOX/ -R
-u someuser@somedomain imapc: 2>&1 | tee /tmp/doveadm-backup.log
Nov 03 09:06:35 dsync(someuser@somedomain): Warning: Mailbox changes
caused a desync. You may want to run dsync again: Remote lost mailbox
GUID c92f64f79f0d1ed01e6d5b314f04886c (maybe it was just deleted?)
doveadm mailbox status -u someuser@somedomain all INBOX
INBOX messages=0 recent=0 uidnext=1 uidvalidity=1577952633 unseen=0
highestmodseq=1 vsize=0 guid=c92f64f79f0d1ed01e6d5b314f04886c
firstsaved=never
Any ideas what else to try or how to debug that further?
I can send you the full log to your personal address, if that helps ...
Ralf
You should rm -rf the target folder first. Can you attach `d
Re: how to configure imapsieve to be used per user
ok so are you good to go??? Happy Friday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 10/28/2022 2:13 AM, Sebastian Bachmann wrote: Okay, I could have enabled debug log earlier, than that would have been easy... Thanks for the hint. The key is to specify the sieve script inside the sieve directory without the .sieve suffix, i.e.: a SETMETADATA test (/shared/imapsieve/script "imap") which points to sieve/imap.sieve in the user's homedir. Now it loads the script: Debug: Mailbox test: Mailbox opened because: UID move Debug: imapsieve: mailbox test: MOVE event Debug: imapsieve: mailbox test: Mailbox attribute /shared/imapsieve/script points to Sieve script `imap' Debug: sieve: file script: Opened script `imap' from `/srv/vmail/username/sieve/imap.sieve' Debug: sieve: Opening script 1 of 1 from `/srv/vmail/username/sieve/imap.sieve' Debug: sieve: Loading script /srv/vmail/username/sieve/imap.sieve Debug: sieve: Script `imap' from /srv/vmail/username/sieve/imap.sieve successfully compiled Debug: sieve: Executing script from `/srv/vmail/username/sieve/imap.sieve' On 27.10.2022 23:33, Paul Kudla wrote: ok fair enuff are you using a db to set the dir's there is a master sieve (all) directory that handles the entire server (message duplicate supression etc) mine is in [17:26:12] mail18.scom.ca [root:0] /usr/local/etc/dovecot/sieve # ll total 38 drwxr-xr-x 2 vmail vmail uarch 4B Apr 2 2022 . drwxr-xr-x 5 root wheel uarch 29B Oct 27 07:41 .. -rw-r--r-- 1 vmail vmail uarch 97B Apr 2 2022 duplicates.sieve -rw-r--r-- 1 vmail vmail uarch 227B Apr 2 2022 duplicates.svbin [17:26:17] mail18.scom.ca [root:0] /usr/local/etc/dovecot/sieve from there each user (assuming dovecot config is correct will have it's own sieve folder under the maildir example : # mbox ab...@scom.ca [17:27:24] mail18.scom.ca [root:0] /data/dovecot/users/scom.ca/ab...@scom.ca # ll total 293 drwx-- 11 vmail vmail uarch 25B Oct 27 16:48 . drwx-- 164 vmail vmail uarch 164B Oct 27 06:52 .. drwx-- 5 vmail vmail uarch 8B Oct 27 16:48 .Drafts drwx-- 5 vmail vmail uarch 8B Oct 27 16:48 .Sent drwx-- 5 vmail vmail uarch 8B Oct 27 16:48 .Trash -rw--- 1 vmail vmail uarch 1.4K Oct 26 16:49 .dovecot.lda-dupes drwx-- 5 vmail vmail uarch 7B Oct 27 16:48 .dovecot.lda-dupes.locks drwx-- 2 vmail vmail uarch 2B Oct 26 16:48 cur -rw--- 1 vmail vmail uarch 8.3K Oct 26 16:49 dovecot-uidlist -rw--- 1 vmail vmail uarch 8B Oct 26 16:49 dovecot-uidvalidity -r--r--r-- 1 vmail vmail uarch 0B Oct 26 16:48 dovecot-uidvalidity.63599d11 -rw--- 1 vmail vmail uarch 3.7K Oct 27 16:48 dovecot.index -rw--- 1 vmail vmail uarch 34K Oct 27 16:48 dovecot.index.cache -rw--- 1 vmail vmail uarch 644B Oct 27 16:48 dovecot.index.log -rw--- 1 vmail vmail uarch 40K Oct 27 16:48 dovecot.index.log.2 -rw--- 1 vmail vmail uarch 968B Oct 27 16:48 dovecot.list.index -rw--- 1 vmail vmail uarch 1.7K Oct 27 16:48 dovecot.list.index.log -rw--- 1 vmail vmail uarch 8.2K Oct 27 16:48 dovecot.list.index.log.2 -rw--- 1 vmail vmail uarch 96B Oct 26 16:48 dovecot.mailbox.log drwx-- 2 vmail vmail uarch 9B Oct 26 16:49 lucene-indexes -rw--- 1 vmail vmail uarch 0B Oct 26 16:48 maildirfolder drwx-- 2 vmail vmail uarch 142B Oct 26 16:49 new drwx-- 3 vmail vmail uarch 6B Oct 26 16:49 sieve -rw--- 1 vmail vmail uarch 29B Oct 26 16:48 subscriptions drwx-- 2 vmail vmail uarch 2B Oct 26 16:49 tmp and then : [17:27:42] mail18.scom.ca [root:0] /data/dovecot/users/scom.ca/ab...@scom.ca/sieve # ll total 67 drwx-- 3 vmail vmail uarch 6B Oct 26 16:49 . drwx-- 11 vmail vmail uarch 25B Oct 27 16:48 .. lrwx-- 1 vmail vmail uarch 13B Oct 27 16:48 .dovecot.sieve -> forward.sieve -rw--- 1 vmail vmail uarch 239B Oct 26 16:49 .dovecot.svbin -rw--- 1 vmail vmail uarch 31B Oct 26 16:48 forward.sieve drwx-- 2 vmail vmail uarch 2B Oct 26 16:48 tmp [17:27:44] mail18.scom.ca [root:0] /data/dovecot/users/scom.ca/ab...@scom.ca/sieve for a user script to be active you need to set the script active (after uploading etc?) which creates a link from dovecot.sieve to the script and a .svbin file (i believe, this is an observation on my side) if all this is setup properly then the script should execute? please note my system is db driven and i am using virtual maildir's if you are doing this manually then make sure the dovecot's user right's are correct you are probably far enough along the set mail_debug = yes in dovecot.conf (
Re: how to configure imapsieve to be used per user
ok fair enuff are you using a db to set the dir's there is a master sieve (all) directory that handles the entire server (message duplicate supression etc) mine is in [17:26:12] mail18.scom.ca [root:0] /usr/local/etc/dovecot/sieve # ll total 38 drwxr-xr-x 2 vmail vmail uarch4B Apr 2 2022 . drwxr-xr-x 5 root wheel uarch 29B Oct 27 07:41 .. -rw-r--r-- 1 vmail vmail uarch 97B Apr 2 2022 duplicates.sieve -rw-r--r-- 1 vmail vmail uarch 227B Apr 2 2022 duplicates.svbin [17:26:17] mail18.scom.ca [root:0] /usr/local/etc/dovecot/sieve from there each user (assuming dovecot config is correct will have it's own sieve folder under the maildir example : # mbox ab...@scom.ca [17:27:24] mail18.scom.ca [root:0] /data/dovecot/users/scom.ca/ab...@scom.ca # ll total 293 drwx-- 11 vmail vmail uarch 25B Oct 27 16:48 . drwx-- 164 vmail vmail uarch 164B Oct 27 06:52 .. drwx--5 vmail vmail uarch8B Oct 27 16:48 .Drafts drwx--5 vmail vmail uarch8B Oct 27 16:48 .Sent drwx--5 vmail vmail uarch8B Oct 27 16:48 .Trash -rw---1 vmail vmail uarch 1.4K Oct 26 16:49 .dovecot.lda-dupes drwx--5 vmail vmail uarch7B Oct 27 16:48 .dovecot.lda-dupes.locks drwx--2 vmail vmail uarch2B Oct 26 16:48 cur -rw---1 vmail vmail uarch 8.3K Oct 26 16:49 dovecot-uidlist -rw---1 vmail vmail uarch8B Oct 26 16:49 dovecot-uidvalidity -r--r--r--1 vmail vmail uarch0B Oct 26 16:48 dovecot-uidvalidity.63599d11 -rw---1 vmail vmail uarch 3.7K Oct 27 16:48 dovecot.index -rw---1 vmail vmail uarch 34K Oct 27 16:48 dovecot.index.cache -rw---1 vmail vmail uarch 644B Oct 27 16:48 dovecot.index.log -rw---1 vmail vmail uarch 40K Oct 27 16:48 dovecot.index.log.2 -rw---1 vmail vmail uarch 968B Oct 27 16:48 dovecot.list.index -rw---1 vmail vmail uarch 1.7K Oct 27 16:48 dovecot.list.index.log -rw---1 vmail vmail uarch 8.2K Oct 27 16:48 dovecot.list.index.log.2 -rw---1 vmail vmail uarch 96B Oct 26 16:48 dovecot.mailbox.log drwx--2 vmail vmail uarch9B Oct 26 16:49 lucene-indexes -rw---1 vmail vmail uarch0B Oct 26 16:48 maildirfolder drwx--2 vmail vmail uarch 142B Oct 26 16:49 new drwx--3 vmail vmail uarch6B Oct 26 16:49 sieve -rw---1 vmail vmail uarch 29B Oct 26 16:48 subscriptions drwx--2 vmail vmail uarch2B Oct 26 16:49 tmp and then : [17:27:42] mail18.scom.ca [root:0] /data/dovecot/users/scom.ca/ab...@scom.ca/sieve # ll total 67 drwx-- 3 vmail vmail uarch6B Oct 26 16:49 . drwx-- 11 vmail vmail uarch 25B Oct 27 16:48 .. lrwx-- 1 vmail vmail uarch 13B Oct 27 16:48 .dovecot.sieve -> forward.sieve -rw--- 1 vmail vmail uarch 239B Oct 26 16:49 .dovecot.svbin -rw--- 1 vmail vmail uarch 31B Oct 26 16:48 forward.sieve drwx-- 2 vmail vmail uarch2B Oct 26 16:48 tmp [17:27:44] mail18.scom.ca [root:0] /data/dovecot/users/scom.ca/ab...@scom.ca/sieve for a user script to be active you need to set the script active (after uploading etc?) which creates a link from dovecot.sieve to the script and a .svbin file (i believe, this is an observation on my side) if all this is setup properly then the script should execute? please note my system is db driven and i am using virtual maildir's if you are doing this manually then make sure the dovecot's user right's are correct you are probably far enough along the set mail_debug = yes in dovecot.conf (remember to restart the server) this should dump a wack of logging somewhere (file or syslog) sieve or pigeonhole will be in there when you try to do something fyi fyi Happy Thursday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 10/27/2022 4:06 PM, Sebastian Bachmann wrote: On 27.10.2022 13:54, Paul Kudla wrote: again may (probably not) what you are looking for but it at least gives another example(s) No, actually I was looking for something different. The TO and me were looking for imapsieve examples and how they can be configured on a per user & per mailbox basis. I tried now some things, and I'm at least one step further. The important parts seemed to be: * Enable IMAP METADATA * Set `imapsieve_url = sieve://server:4190` (is that correct?) Now you can add the metadata, for example to the mailbox "test": a SETMETADATA test (/shared/imapsieve/script "sieve/imap.sieve") However, from this point on it does not work. I created a very simple example, which should simple copy any mail that is moved into the folder (right?): require ["copy"]; redirect :copy "some_other_email_
Re: how to configure imapsieve to be used per user
My apologies to the response eariler I was making the assumption that you were using pigeonholes it needs to be compiled seperately after making dovecot's server installs basically the pigeonholes has to be compiled against the dovecot version you are running after which my post info will be valid. fyi . Happy Thursday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 10/27/2022 9:48 AM, dovecot-boun...@dovecot.org wrote: On 2022-10-27 02:28, Stephan Bosch wrote: On 24-10-2022 12:00, Sebastian Bachmann wrote: according to the documentation, this has to be added to the IMAP METADATA dict per mailbox (https://doc.dovecot.org/configuration_manual/imap_metadata/): https://doc.dovecot.org/configuration_manual/sieve/plugins/imapsieve/ says: The basic IMAPSIEVE capability allows attaching a Sieve script to a mailbox for any mailbox by setting a special IMAP METADATA entry. This way, users can configure Sieve scripts that are run for IMAP events in their mailboxes. But I can not find any example how this should work, neither which client supports setting those things. My guess is that these keys are used: https://www.iana.org/assignments/imap-metadata/imap-metadata.xhtml#imap-metadata-2 I would also be interested to know if and how that works, especially if you can add a rule when moving mails (from anywhere) to a certain mailbox for a single user. The basic capability works according to the specification: https://www.rfc-editor.org/rfc/rfc6785 This allows the users to configure these scripts. If you want to arrange this solely at the administrator's discretion, you can use the _before/_after settings documented in https://doc.dovecot.org/configuration_manual/sieve/plugins/imapsieve Best, Sebastian On 17.10.2022 12:46, Marc wrote: I only see configurations that are active for all users, how to configure this in the user sieve rules. I only need this for specific users. Why dont you use pigeonholes? Also, I recommend to look for Symlink creation titled post here in the mailing list, there is few points in about setting up per user sieve scripts that will be helpful to you. Also, there is other posts on how to setup sieve for per user scripts. Zakaria.
Re: how to configure imapsieve to be used per user
'Setting Active' )
tn.write('SETACTIVE "forward"\r\n')
status = tn.expect(['OK','NO'],5)
log_debug (debug, 'Write Status : %s' %str(status) )
#logout
tn.write('LOGOUT')
status = tn.expect(['OK','NO'],5)
log_debug (debug, 'Logout Status : %s' %str(status) )
___
Happy Thursday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 10/26/2022 9:28 PM, Stephan Bosch wrote:
On 24-10-2022 12:00, Sebastian Bachmann wrote:
according to the documentation, this has to be added to the IMAP
METADATA dict per mailbox
(https://doc.dovecot.org/configuration_manual/imap_metadata/):
https://doc.dovecot.org/configuration_manual/sieve/plugins/imapsieve/
says:
The basic IMAPSIEVE capability allows attaching a Sieve script to a
mailbox for any mailbox by setting a special IMAP METADATA entry.
This way, users can configure Sieve scripts that are run for IMAP
events in their mailboxes.
But I can not find any example how this should work, neither which
client supports setting those things.
My guess is that these keys are used:
https://www.iana.org/assignments/imap-metadata/imap-metadata.xhtml#imap-metadata-2
I would also be interested to know if and how that works, especially
if you can add a rule when moving mails (from anywhere) to a certain
mailbox for a single user.
The basic capability works according to the specification:
https://www.rfc-editor.org/rfc/rfc6785
This allows the users to configure these scripts.
If you want to arrange this solely at the administrator's discretion,
you can use the _before/_after settings documented in
https://doc.dovecot.org/configuration_manual/sieve/plugins/imapsieve
Best,
Sebastian
On 17.10.2022 12:46, Marc wrote:
I only see configurations that are active for all users, how to
configure this in the user sieve rules. I only need this for specific
users.
Re: ot: how to t/s TBird problems ?
may or may not be off topic how many emails in the inbox or folder in question i generally find thunderbird for example can handle an inbox of up to 20,000 emails in it without much issue (assuming half decent network connection etc) after that it slows down outlook don't even use if more then 200 rarely see a 40 second delay unless actually doing a physical search on the server side then rebuilding would become an issue? it's usually best to divert large inbox's to other subfolders if possible. might help? Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 10/24/2022 7:54 PM, Joseph Tam wrote: Voytek Eymont) wrote: yesterday it was --- I'm still experiencing a 40 second delay to retrieve emails for xxx If *this* is the problem you saw (and not the 2 hour delay mentioned further in the thread), you can get a hint where the problem lies if you see a 40s gap in the session logs: it will tell you who was doing what when the pause happened (e.g. during authentication? During LIST fetch? During message fetch?) For example, if dovecot was busy mulching through a large INBOX rebuilding indices, I can see how it can chew up 40s under some circumstances. Joseph Tam
Re: how to clean virtual users correctly
ok in general
the mail client (thunderbird etc) will usually create these upon the
first login
i on the other hand (because this was always a crap shoot with cyrus)
take the time to make the special folders manually.
example (done in python):
command2 =
commands.getoutput('/usr/local/bin/doveadm mailbox create -s -u %s Sent'
%str(username))
print 'Command2 : %s' %command2
command3 =
commands.getoutput('/usr/local/bin/doveadm mailbox create -s -u %s
Trash' %str(username))
print 'Command3 : %s' %command3
command4 =
commands.getoutput('/usr/local/bin/doveadm mailbox create -s -u %s
Drafts' %str(username))
print 'Command4 : %s' %command4
also note dovecot config's
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
}
will / should do the same thing.
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 10/24/2022 11:07 PM, Henry R wrote:
Hello
I removed the user in static userdb file, and deleted
/var/mail/vhosts/$domain/$user dir.
when I recreated the user, the system dirs (sent, draft etc) disappeared.
Anything wrong here? Thank you.
Re: lmtp userdb can't resolve users
Question are you using a db like postgresql or mysql etc
when running virtual mailboxes it is just simply a better solution
my setup is as follows
i use a django project to drive it
here are the basic's
in the dovecot.conf :
passdb {
args = /usr/local/etc/dovecot/dovecot-pgsql.conf
driver = sql
}
mail_plugins = " virtual notify replication fts fts_lucene "
service lmtp {
process_limit=1000
vsz_limit = 512m
client_limit=1
unix_listener /usr/home/postfix.local/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
mail_location = maildir:~/
&
# cat dovecot-pgsql.conf
driver = pgsql
connect = host=localhost port=5433 dbname=scom_billing user=pgsql
password=xxx
default_pass_scheme = PLAIN
password_query = SELECT username as user, password FROM email_users
WHERE username = '%u' and password <> 'alias' and status = True and
destination = '%u'
user_query = SELECT home, uid, gid FROM email_users WHERE username =
'%u' and password <> 'alias' and status = True and destination = '%u'
#iterate_query = SELECT user, password FROM email_users WHERE username =
'%u' and password <> 'alias' and status = True and destination = '%u'
iterate_query = SELECT "username" as user, domain FROM email_users WHERE
status = True and alias_flag = False
Happy Friday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 10/21/2022 3:18 AM, George Asenov wrote:
Hello,
I use postfix with dovecot as an lmtp LDA with unix users and multiple
domain names and mailboxes in Maildir format placed in domain directory
like:
/home/mainuser/homes/u...@domain.tld/Maildir
or
/home/mainuser/domains/domain2.tld/homes/u...@domain2.tld/Maildir
which have the main user as a group and u...@domain.tld/u...@domain2.tld
as owner
Postfix have
virtual_alias_maps = hash:/etc/postfix/virtual
there in virtual file there is map like:
u...@domain.tld user-dom...@domain.tld
and in /etc/passwd
there are actually 2 users with the same home dir and
same UID/GID (only the username is different)
and in postfix
mailbox_transport = lmtp:unix:private/dovecot-lmtp
The issue is that when postfix passes the email for local delivery to
dovecot lmtp it sends the username as user-domain@serverhostname.tld
but dovecot is configured with
!include auth-system.conf.ext
can't resolve this username thus fails to deliver.
I've found a workaround in the net to use custom userdb just for lmtp
like this:
protocol lmtp {
mail_plugins = $mail_plugins sieve
userdb {
driver = passwd-file
args = username_format=%n /etc/passwd
}
}
which works but produce some warnings because there is the root user (ID
0) and actually is a dirty workaround
Is there more elegant solution??
Re: SNI Config
much appreciated for the response
maybe a feature down the road??
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 10/12/2022 8:12 AM, Aki Tuomi wrote:
Hi!
The pipe syntax has never worked, no idea why you think it would have.
Unfortunately at the moment, files are your best option. I do understand the
annoyance.
Aki
On 12/10/2022 13:54 EEST Paul Kudla (SCOM.CA Internet Services Inc.)
wrote:
ok thanks for your input
I finally tracked down the issue
It was how i was loading the certificates in the first place
that being said (and i must have missed this) 2.3.18 seems to allow
importing a cert from a program
thus sni config
local_name mail.paulkudla.net {
ssl_key =/programs/common/getssl.cert -k mail.paulkudla.net -q yes
ssl_cert =/programs/common/getssl.cert -r mail.paulkudla.net -q yes
ssl_ca =/programs/common/getssl.cert -i mail.paulkudla.net -q yes
}
would work instead of file pipes from individual text files.
#local_name mail.paulkudla.net {
# ssl_key =http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 10/11/2022 12:46 PM, Jochen Bern wrote:
On 11.10.22 17:46, Paul Kudla (SCOM.CA Internet Services Inc.) wrote:
ok according to
https://www.openssl.org/docs/man1.0.2/man5/x509v3_config.html
SAN is not a valid option along with CN
... I don't see that being said in the page you refer to?
Anyhow, "stop giving a CN, use SANs instead" is a rather recent
development coming from the CA/Browser Forum - and IIUC still not a
*requirement*, not even for web browsers/servers. I would be surprised
if OpenSSL (already) were trying to enforce that policy.
Hmmm, what's our company's "IMAPS server" throwing at my TB again ... ?
$ openssl s_client -connect outlook.office365.com:993 -showcerts |
openssl x509 -noout -text
[...]
Subject: C = US, ST = Washington, L = Redmond, O = Microsoft
Corporation, CN = outlook.com
[...]
X509v3 Subject Alternative Name:
DNS:*.clo.footprintdns.com, DNS:*.hotmail.com,
DNS:*.internal.outlook.com, [...]
... yeah, no, nothing that Thunderbird (from 69-ish to 102) should get
indigestion over.
Upoin further testing thunderbird seems to be locking onto the primary
domain (*.scom.ca) of the server skipp any sni setup ??
You might want to get a network trace of your Thunderbird talking to the
server to see what cert actually is presented by the server, and
ideally, what domain is requested by SNI (if at all). That all happens
before the connection starts to be encrypted, so you should be able to
read it (say, with Wireshark) without having to crack any crypto ...
Kind regards,
Re: SNI Config
ok thanks for your input
I finally tracked down the issue
It was how i was loading the certificates in the first place
that being said (and i must have missed this) 2.3.18 seems to allow
importing a cert from a program
thus sni config
local_name mail.paulkudla.net {
ssl_key =/programs/common/getssl.cert -k mail.paulkudla.net -q yes
ssl_cert =/programs/common/getssl.cert -r mail.paulkudla.net -q yes
ssl_ca =/programs/common/getssl.cert -i mail.paulkudla.net -q yes
}
would work instead of file pipes from individual text files.
#local_name mail.paulkudla.net {
# ssl_key =I am sure you can appreciate generating files for 1000+ ssl certs can
become a nightmare management wise
either that or a pgsql select ?
I have gone back to text files in the mean time ?
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 10/11/2022 12:46 PM, Jochen Bern wrote:
On 11.10.22 17:46, Paul Kudla (SCOM.CA Internet Services Inc.) wrote:
ok according to
https://www.openssl.org/docs/man1.0.2/man5/x509v3_config.html
SAN is not a valid option along with CN
... I don't see that being said in the page you refer to?
Anyhow, "stop giving a CN, use SANs instead" is a rather recent
development coming from the CA/Browser Forum - and IIUC still not a
*requirement*, not even for web browsers/servers. I would be surprised
if OpenSSL (already) were trying to enforce that policy.
Hmmm, what's our company's "IMAPS server" throwing at my TB again ... ?
$ openssl s_client -connect outlook.office365.com:993 -showcerts |
openssl x509 -noout -text
[...]
Subject: C = US, ST = Washington, L = Redmond, O = Microsoft
Corporation, CN = outlook.com
[...]
X509v3 Subject Alternative Name:
DNS:*.clo.footprintdns.com, DNS:*.hotmail.com,
DNS:*.internal.outlook.com, [...]
... yeah, no, nothing that Thunderbird (from 69-ish to 102) should get
indigestion over.
Upoin further testing thunderbird seems to be locking onto the primary
domain (*.scom.ca) of the server skipp any sni setup ??
You might want to get a network trace of your Thunderbird talking to the
server to see what cert actually is presented by the server, and
ideally, what domain is requested by SNI (if at all). That all happens
before the connection starts to be encrypted, so you should be able to
read it (say, with Wireshark) without having to crack any crypto ...
Kind regards,
Re: Thunderbird can't connect to Dovecot (bad certificate: SSL alert number 42) - sni
ok according to https://www.openssl.org/docs/man1.0.2/man5/x509v3_config.html SAN is not a valid option along with CN CN is part of the subject ?? Upoin further testing thunderbird seems to be locking onto the primary domain (*.scom.ca) of the server skipp any sni setup ?? again thoughts Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 10/11/2022 9:17 AM, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: ok it appears that all this revolves around openssl does anyone have explicit instructions on how to generate a proper ssl key, csr etc file with the proper SAN & CN etc i tried # openssl req -new -nodes -newkey rsa:2048 -config ./openssl.cnf -reqexts req_ext -keyout mail.paulkudla.net.key -out mail.paulkudla.net.csr Error Loading request extension section req_ext 34371092480:error:22075075:X509 V3 routines:v2i_GENERAL_NAME_ex:unsupported option:/usr/src/crypto/openssl/crypto/x509v3/v3_alt.c:534:name=SAN.1 34371092480:error:22098080:X509 V3 routines:X509V3_EXT_nconf:error in extension:/usr/src/crypto/openssl/crypto/x509v3/v3_conf.c:47:name=subjectAltName, value=@alt_names and got the errors above there not seem to be much on the web about how to generate these certs?? Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 10/11/2022 7:47 AM, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: Good morning to all i guess things have changed yet again to keep this simple : i buy a certificate (example) : mail.paulkudla.net i generated the key / csr as per normal using data = '/usr/local/bin/openssl req -new -key /tmp/temp.key -out /tmp/temp.csr -subj "/C=%s/ST=%s/L=%s/O=%s/CN=%s"' %(country,state,location,organization,self.domain) please note the above is done in django (yes i am running thunderbird v102) i go buy the certificate i database the CRT & CA CSR is : -BEGIN CERTIFICATE REQUEST- MIICpzCCAY8CAQAwYjELMAkGA1UEBhMCQ0ExEDAOBgNVBAgMB09udGFyaW8xDzAN BgNVBAcMBldoaXRieTETMBEGA1UECgwKUGF1bCBLdWRsYTEbMBkGA1UEAwwSbWFp bC5wYXVsa3VkbGEubmV0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA mSWAdwbxwjkjALQa4UdgOBHcFJDA5XkGI/8SswotYMnzjRAAE4S88vUTO3ltMasY rprEvWEiEzUrRon3hh1ZZguV775fNCbyKUGKwGLKPDpmKxYCsE4gi2z7LKY13wSv lLE8++Hqvt3cmZZ+wxWP/hy6LcS/6PvUPgN7S+cEC5TNLQ6VRZdpSGolRCrN9hsN 15GWYEQ/zcLW2PeCWav9DOr6NHBRE+fruDy3jFT0TkHWf3H+GKB0/RZ0agMJcEGc ZLdJ1LkvNAn6gslppm3otZyu7XTvY9qZXcYOlMN0KL3a3488OwXTwWJHEN58eCMc juax1f7ad8Z/+Pi+OFwfWQIDAQABoAAwDQYJKoZIhvcNAQELBQADggEBAFgL24yi WPat73tg1fANvutWXa2WEXeegqOawqvsV74lcyqMes8yhxiz/niOAt3oOLmViRF4 VlorgUwL0eAxtNeY4lgURW6XM5oz8TBINnPPohSAuDL9azLV1U1+M/vAvLs+LRd9 7wfVCN5bov7y735u2w38GAjmXJCBdoc+glUa+eGd5WH2+r/QQW/lRqVTDq+arqNk 9DTZc73gDCDmV45vTtbrlLnOxtmpqaQKsoFCCJW8OWaaDXfc8I+TdClVsThsbrWu iz1/QClBPbKvfufNb+asTQSCDeJFc2EynDSE1yeYzliMLo+77ZoMqJPvI9IJCuj5 yq88NESoIYaO6Do= -END CERTIFICATE REQUEST- CRT is : -BEGIN CERTIFICATE- MIIGRTCCBS2gAwIBAgIRAKTmHoDG9LF3heBvAT8gZkYwDQYJKoZIhvcNAQELBQAw gY8xCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAO BgNVBAcTB1NhbGZvcmQxGDAWBgNVBAoTD1NlY3RpZ28gTGltaXRlZDE3MDUGA1UE AxMuU2VjdGlnbyBSU0EgRG9tYWluIFZhbGlkYXRpb24gU2VjdXJlIFNlcnZlciBD QTAeFw0yMjA2MTYwMDAwMDBaFw0yMzA2MTYyMzU5NTlaMB0xGzAZBgNVBAMTEm1h aWwucGF1bGt1ZGxhLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB AJklgHcG8cI5IwC0GuFHYDgR3BSQwOV5BiP/ErMKLWDJ840QABOEvPL1Ezt5bTGr GK6axL1hIhM1K0aJ94YdWWYLle++XzQm8ilBisBiyjw6ZisWArBOIIts+yymNd8E r5SxPPvh6r7d3JmWfsMVj/4cui3Ev+j71D4De0vnBAuUzS0OlUWXaUhqJUQqzfYb DdeRlmBEP83C1tj3glmr/Qzq+jRwURPn67g8t4xU9E5B1n9x/higdP0WdGoDCXBB nGS3SdS5LzQJ+oLJaaZt6LWcru1072PamV3GDpTDdCi92t+PPDsF08FiRxDefHgj HI7msdX+2nfGf/j4vjhcH1kCAwEAAaOCAwswggMHMB8GA1UdIwQYMBaAFI2MXsRU rYrhd+mb+ZsF4bgBjWHhMB0GA1UdDgQWBBROA5NFqfrlHGbkp9v1JBxZe0fZsDAO BgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggrBgEFBQcD AQYIKwYBBQUHAwIwSQYDVR0gBEIwQDA0BgsrBgEEAbIxAQICBzAlMCMGCCsGAQUF BwIBFhdodHRwczovL3NlY3RpZ28uY29tL0NQUzAIBgZngQwBAgEwgYQGCCsGAQUF BwEBBHgwdjBPBggrBgEFBQcwAoZDaHR0cDovL2NydC5zZWN0aWdvLmNvbS9TZWN0 aWdvUlNBRG9tYWluVmFsaWRhdGlvblNlY3VyZVNlcnZlckNBLmNydDAjBggrBgEF BQcwAYYXaHR0cDovL29jc3Auc2VjdGlnby5jb20wNQYDVR0RBC4wLIISbWFpbC5w YXVsa3VkbGEubmV0ghZ3d3cubWFpbC5wYXVsa3VkbGEubmV0MIIBfQYKKwYBBAHW eQIEAgSCAW0EggFpAWcAdgCt9776fP8QyIudPZwePhhqtGcpXc+xDCTKhYY069yC igAAAYFsxJHxAAAEAwBHMEUCIQDxa9L+JaMJJImKuYPmfCAwJOiGXwECgtruOegv vPqGpwIgWW8B0SWqVNPEFBveoBlIZF3jjj4nQIzYi2LnLizoVDMAdQB6MoxU2Lct tiDqOOBSHumEFnAyE4VNO9IrwTpXo1LrUgAAAYFsxJHJAAAEAwBGMEQCIDIgNptW Qum0KFyemHNTTfonlq4FvWTgzR1AGUnOgotPAiAAiwyN9MjZNiP76P3fel6BqEqj jwnSVleJR1DgLIoyPQB2AOg+0No+9QY1MudXKLyJa8kD08vREWvs62nhd31tBr1u AAAB
Re: Thunderbird can't connect to Dovecot (bad certificate: SSL alert number 42) - sni
ok it appears that all this revolves around openssl does anyone have explicit instructions on how to generate a proper ssl key, csr etc file with the proper SAN & CN etc i tried # openssl req -new -nodes -newkey rsa:2048 -config ./openssl.cnf -reqexts req_ext -keyout mail.paulkudla.net.key -out mail.paulkudla.net.csr Error Loading request extension section req_ext 34371092480:error:22075075:X509 V3 routines:v2i_GENERAL_NAME_ex:unsupported option:/usr/src/crypto/openssl/crypto/x509v3/v3_alt.c:534:name=SAN.1 34371092480:error:22098080:X509 V3 routines:X509V3_EXT_nconf:error in extension:/usr/src/crypto/openssl/crypto/x509v3/v3_conf.c:47:name=subjectAltName, value=@alt_names and got the errors above there not seem to be much on the web about how to generate these certs?? Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 10/11/2022 7:47 AM, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: Good morning to all i guess things have changed yet again to keep this simple : i buy a certificate (example) : mail.paulkudla.net i generated the key / csr as per normal using data = '/usr/local/bin/openssl req -new -key /tmp/temp.key -out /tmp/temp.csr -subj "/C=%s/ST=%s/L=%s/O=%s/CN=%s"' %(country,state,location,organization,self.domain) please note the above is done in django (yes i am running thunderbird v102) i go buy the certificate i database the CRT & CA CSR is : -BEGIN CERTIFICATE REQUEST- MIICpzCCAY8CAQAwYjELMAkGA1UEBhMCQ0ExEDAOBgNVBAgMB09udGFyaW8xDzAN BgNVBAcMBldoaXRieTETMBEGA1UECgwKUGF1bCBLdWRsYTEbMBkGA1UEAwwSbWFp bC5wYXVsa3VkbGEubmV0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA mSWAdwbxwjkjALQa4UdgOBHcFJDA5XkGI/8SswotYMnzjRAAE4S88vUTO3ltMasY rprEvWEiEzUrRon3hh1ZZguV775fNCbyKUGKwGLKPDpmKxYCsE4gi2z7LKY13wSv lLE8++Hqvt3cmZZ+wxWP/hy6LcS/6PvUPgN7S+cEC5TNLQ6VRZdpSGolRCrN9hsN 15GWYEQ/zcLW2PeCWav9DOr6NHBRE+fruDy3jFT0TkHWf3H+GKB0/RZ0agMJcEGc ZLdJ1LkvNAn6gslppm3otZyu7XTvY9qZXcYOlMN0KL3a3488OwXTwWJHEN58eCMc juax1f7ad8Z/+Pi+OFwfWQIDAQABoAAwDQYJKoZIhvcNAQELBQADggEBAFgL24yi WPat73tg1fANvutWXa2WEXeegqOawqvsV74lcyqMes8yhxiz/niOAt3oOLmViRF4 VlorgUwL0eAxtNeY4lgURW6XM5oz8TBINnPPohSAuDL9azLV1U1+M/vAvLs+LRd9 7wfVCN5bov7y735u2w38GAjmXJCBdoc+glUa+eGd5WH2+r/QQW/lRqVTDq+arqNk 9DTZc73gDCDmV45vTtbrlLnOxtmpqaQKsoFCCJW8OWaaDXfc8I+TdClVsThsbrWu iz1/QClBPbKvfufNb+asTQSCDeJFc2EynDSE1yeYzliMLo+77ZoMqJPvI9IJCuj5 yq88NESoIYaO6Do= -END CERTIFICATE REQUEST- CRT is : -BEGIN CERTIFICATE- MIIGRTCCBS2gAwIBAgIRAKTmHoDG9LF3heBvAT8gZkYwDQYJKoZIhvcNAQELBQAw gY8xCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAO BgNVBAcTB1NhbGZvcmQxGDAWBgNVBAoTD1NlY3RpZ28gTGltaXRlZDE3MDUGA1UE AxMuU2VjdGlnbyBSU0EgRG9tYWluIFZhbGlkYXRpb24gU2VjdXJlIFNlcnZlciBD QTAeFw0yMjA2MTYwMDAwMDBaFw0yMzA2MTYyMzU5NTlaMB0xGzAZBgNVBAMTEm1h aWwucGF1bGt1ZGxhLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB AJklgHcG8cI5IwC0GuFHYDgR3BSQwOV5BiP/ErMKLWDJ840QABOEvPL1Ezt5bTGr GK6axL1hIhM1K0aJ94YdWWYLle++XzQm8ilBisBiyjw6ZisWArBOIIts+yymNd8E r5SxPPvh6r7d3JmWfsMVj/4cui3Ev+j71D4De0vnBAuUzS0OlUWXaUhqJUQqzfYb DdeRlmBEP83C1tj3glmr/Qzq+jRwURPn67g8t4xU9E5B1n9x/higdP0WdGoDCXBB nGS3SdS5LzQJ+oLJaaZt6LWcru1072PamV3GDpTDdCi92t+PPDsF08FiRxDefHgj HI7msdX+2nfGf/j4vjhcH1kCAwEAAaOCAwswggMHMB8GA1UdIwQYMBaAFI2MXsRU rYrhd+mb+ZsF4bgBjWHhMB0GA1UdDgQWBBROA5NFqfrlHGbkp9v1JBxZe0fZsDAO BgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggrBgEFBQcD AQYIKwYBBQUHAwIwSQYDVR0gBEIwQDA0BgsrBgEEAbIxAQICBzAlMCMGCCsGAQUF BwIBFhdodHRwczovL3NlY3RpZ28uY29tL0NQUzAIBgZngQwBAgEwgYQGCCsGAQUF BwEBBHgwdjBPBggrBgEFBQcwAoZDaHR0cDovL2NydC5zZWN0aWdvLmNvbS9TZWN0 aWdvUlNBRG9tYWluVmFsaWRhdGlvblNlY3VyZVNlcnZlckNBLmNydDAjBggrBgEF BQcwAYYXaHR0cDovL29jc3Auc2VjdGlnby5jb20wNQYDVR0RBC4wLIISbWFpbC5w YXVsa3VkbGEubmV0ghZ3d3cubWFpbC5wYXVsa3VkbGEubmV0MIIBfQYKKwYBBAHW eQIEAgSCAW0EggFpAWcAdgCt9776fP8QyIudPZwePhhqtGcpXc+xDCTKhYY069yC igAAAYFsxJHxAAAEAwBHMEUCIQDxa9L+JaMJJImKuYPmfCAwJOiGXwECgtruOegv vPqGpwIgWW8B0SWqVNPEFBveoBlIZF3jjj4nQIzYi2LnLizoVDMAdQB6MoxU2Lct tiDqOOBSHumEFnAyE4VNO9IrwTpXo1LrUgAAAYFsxJHJAAAEAwBGMEQCIDIgNptW Qum0KFyemHNTTfonlq4FvWTgzR1AGUnOgotPAiAAiwyN9MjZNiP76P3fel6BqEqj jwnSVleJR1DgLIoyPQB2AOg+0No+9QY1MudXKLyJa8kD08vREWvs62nhd31tBr1u AAABgWzEkYoAAAQDAEcwRQIgOYjevKp5RI+c0JhIi6JflaxiNokRTSeXN6LrdIVt Cf8CIQCG+aLreYVV8xCPV0skr0ats5zMf5PLPN2y8EIxGPPNVTANBgkqhkiG9w0B AQsFAAOCAQEAJX544qDTgkGGLUOher7tH7yUgEhQFYkBDAirO37MXrhtuzH6pGSp XfYVNB9e2ydprfmLDh8O8oTaXpaQfp/jwK3U0GfvG57MfdQTLOunpWnCjaMUPUcv jPU90/mXc5oWlO5iJ6jPDkS/x47K03P6vftSr7AMwnLq4kYwuG9fHLslMHhoojen 9S2G1QjKVp5jkFecmQib+JOZV9Ub9r6iumHICfdcSO+tyBL2IDqWDQhuAVUXgyOV 11O9ZgikoeRhgsMhwiQA1z/Fs6Xqx/XCs6nUciebRiQuuHYm/PUG2H+tg0sLhJ6L ntIEhjjkumL0oJEfDidP/8wmrsPuwfSDCQ== -END CERTIFICATE- CA (INTER) : -BEGIN CERTIFICATE- MIIGEzCCA/ugAwIBAgIQfVtRJrR2uhHbdBYLvFMN
Thunderbird can't connect to Dovecot (bad certificate: SSL alert number 42) - sni
Good morning to all i guess things have changed yet again to keep this simple : i buy a certificate (example) : mail.paulkudla.net i generated the key / csr as per normal using data = '/usr/local/bin/openssl req -new -key /tmp/temp.key -out /tmp/temp.csr -subj "/C=%s/ST=%s/L=%s/O=%s/CN=%s"' %(country,state,location,organization,self.domain) please note the above is done in django (yes i am running thunderbird v102) i go buy the certificate i database the CRT & CA CSR is : -BEGIN CERTIFICATE REQUEST- MIICpzCCAY8CAQAwYjELMAkGA1UEBhMCQ0ExEDAOBgNVBAgMB09udGFyaW8xDzAN BgNVBAcMBldoaXRieTETMBEGA1UECgwKUGF1bCBLdWRsYTEbMBkGA1UEAwwSbWFp bC5wYXVsa3VkbGEubmV0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA mSWAdwbxwjkjALQa4UdgOBHcFJDA5XkGI/8SswotYMnzjRAAE4S88vUTO3ltMasY rprEvWEiEzUrRon3hh1ZZguV775fNCbyKUGKwGLKPDpmKxYCsE4gi2z7LKY13wSv lLE8++Hqvt3cmZZ+wxWP/hy6LcS/6PvUPgN7S+cEC5TNLQ6VRZdpSGolRCrN9hsN 15GWYEQ/zcLW2PeCWav9DOr6NHBRE+fruDy3jFT0TkHWf3H+GKB0/RZ0agMJcEGc ZLdJ1LkvNAn6gslppm3otZyu7XTvY9qZXcYOlMN0KL3a3488OwXTwWJHEN58eCMc juax1f7ad8Z/+Pi+OFwfWQIDAQABoAAwDQYJKoZIhvcNAQELBQADggEBAFgL24yi WPat73tg1fANvutWXa2WEXeegqOawqvsV74lcyqMes8yhxiz/niOAt3oOLmViRF4 VlorgUwL0eAxtNeY4lgURW6XM5oz8TBINnPPohSAuDL9azLV1U1+M/vAvLs+LRd9 7wfVCN5bov7y735u2w38GAjmXJCBdoc+glUa+eGd5WH2+r/QQW/lRqVTDq+arqNk 9DTZc73gDCDmV45vTtbrlLnOxtmpqaQKsoFCCJW8OWaaDXfc8I+TdClVsThsbrWu iz1/QClBPbKvfufNb+asTQSCDeJFc2EynDSE1yeYzliMLo+77ZoMqJPvI9IJCuj5 yq88NESoIYaO6Do= -END CERTIFICATE REQUEST- CRT is : -BEGIN CERTIFICATE- MIIGRTCCBS2gAwIBAgIRAKTmHoDG9LF3heBvAT8gZkYwDQYJKoZIhvcNAQELBQAw gY8xCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAO BgNVBAcTB1NhbGZvcmQxGDAWBgNVBAoTD1NlY3RpZ28gTGltaXRlZDE3MDUGA1UE AxMuU2VjdGlnbyBSU0EgRG9tYWluIFZhbGlkYXRpb24gU2VjdXJlIFNlcnZlciBD QTAeFw0yMjA2MTYwMDAwMDBaFw0yMzA2MTYyMzU5NTlaMB0xGzAZBgNVBAMTEm1h aWwucGF1bGt1ZGxhLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB AJklgHcG8cI5IwC0GuFHYDgR3BSQwOV5BiP/ErMKLWDJ840QABOEvPL1Ezt5bTGr GK6axL1hIhM1K0aJ94YdWWYLle++XzQm8ilBisBiyjw6ZisWArBOIIts+yymNd8E r5SxPPvh6r7d3JmWfsMVj/4cui3Ev+j71D4De0vnBAuUzS0OlUWXaUhqJUQqzfYb DdeRlmBEP83C1tj3glmr/Qzq+jRwURPn67g8t4xU9E5B1n9x/higdP0WdGoDCXBB nGS3SdS5LzQJ+oLJaaZt6LWcru1072PamV3GDpTDdCi92t+PPDsF08FiRxDefHgj HI7msdX+2nfGf/j4vjhcH1kCAwEAAaOCAwswggMHMB8GA1UdIwQYMBaAFI2MXsRU rYrhd+mb+ZsF4bgBjWHhMB0GA1UdDgQWBBROA5NFqfrlHGbkp9v1JBxZe0fZsDAO BgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggrBgEFBQcD AQYIKwYBBQUHAwIwSQYDVR0gBEIwQDA0BgsrBgEEAbIxAQICBzAlMCMGCCsGAQUF BwIBFhdodHRwczovL3NlY3RpZ28uY29tL0NQUzAIBgZngQwBAgEwgYQGCCsGAQUF BwEBBHgwdjBPBggrBgEFBQcwAoZDaHR0cDovL2NydC5zZWN0aWdvLmNvbS9TZWN0 aWdvUlNBRG9tYWluVmFsaWRhdGlvblNlY3VyZVNlcnZlckNBLmNydDAjBggrBgEF BQcwAYYXaHR0cDovL29jc3Auc2VjdGlnby5jb20wNQYDVR0RBC4wLIISbWFpbC5w YXVsa3VkbGEubmV0ghZ3d3cubWFpbC5wYXVsa3VkbGEubmV0MIIBfQYKKwYBBAHW eQIEAgSCAW0EggFpAWcAdgCt9776fP8QyIudPZwePhhqtGcpXc+xDCTKhYY069yC igAAAYFsxJHxAAAEAwBHMEUCIQDxa9L+JaMJJImKuYPmfCAwJOiGXwECgtruOegv vPqGpwIgWW8B0SWqVNPEFBveoBlIZF3jjj4nQIzYi2LnLizoVDMAdQB6MoxU2Lct tiDqOOBSHumEFnAyE4VNO9IrwTpXo1LrUgAAAYFsxJHJAAAEAwBGMEQCIDIgNptW Qum0KFyemHNTTfonlq4FvWTgzR1AGUnOgotPAiAAiwyN9MjZNiP76P3fel6BqEqj jwnSVleJR1DgLIoyPQB2AOg+0No+9QY1MudXKLyJa8kD08vREWvs62nhd31tBr1u AAABgWzEkYoAAAQDAEcwRQIgOYjevKp5RI+c0JhIi6JflaxiNokRTSeXN6LrdIVt Cf8CIQCG+aLreYVV8xCPV0skr0ats5zMf5PLPN2y8EIxGPPNVTANBgkqhkiG9w0B AQsFAAOCAQEAJX544qDTgkGGLUOher7tH7yUgEhQFYkBDAirO37MXrhtuzH6pGSp XfYVNB9e2ydprfmLDh8O8oTaXpaQfp/jwK3U0GfvG57MfdQTLOunpWnCjaMUPUcv jPU90/mXc5oWlO5iJ6jPDkS/x47K03P6vftSr7AMwnLq4kYwuG9fHLslMHhoojen 9S2G1QjKVp5jkFecmQib+JOZV9Ub9r6iumHICfdcSO+tyBL2IDqWDQhuAVUXgyOV 11O9ZgikoeRhgsMhwiQA1z/Fs6Xqx/XCs6nUciebRiQuuHYm/PUG2H+tg0sLhJ6L ntIEhjjkumL0oJEfDidP/8wmrsPuwfSDCQ== -END CERTIFICATE- CA (INTER) : -BEGIN CERTIFICATE- MIIGEzCCA/ugAwIBAgIQfVtRJrR2uhHbdBYLvFMNpzANBgkqhkiG9w0BAQwFADCB iDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcTC0pl cnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAsBgNV BAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTgx MTAyMDAwMDAwWhcNMzAxMjMxMjM1OTU5WjCBjzELMAkGA1UEBhMCR0IxGzAZBgNV BAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEYMBYGA1UE ChMPU2VjdGlnbyBMaW1pdGVkMTcwNQYDVQQDEy5TZWN0aWdvIFJTQSBEb21haW4g VmFsaWRhdGlvbiBTZWN1cmUgU2VydmVyIENBMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEA1nMz1tc8INAA0hdFuNY+B6I/x0HuMjDJsGz99J/LEpgPLT+N TQEMgg8Xf2Iu6bhIefsWg06t1zIlk7cHv7lQP6lMw0Aq6Tn/2YHKHxYyQdqAJrkj eocgHuP/IJo8lURvh3UGkEC0MpMWCRAIIz7S3YcPb11RFGoKacVPAXJpz9OTTG0E oKMbgn6xmrntxZ7FN3ifmgg0+1YuWMQJDgZkW7w33PGfKGioVrCSo1yfu4iYCBsk Haswha6vsC6eep3BwEIc4gLw6uBK0u+QDrTBQBbwb4VCSmT3pDCg/r8uoydajotY uK3DGReEY+1vVv2Dy2A0xHS+5p3b4eTlygxfFQIDAQABo4IBbjCCAWowHwYDVR0j BBgwFoAUU3m/WqorSs9UgOHYm8Cd8rIDZsswHQYDVR0OBBYEFI2MXsRUrYrhd+mb +ZsF4bgBjWHhMA4GA1UdDwEB/wQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMB0G A1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAbBgNVHSAEFDASMAYGBFUdIAAw CAYGZ4EMAQIBMFAGA1UdHwRJMEcwRaBDoEGGP2h0dHA6Ly9jcmwudXNlcnRydXN0
Re: new feature: sieve forward plugin
2a01:111:f400::/48 ip6:2a01:111:f403::/49 ip6:2a01:111:f403:8000::/50 ip6:2a01:111:f403:c000::/51 ip6:2a01:111:f403:f000::/52 include:spfd.protection.outlook.com -all" all microsoft had to do was change ip4:40.92.0.0/15 to ip4:40.92.0.0/14 and the offending microsoft servers would have passed I was forced to whitelist this in my spf hoping this would not let anything bad through. I now have to track spf bounces daily from microsoft in case they change something else upstream that messes up my customer. Whats the point of spf if it does not get used correctly in the config record? Typical microsoft, however google & bell.ca (canadian isp) are having similiar issues when a customer of mine sends an email to 6 different people at the same time (just a normal email list nothing fancy), bell.ca (for example) seems to have that destination forwarded to google and is bouncing back as an spf error anyways (and this is on a normal send) So in the case of bell.ca's customer forwarding his/her account to google somewhere along the line the origional spf record (from my server) is being passed along upstream and when bell.ca rewrites it with their own something is getting scrambled and google thinks it is coming from my origional server but because it is going through bell.ca and forwarding they are obviously having the same problem as described here? Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 9/26/2022 6:22 AM, Marc wrote: As this page[1] describes a more often occurring problem of forwarding messages from servers that are not included in the spf records. Maybe there should be a plugin that offers this forward functionality. Something like get the spf records of the sender check if there is a -all then apply the sender substitution. https://doc.dovecot.org/configuration_manual/sieve/configuring_auto_forward_sender_address/
Re: Get a list of currently active IMAP connections?
NO it is showing active open imap connections fyi Happy Friday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 8/19/2022 3:40 AM, Narcis Garcia wrote: Do you mean ps is reading dovecot.conf ?! Narcis Garcia __ I'm using this dedicated address because personal addresses aren't masked enough at this mail public archive. Public archive administrator should fix this against automated addresses collectors. El 19/8/22 a les 9:40, Aki Tuomi ha escrit: dovecot.conf, not ps config. Aki On 19/08/2022 10:38 EEST Narcis Garcia wrote: What config? I see no configuration file documented on ps manpage. Narcis Garcia __ I'm using this dedicated address because personal addresses aren't masked enough at this mail public archive. Public archive administrator should fix this against automated addresses collectors. El 19/8/22 a les 9:33, 202107-dove...@planhack.com ha escrit: Add `verbose_proctitle = yes` to your config to get usernames and IPs in the ps listing.
Re: Get a list of currently active IMAP connections?
I use ps : (greping by imap & idle) # ps -axww | grep imap | grep IDLE thant and split() in python 8606 - S 0:08.78 imap: [ke...@elirpa.com 54.242.98.60 IDLE] (imap) 12234 - I 0:01.00 imap: [recept...@clancyca.com 72.143.119.178 IDLE] (imap) 20668 - S 0:02.01 imap: [p...@scom.ca 216.58.25.131 IDLE] (imap) 23219 - I 0:00.33 imap: [cla...@clancyca.com 72.143.119.178 IDLE] (imap) 26761 - S 0:00.52 imap: [ed.ha...@ekst.ca 204.237.91.165 IDLE] (imap) 26785 - I 0:00.87 imap: [e...@scom.ca 204.237.91.165 IDLE] (imap) 26787 - I 0:00.80 imap: [ed.ha...@dssmgmt.com 204.237.91.165 IDLE] (imap) 27378 - S 0:00.42 imap: [e...@scom.ca 204.237.91.165 IDLE] (imap) 31404 - S 0:03.90 imap: [p...@scom.ca 216.58.25.131 IDLE] (imap) 32494 - S 0:00.13 imap: [install...@tomkudla.ca 167.94.196.10 IDLE] (imap) 32497 - S 0:00.13 imap: [install...@tomkudla.ca 167.94.196.10 IDLE] (imap) 33809 - I 0:00.28 imap: [cla...@clancyca.com 72.143.119.178 IDLE] (imap) 36321 - I 0:00.21 imap: [cla...@clancyca.com 72.143.119.178 IDLE] (imap) 39188 - I 0:00.39 imap: [cla...@clancyca.com 72.143.119.178 IDLE] (imap) 42706 - S 0:00.45 imap: [e...@scom.ca 204.237.91.165 IDLE] (imap) 46356 - S 0:02.98 imap: [rco...@tnky.ca 198.91.141.141 IDLE] (imap) 46422 - S 0:01.32 imap: [rco...@tnky.ca 198.91.141.141 IDLE] (imap) 46424 - S 0:01.27 imap: [rco...@tnky.ca 198.91.141.141 IDLE] (imap) 50756 - S 0:01.36 imap: [rco...@tnky.ca 198.91.141.141 IDLE] (imap) 58656 - I 0:00.07 imap: [ditchb...@clancyca.com 216.58.50.30 IDLE] (imap) 63886 - S 0:00.70 imap: [rco...@tnky.ca 198.91.141.141 IDLE] (imap) 68246 - I 0:00.08 imap: [l...@clancyca.com 72.143.119.178 IDLE] (imap) 74719 - I 0:00.03 imap: [d...@elirpa.com 142.183.30.44 IDLE] (imap) 76580 - I 0:00.02 imap: [i...@willsagriquipandfencing.ca 173.32.244.194 IDLE] (imap) 76584 - I 0:00.02 imap: [how...@willsagriquipandfencing.ca 173.32.244.194 IDLE] (imap) 77567 - S 0:00.04 imap: [rco...@tnky.ca 198.91.141.141 IDLE] (imap) 77569 - I 0:00.03 imap: [rco...@tnky.ca 198.91.141.141 IDLE] (imap) Happy Friday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 8/18/2022 6:28 PM, J Doe wrote: On 2022-08-16 16:46, Antonio Leding wrote: At the risk of being pedestrian, I just use something like |sudo netstat -an | grep ‘:[ IMAP_PORT ]’| I’m pretty sure you thought of this but still, thought I would toss it out… Hi Antonio and Jaroslaw, I don't think the second solution is pedestrian; I think it's cool that people have come up with different solutions for the same problem! I am thinking that this may not be the solution that Jaroslaw is looking for, as this also requires spawning a process to run netstat and then capturing the results. The socket approach avoids an additional process. - J
Re: dovecot/config processes open, and consuming all memory
for what's it is worth
i am not running any vsize on the config
service config {
unix_listener config {
user = vmail
}
}
i'm just running defaults
i do use vsz_limit elsewhere main to curve the replication processes ??
never had an issue 2.3.17 / 18 / 19
never used 16 (fyi)
maybe 2048M is insufficent system wide, try increasing 10 1g ?
Happy Friday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 8/12/2022 6:06 PM, dovecot-boun...@dovecot.org wrote:
I'm having strange behavior in dovecot 2.3.16.
It's opening dozens of dovecot/config process and consuming all server
memory. Normally each process consumes between 700Mb and 1Gb of ram.
Would anyone have an idea about this?
service config {
vsz_limit = 2048M
idle_kill = 60s
service_count = 1024
}
pstree
systemd─┬─ModemManager───2*[{ModemManager}]
├─agetty
├─cron
├─dbus-daemon
├─dovecot─┬─anvil
│ ├─6*[auth]
│ ├─46*[config]
│ ├─1212*[imap]
│ ├─155*[imap-login]
│ ├─12*[lmtp]
│ ├─38*[log]
│ ├─10*[managesieve]
│ ├─19*[pop3]
│ ├─3*[pop3-login]
│ └─18*[stats]
root 45831 0.0 1.1 774688 752732 ? S 09:31 0:31
dovecot/config
root 388792 0.0 1.1 775060 753276 ? S 14:00 0:15
dovecot/config
root 510685 0.0 1.1 775384 753604 ? S 15:06 0:20
dovecot/config
root 675638 0.0 1.1 775348 753620 ? S 16:56 0:15
dovecot/config
root 795375 0.0 1.1 775460 753516 ? S 18:03 0:07
dovecot/config
root 798754 0.2 1.1 775592 753712 ? S 18:05 0:30
dovecot/config
root 1082696 0.2 1.1 774892 753216 ? S 21:10 0:07
dovecot/config
root 1098433 0.4 1.1 774924 753244 ? S 21:33 0:07
dovecot/config
root 1109255 0.9 1.1 774924 753344 ? S 21:50 0:07
dovecot/config
root 1112976 2.0 1.1 774956 753528 ? S 21:57 0:07
dovecot/config
root 1114137 3.0 1.1 775028 753308 ? S 21:59 0:07
dovecot/config
root 1115382 5.4 1.1 774924 753496 ? S 22:01 0:06
dovecot/config
root 1883627 0.0 1.1 759120 728832 ? S Aug11 0:07
dovecot/config
root 1889705 0.0 1.8 1251460 1221872 ? S Aug11 0:11
dovecot/config
root 1895022 0.0 1.8 1253280 1224284 ? S Aug11 0:11
dovecot/config
root 1900690 0.0 1.8 1255684 1227528 ? S Aug11 0:12
dovecot/config
root 1905648 0.0 1.8 1257880 1229912 ? S Aug11 0:12
dovecot/config
root 1910857 0.0 1.8 1259156 1231552 ? S Aug11 0:12
dovecot/config
root 1914332 0.0 1.1 764328 736552 ? S Aug11 0:20
dovecot/config
root 2343896 0.0 1.8 1259472 1231516 ? S Aug11 0:12
dovecot/config
root 2346351 0.0 1.8 1259472 1231836 ? S Aug11 0:13
dovecot/config
root 2348559 0.0 1.1 764704 736440 ? S Aug11 0:14
dovecot/config
root 2445701 0.0 1.1 764276 736540 ? S Aug11 0:19
dovecot/config
root 2572525 0.0 1.1 764640 736880 ? S Aug11 0:18
dovecot/config
root 2734251 0.0 1.1 764776 737696 ? S Aug11 0:08
dovecot/config
root 2740980 0.0 1.1 764768 737244 ? S Aug11 0:17
dovecot/config
root 2899925 0.0 1.1 764624 737760 ? S Aug11 0:43
dovecot/config
root 3517063 0.0 1.1 764984 738004 ? S Aug11 0:09
dovecot/config
root 3541465 0.0 1.1 765224 738756 ? S Aug11 0:07
dovecot/config
root 3545589 0.0 1.1 766452 740408 ? S Aug11 0:06
dovecot/config
root 3549259 0.0 1.1 766796 741048 ? S Aug11 0:07
dovecot/config
root 3553902 0.0 1.1 767812 742284 ? S Aug11 0:07
dovecot/config
root 3558080 0.0 1.1 768440 743524 ? S Aug11 0:08
dovecot/config
root 3562091 0.0 1.1 769224 744424 ? S Aug11 0:07
dovecot/config
root 3568721 0.0 1.1 769388 744888 ? S Aug11 0:07
dovecot/config
root 3573024 0.0 1.1 770048 745904 ? S Aug11 0:08
dovecot/config
root 3578416 0.0 1.1 770836 746736 ? S Aug11 0:06
dovecot/config
root 3581765 0.0 1.1 771948 748492 ? S Aug11 0:06
dovecot/config
root 3585837 0.0 1.1 772828 749860 ? S Aug11 0:07
dovecot/config
root 3590276 0.0 1.1 773560 750520 ? S Aug11 0:07
dovecot/config
root 3594640 0.0 1.1 774612 752552 ? S Aug11 0:08
dovecot/config
root 3597417 0.0 1.1 774968 753076 ? S Aug11 0:35
dovecot/config
root 4116314 0.0 1.9 1280496 1258580 ? S 05:31 0:13
d
Re: Replication not working - GUIDs conflict - will be merged later
ok i went through this as well a bit
there is a replication full sync variable (i am having trouble finding it)
24h is the default but i might have rebuilt dovecot modifying this default
after i got things working i put everything back to default code.
yep i did
from dovecot-2.3.19/src/replication
see :
aggregator/replicator-connection.c:#define MAX_INBUF_SIZE 1024
aggregator/replicator-connection.c:#define REPLICATOR_MEMBUF_MAX_SIZE
1024*1024
aggregator/replicator-connection.c: conn->queue[i] =
buffer_create_dynamic(default_pool, 1024);
Binary file replicator/replicator-brain.o matches
replicator/replicator-settings.c: .replication_full_sync_interval
= 60*60*24,
replicator/notify-connection.c:#define MAX_INBUF_SIZE (1024*64)
Binary file replicator/doveadm-connection.o matches
Binary file replicator/.libs/replicator matches
replicator/replicator-brain.c: pool =
pool_alloconly_create("replication brain", 1024);
replicator/replicator-queue.c: queue->user_queue =
priorityq_init(user_priority_cmp, 1024);
replicator/replicator-queue.c: hash_table_create(>user_hash,
default_pool, 1024,
Binary file replicator/notify-connection.o matches
Binary file replicator/dsync-client.o matches
I do not believe there is a settable variable in dovecot.conf ?
I could be wrong.
the actual code containing the variable is below, change and recompile
all and that should/might help.
replicator/replicator-settings.c: .replication_full_sync_interval
= 60*60*24,
change to 24 so something more practical ?
note 60*60*24 is math (ie how many seconds in between full syncs) - ie
do not change 24 to 24h for example.
do this on both servers.
note that a full sync interval stress wise on the server is dependant on
how much physical mail you have in the mbox.
note that the full resync interval syncs both accounts from scratch.
also note 6hrs is not a bad place to start?
the replicator service will deal with this in the background
there are also other variables hard set (like i believe 15m for the
retry bad sync interval ?)
you will need to dig through the replicator code to find these.
Happy Tuesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 8/2/2022 9:30 AM, Sebastian Marske wrote:
On 8/1/22 11:15, Patrick Westenberg wrote:
Very interesting new insights:
When I use imapsync and let it synchronize mails from INBOX to
INBOX/testfolder, the automatic replication works fine.
All mails are synchronized between my two backends.
When I move the mails to the INBOX (doveadm move -u m...@example.com
INBOX mailbox INBOX/testfolder all), these mails are lost on the
replica! They are neither in INBOX, nor in INBOX/testfolder
Regards
Patrick
Hi,
every now and then I have the same problem on our servers. Currently,
I'm running Dovecot 2.3.19.1 as well, but I upgraded directly from
2.3.16 due to other issues with the versions in between.
Last time I observed a de-sync due to a GUID change, it appeared like
the user had moved a folder around in their mailbox. And indeed, the
output of 'doveadm mailbox status -u someuser guid '*' listed different
GUIDs. Dovecot actually logged some errors for this case:
Dovecot log from replica1:
Jul 27 12:06:08 replica1 dovecot[3431]:
doveadm(someuser)<10206>: Error: Duplicate
mailbox GUID 78c9dc2c0c0ee162c1080ca22142 for mailboxes
path/to/folder and path/to/folder-temp-1 - giving a new GUID
b0053e390f0ee162de27c9042436 to path/to/folder
Jul 27 12:06:08 replica1 dovecot[3431]:
doveadm(someuser)<10208>: Error: Duplicate
mailbox GUID 78c9dc2c0c0ee162c1080ca22142 for mailboxes
path/to/folder and path/to/folder-temp-1 - giving a new GUID
5823fe0d100ee162e027c9042436 to path/to/folder
Dovecot log from replica2:
Jul 27 12:06:04 replica2 dovecot[47018]:
doveadm(someuser)<2239>: Warning: Failed to do
incremental sync for mailbox path/to/folder, retry with a full sync
(uidnext 1 < 13)
Jul 27 12:06:04 replica2 dovecot[47018]:
doveadm(someuser)<2241>: Error: Duplicate
mailbox GUID 0ccaab01079031620e1e0ca22142 for mailboxes
path/to/folder and some/folder - giving a new GUID
78c9dc2c0c0ee162c1080ca22142 to path/to/folder
At that time, only replica2 was accepting imap connections.
In this particular case, Dovecot eventually managed to get things back
in sync after way over 24h, but I also had users out of sync for
multiple days.
Running 'doveadm -Dv sync -u someuser -d' manually gave me the same
error message, but didn't change anything.
Other things I've observed:
* it's not limited to a fixed set of users (unlike the
too-many-folders-thing with Dovecot 2.3.1[78])
* it's not limited to newly created users, but also affects users, that
have been in sync for months/years
* it's not limited to ma
Re: Doveadm Move Query
ok u...@domain.com needs to exist before any operations can be done on it. I discovered that dovecot does not consider a virtual mailbox active until it is returned in the user database see : doveadm user '*' both accounts MUST be returned in the list (user@.net & user@.com) from there it should work as expected. i went through this with my django email user interface as the user was not being saved in the database until the django model had completing saving a new entry, thus when creating the new account i had to put a delay check in my create email account that continued to loop until django had finished it's processing, very anoying (not dovecot's issue) but i think you are facing something similiar? it seems you might be renaming the mbox ? again both user@.net & user@.com must exist along the way before the account(s) can be accessed. if renaming the mbox is your intention than add the user@.com account move should now work then delete the user@.net account. Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 8/2/2022 6:49 AM, Simon B wrote: I have a production Dovecot problem and although I searched the mailing lists, I could not find an answer and I hope you can give me a quick answer/pointer in the right direction. I have mails for a user (u...@domain.net <mailto:u...@domain.net>) under /var/spool/mail/virtual/domain.net/user <http://domain.net/user> and I want to move ALL the mails to /var/spool/mail/virtual/domain.com/user <http://domain.com/user> If I use #doveadm -Dv move -u u...@domain.net <mailto:u...@domain.net> Maildir:/var/spool/mail/virtual/domain.net/user <http://domain.net/user> Maildir:/var/spool/mail/virtual/domain.com/user <http://domain.com/user> ALL I get doveadm(root): Fatal: Unknown argument MAILDIR:/var/spool/mail/virtual/domain.com/user <http://domain.com/user> if I use #doveadm -Dv move -uu...@domain.net <mailto:u...@domain.net> Maildir:/var/spool/mail/virtual/domain.net/user <http://domain.net/user> /var/spool/mail/virtual/domain.com/user <http://domain.com/user>ALL doveadm(root): Fatal: Unknown argument /var/spool/mail/virtual/domain.com/user <http://domain.com/user> What the hell am I doing wrong!? :) Thanks. Simon -- This message has been scanned for viruses and dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is believed to be clean.
Re: variable %w recursive expanding
ok not what you probably want to hear
c, python etc uses % as a variable indicator along with the next
character using a format variable (like %s being a string variable)
this is what is causing the issue
Not saying for sure but experience wise i am not familiar with any
system allowing a % sign in the password now a days ?
could be wrong, i guess it depends on the backbone password fetcher process
even if you could get dovecot (or any c based programming) to allow for
this clients would probably have similiar issues which there would be no
real control over.
another charater would also be '\' as it is used to being a delimiter
for the next character being absolute
along with \x00 (non ascii characters in a string)
again just an experience fyi
Happy Tuesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 8/1/2022 4:47 AM, Franz Beslmeisl wrote:
In order to change the password scheme I wrote a script named
updateproxy that needs the plain text password from the user.
To get that I use the line
password_query = SELECT username as user, password, \
'%w' as userdb_plain_pass FROM auth_user WHERE username='%n'
This works nicely with almost all passwords but not with this one
1234567%&/abcd
the error message being
dovecot: Failed to expand plugin setting plain_pass =
'1234567%&/abcd': Unknown variable '%&'
It seems to me that dovecot tries to do another level of variable
evaluation upon the **value** of the already evaluated variable.
So I searched for ways to escape problematic characters like %
and changed my line to
password_query = SELECT username as user, password, \
'%E{w}' as userdb_plain_pass FROM auth_user WHERE username='%n'
but this produces problems with password values containing quotes.
So how can I get a plain text password containing any ascii char
(or even better any utf-8 char) safely to my script?
Thanks for your suggestions
-- here the nasty details, if you want -
$ dovecot -n
# 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.7.2 ()
# OS: Linux 5.4.0-122-generic x86_64 Ubuntu 20.04.4 LTS
# Hostname: mx-10-2.bildung.hessen.de
auth_mechanisms = plain login
auth_username_chars =
abcdefghijklmnopqrstuvwxyz_0123456789.ABCDEFGHIJKLMNOPQRSTUVWXYZ-@
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_location = maildir:~/Maildir
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
# the following file contains a '%w'-line
args = /etc/dovecot/db1.conf
driver = sql
}
passdb {
# the following file contains a '%w'-line
args = /etc/dovecot/db2.conf
driver = sql
}
passdb {
# the following file contains no '%w'-line (just for detail)
args = /etc/dovecot/db3.conf
driver = sql
}
plugin {
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
sieve_max_actions = 64
sieve_max_redirects = 16
sieve_max_script_size = 10M
sieve_trace_debug = yes
sieve_user_log = ~/sievelog
sieve_vacation_dont_check_recipient = yes
sieve_vacation_use_original_recipient = yes
}
protocols = imap sieve lmtp
service auth {
unix_listener /var/spool/postfix/private/dovecot-auth {
group = postfix
mode = 0660
user = postfix
}
}
service imap {
executable = imap after-login
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
}
}
service after-login {
executable = script-login /etc/dovecot/updateproxy
user = vmail
}
service stats {
unix_listener stats-reader {
group = mail
mode = 0666
}
unix_listener stats-writer {
group = mail
mode = 0666
}
}
ssl_cert = was automatically rejected:%n%r
}
protocol imap {
imap_client_workarounds = delay-newmail
mail_max_userip_connections = 300
}
Re: Replication not working - GUIDs conflict - will be merged later
etdevelopments.ca
-q yes
ssl_cert = /programs/common/getssl.cert -c mail.hamletdevelopments.ca
-q yes
ssl_ca = /programs/common/getssl.cert -c mail.hamletdevelopments.ca
-q yes
}
note the sni.conf above suck in the certs from a db.
Another thought is are you running duplicate supression, i am not sure
how that would work when using imapsync (ie i have to assume a lot of
emails when you run a sync would carry the same info)
duplicate suppression seems to pickup on job numbers, to, from etc to
decide if an email is a duplicate. Maybe this is also an issue.
# cat duplicates.sieve
require "duplicate"; # for dovecot >= 2.2.18
if duplicate {
discard;
stop;
}
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 8/1/2022 5:15 AM, Patrick Westenberg wrote:
Very interesting new insights:
When I use imapsync and let it synchronize mails from INBOX to
INBOX/testfolder, the automatic replication works fine.
All mails are synchronized between my two backends.
When I move the mails to the INBOX (doveadm move -u m...@example.com
INBOX mailbox INBOX/testfolder all), these mails are lost on the
replica! They are neither in INBOX, nor in INBOX/testfolder
Regards
Patrick
Re: Replication not working - GUIDs conflict - will be merged later
Ok This is speculation but i understand the issue at a programming level what needs to be understood is that imap's uids & ugid's are relative to the host server the email is coming from. this is generally not an issue with replication on cyrus or dovecot because the server and the replication is being handled by the same server set (ie the same uids & guids etc are generated as things happen) example replicated data : -rw---1 vmail vmail uarch 185K Jul 29 09:30 1659101404.M875201P20192.mail19.scom.ca,S=189252,W=192431:2,S -rw---1 vmail vmail uarch 1.5K Jul 29 09:53 1659102818.M268117P41331.mail18.scom.ca,S=1583,W=1639:2,S -rw---1 vmail vmail uarch 1.0M Jul 29 12:52 1659113530.M841469P58214.mail18.scom.ca,S=1095861,W=1113817:2,S -rw---1 vmail vmail uarch 210K Jul 29 13:15 1659114913.M958008P31982.mail19.scom.ca,S=215405,W=219216:2,S you will note the originating server is in the mail file name (mail19 & mail18 in my case) this is how dovecot sorts out the uids etc on the fly. (i think) If i have read this correctly you are trying to sync to an external imap server that carries its own uids guids etc which will be different. where you are saying that you are using imap sync i assume you are using the unix version # imapsync Name: imapsync - Email IMAP tool for syncing, copying, migrating and archiving email mailboxes between two imap servers, one way, and without duplicates. Version: This documentation refers to Imapsync $Revision: 1.977 $ if so look at the --useuid: Use UIDs instead of headers as a criterion to recognize messages. Option --usecache is then implied unless --nousecache is used. and the --logfile (ie run a logging file when connecting the the external account), it might help with any errors being generated (run imapsync in debug mode to get full detail) basically using useuid deals with sometimes getting a different uid back from the origional server i go through this issue more with pop3 as it returns the id list starting at uid 1 (for example) instead of the actual uid against the email on the server. uids will force a proper sync (imap or pop3) because the uid on the server will always return the same uid for that email message and increments forward inside the account. if so then imap sync should be sorting this out when syncing the imap accounts ? (ie creating new usid guids etc) so assuming the above is happening the next question is are you using replication that is fully setup between the two servers or are you doing manual replication (ie running the doveadm command to do the sync?) (you mentioned using the backup command which would kinda work but full replication does the changes on the fly and should work) if you are running manual replication you should consider going to the live replication, it will sort out stuff as the imap folders sync etc. (or it should) The next thing to consider is there were some issues that were fixed in 2.3.19 replication, are you running the same dovecot versions on both servers ? I do a ton of emails, reporting etc and find that replication works well on dovecot 2.3.19 bewteen both of my mail servers. ie it does not matter which one receives the email it gets sorted out. If there is an error the replication will sort it out on the next sync run through the replication process running in the background. you can set all of the retries etc for replication in the config files. the merged later is probably indicating that dovecot will sort stuff out in the background (ie a reindex etc) but that is putting extra stress on the server(s), i used to get the merge or duplicate uids, guids on cyrus and it would try to sort it out on the fly. this would occur when one replicated server was offline and i was forcing a sync update after bringing it back online, this was the case because both servers had received emails into the same account from seperate sources thus the same uid was set for two different messages on each server. (fyi) with syrus a rebuild was the only was to sort this out dovecot seems way more resiliant in this department. again full replication setup would sort these issue out i expect as each server would handle stuff as it happens and adjust uid,guids accordingly. Happy Sunday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 7/31/2022 8:16 AM, Patrick Westenberg wrote: Hi everyone, I have a weird problem with replication that I'm unable to solve. A new account is sychronized from an external provider via imapsync. The mails end up on my backend1. I see that the folder structure is immediately replicated to backend2. However, a lot of mails are missing and "doveadm
Re: rawlog data in a lua script
pid_process =
( command.output.split(' ')[0] ) #whats left should be my process ?
break
print ('PID Process : %s ' %pid_process )
if options.destination == 'local' :
#Send to log here
print ('Sending to Local Syslog')
log = open ('/var/run/log','w')
log.write ('hello')
log.close()
sys.exit()
else : #Send via socket
#Make the line in freebsd
syslog format
MESSAGE = '<' + str(count) +
'>' + str( time.strftime("%b %d %H:%M:%S ") ) + str(options.label) +
'[' + str(pid_process) + ']: ' + str(line)
print ('Sent : %s' %MESSAGE )
count = count + 1
if count > 255 :
count = 10
# send to udp logger port specified
sock =
socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes(MESSAGE,
"utf-8"), (UDP_IP, UDP_PORT))
sock.close()
else : #No data
pass
except Exception as e:
exc_type, exc_obj, exc_tb = sys.exc_info()
fname =
os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
e = str(e) + '\n\n' + str(exc_type) + '\n' +
str(fname) + '\n' + str(exc_tb.tb_lineno)
print ('\n\nCaught Exception : %s' %e )
print ("Could not read cmd pipe, skipping ...")
sys.exit()
---
and lib.py
--
## cat lib3.py
#Python Library written by paul kudla (c) 2011
#Load the librarys for the system
import os,sys,time,socket
import string
from ftplib import FTP
from decimal import *
from datetime import date
import datetime
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.base import MIMEBase
from email.mime.text import MIMEText
from email.utils import COMMASPACE, formatdate
from email import encoders
import subprocess
getcontext().prec = 20
class commands:
def __init__(self,command) :
self.command = command
#print (self.command)
self.output = 'Error'
self.status = '255'
#sample
#rc, gopath = subprocess.getstatusoutput('ls -a')
self.status, self.output =
subprocess.getstatusoutput(self.command)
try:
self.cr = self.output.split('\n')
except :
self.cr = []
try:
self.count = len(self.cr)
except :
self.count = 0
self.status = int(self.status)
#return count=number of lines, cr = lines split,
getoutput = actual output returned, status = return code
return
#Email with attachment
class sendmail:
def __init__(self, send_from, send_to, send_subject, send_text,
send_files):
#send_from, send_to, send_subject, send_text, send_files):
#print ('lib.py sending email')
assert type(send_to)==list
assert type(send_files)==list
msg = MIMEMultipart()
msg['From'] = send_from
msg['To'] = COMMASPACE.join(send_to)
msg['Date'] = formatdate(localtime=True)
msg['Subject'] = send_subject
msg.attach( MIMEText(send_text) )
for f in send_files:
part = MIMEBase('application', "octet-stream")
part.set_payload( open(f,"rb").read() )
Encoders.encode_base64(part)
part.add_header('Content-Disposition',
'attachment; filename="%s"' % os.path.basename(f))
msg.attach(part)
try : #Send Local?
smtp = smtplib.SMTP('mail.local.scom.ca')
#smtp.login('bac...@scom.ca','522577')
#print ('Sending Email to : %s' %send_to)
smtp.sendmail(send_from, send_to, msg.as_string())
smtp.close()
except :
Thunderbird / Copy to Send folder times out
I know this has been asked before in both Dovecot & Thunderbird Forums (i will posting this info there as well) It seems every once in a while that copy to sent folder timeout occurs and you have to hit retry a few time for it to complete I know this is a network issue (ie network speed, congestion etc does cause an issue) That being said is there an imap timeout variable that would addres this issue and or does anyone know about a thunderbird variable I have been unable to locate anything specific to this issue and am aware that we probably need to patch both sides? ideas anyone ?? -- Happy Thursday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca
Re: Is multi factor authentication practical/feasible?
Please note this is my opinion only It seems any kind of dual auth will need a security app running on YOUR server saving toikens, logins etc etc this is what lead to microsoft, gmail etc having their own api which will only work for them this is also (mainly because of https authing the device) what makes it hard to proxy oauth2 etc If you look at sogo's documentation they have a java server applet Still working on the install to make work with my system but in general you need your own whatever app to track oauth2 5.7. Authenticating using C.A.S. SOGo natively supports C.A.S. authentication. For activating C.A.S. authentication you need first to make sure that the SOGoAuthenticationType setting is set to cas, SOGoXSRFValidationEnabled is set to NO and that the SOGoCASServiceURL setting is configured appropriately. I myself will eventually get around to implimenting this on one of my servers ? logically i will have to track tokens etc via https like google etc basically the reality is every server will have it's own token base etc thus preventing any kind of a standard. Happy Sunday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 7/3/2022 9:50 AM, John Gateley wrote: On 7/3/22 8:31 AM, John Gateley wrote: The protocols were designed long before SAML and OIDC. SAML/OIDC give you more control over authn/z and allow easily adding in MFA or other different types of auth. To do this right, you'd need to extend the protocol to allow OIDC or SAML. I did find this RFC - I haven't read it, but it applies directly: https://datatracker.ietf.org/doc/html/rfc7628 j
Re: Multidomain ssl config ?
John please send me a direct email address
I understand what you need and my customers are all seperate certs per domain
on both sides
I spent over three months setting stuff up
I wil send complete instructions for both postfix & dovecot
Plus auto scripts etc
You will need to be running a postgresql database for my stuff to work without
mods
And running python 2.xx
thanks - paul
Paul Kudla SCOM.CA Internet Services Inc.004-1009 Byron Street
South Whitby, Ontario - Canada L1N 4S3Toronto 416.642.7266 Main
1.866.411.7266 Fax 1.888.892.7266
On Jun 29, 2022 at 16:39:29 EDT, John Stoffel
wrote:
>>>>> "Maurizio" == Maurizio Caloro writes:
Maurizio> on postfix now this seems to run, and with dovecot i need
Maurizio> also handle this two domains, but appairing this error
Maurizio> messages. like:
Why aren't you just using a single domain as the MX record for all the
domains? Then you only need one SSL cert pair for all of this, and if
you publish the right SPF records, each domain can send from the same
MX host as well.
Maurizio> Jun 29 20:49:28 Dovecot/imap-login: Info: Disconnected (no auth
attempts in 0 secs): user=<>,
Maurizio> rip=a.b.c.d, lip=37.120.190.188, TLS handshaking: SSL_accept()
failed: error:14094416:SSL routines:
Maurizio> ssl3_read_bytes:sslv3 alert certificate unknown: SSL alert number 46,
session=
Maurizio> Running with Debian Buster
Maurizio> # dovecot --version
Maurizio> 2.3.4.1 (f79e8e7e4)
Maurizio> # nmail.caloro.ch
Maurizio> local_name nmail.caloro.ch {
Maurizio> ssl_cert = ssl_key = }
Maurizio> # nmail.calm-ness.ch
Maurizio> local_name nmail.calm-ness.ch {
Maurizio> ssl_cert = ssl_key = }
Maurizio> thanks for possible help
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Re: Issue with one user only, exceeding connections
ok the idle connection per folder is a factor
however in thunderbird i believe it defaults to 2 simultanious connections
mine is set to 5
in thunderbird see
The solution is to reduce the maximum number of connections in
Thunderbird. This can be done from Edit > Account Settings > Server
Settings (under the mail account for which the setting should be
modified) > Advanced > Maximum Number of server connections to cache.
I dont know of anything that would get it to 500?
as for outlook idle was not / is not supported past 2010 (if even that)
you need to go into
file --> options --> advanced --> send/receive
all you can change in there is the timing which defaults to 30 minutes,
i recommend 3 or 5
I am unaware of how outlook handles physical connections (maybe
registery?) and google revieled nothing, outlook since 2010 just does
not support imap, microsofts way of forcing everyone onto exchange /
outlook 365
377,000 hits last time i googled imap issues in outlook.
Best suggestion is to run
# ps -axww | grep imap
25500 - S 0:00.57 imap: [p...@hiscomputer.ca 172.97.150.95
IDLE] (imap)
25530 - S 0:00.36 imap: [p...@hiscomputer.ca 172.97.150.95
IDLE] (imap)
26014 - I 0:00.39 imap: [rco...@tnky.ca 172.97.128.227 IDLE]
(imap)
26018 - I 0:00.38 imap: [rco...@tnky.ca 172.97.128.227 IDLE]
(imap)
26210 - I 0:00.07 imap: [spa...@scom.ca 99.238.154.160 IDLE]
(imap)
38911 - S 0:00.17 imap: [marilynla...@scom.ca 142.188.149.199
IDLE] (imap)
38912 - S 0:00.13 imap: [marilynla...@scom.ca 142.188.149.199
IDLE] (imap)
41306 - S 0:00.73 imap: [ed.ha...@dssmgmt.com 204.237.48.37
IDLE] (imap)
41312 - S 0:00.63 imap: [ed.ha...@ekst.ca 204.237.48.37 IDLE]
(imap)
45232 - I 0:00.23 imap: [rco...@tnky.ca 172.97.128.227 IDLE]
(imap)
55504 - I 0:00.16 imap: [rco...@tnky.ca 172.97.128.227 IDLE]
(imap)
which shows all imap connections and from where
if you are overflowing 500+ connections then it has to show up here.
Happy Thursday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 6/8/2022 6:41 PM, Jeremy Schaeffer wrote:
Ahhh, Ok, I did not know that and now that makes sense. I did not
realize it held a open connection for each folder. I increased that and
I will see what happens. I wonder if that will also effect the outlook
issues. Thanks! - Jeremy
On 6/8/2022 14:28, Frank-Ulrich Sommer wrote:
I think if IMAP IDLE is used you need one connection per folder. If I
remember correctly at least either Thunderbird or K9 Mail (I'm using
both too) use one connection per selected directory. Simply increasing
the number of connections was the easiest solution as I only have very
few users too.
Regards
Frank
Am 8. Juni 2022 21:14:23 MESZ schrieb Jeremy Schaeffer
:
I keep having this issue with one user, and I have to restart dovecot
several times a day to clear it. What I have is a postfix / dovecot mail
server (Centos 7) and about a dozen users. All mailboxes are imap ssl. I
monitor about 4 mailboxes on my computer and tablet. I use Thunderbird
on the computer (cache connections at 2) and K9 on the tablet, but one
user of the four I keep getting "Maximum number of connections from
user+IP exceeded" and I have the maximum at 50
"(mail_max_userip_connections=50)" so its hard for me to believe I am
actually exceeding it unless dovecot/client is not dropping connections
and keeps starting new ones until it reaches the maximum, but again,
only for one user, even though I am monitoring 4 on the same devices.
Any idea how to troubleshoot this? I don't know if I should be looking
at dovecot or the clients, or what I need to look for. It's been going
on since I put this server in use over a year ago. I also have issues
with Outlook clients disconnecting, just outlook, is there any
recommended settings to make Outlook work smoother?
Thanks! - Jeremy
Config -
# 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-1160.11.1.el7.x86_64 x86_64 CentOS Linux release
7.9.2009 (Core)
# Hostname: ***
auth_mechanisms = plain login
debug_log_path = /var/log/dovecot_debug.log
first_valid_gid = 500
last_valid_gid = 600
last_valid_uid = 600
listen = *
mail_location = maildir:~/Maildir
mbox_write_locks = fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
Re: Occasional service disruptions
ok i do NOT use roundcube however trying to help
regarding :
> Jun 06 12:01:22 imap(user1)<29639>: Warning: Inotify
> instance limit for user 500 (UID vmail) exceeded, disabling. Increase
> /proc/sys/fs/inotify/max_user_instances
this to me indicates that all users are logging in under one user name /
process at least according to dovecot?
usually when an imap connection is started it starts its own pid per
mailbox opened process (at least for me it does)
both dovecot & cyrus work this way
number of simitanulus connections is usually handled by the client
(thunderbird, outlook whatever)
roundcube would be considered a client thus the overflow in connections
if it is opening everthing under one connection / user
it gets complicated but i would start by checking if different users are
actually being logged in
if so then try closing the connection via roundqube and see if the
connection drops off on the dovecot server.
# dovecot.who
username # proto (pids)(ips)
epo...@scom.ca 4 imap (20263 74767 74743 75194) (174.114.171.16)
install...@tomkudla.ca 7 imap (28281 28280 69830 69832 69834 69836
69838) (167.94.196.10)
ditchb...@scom.ca 1 imap (41136) (65.39.148.2)
recept...@clancyca.com 1 imap (41133) (65.39.148.2)
e...@scom.ca 4 imap (36344 25879 89306 89308) (204.237.48.37)
rco...@tnky.ca 6 imap (91131 23791 8700 16087 91176 91179)
(172.97.128.227)
ca...@scom.ca 1 imap (88120) (216.58.34.142)
p...@scom.ca 1 imap (36202) (69.60.225.80)
ditchb...@clancyca.com 1 imap (40942) (65.39.148.2)
the max_user_instances is meant to control how many connections per user
thus 500 is way more for multiple clients per user to log in with
i have several customers (like myself) that open connections from
multiple locations without issues.
is it possible that roundcube is opening the same user multiple times
and not closing the connection after a while?
I get the above logged in list above from
doveadm mailbox status -t all -u $1 '*'
something to consider.
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 6/6/2022 11:38 AM, Nikolaos Milas wrote:
Hello,
On a server with (Postfix and) Dovecot 2.3.18 (on a VM running CentOS 7
- 1 CPU, 5 GB RAM) with the config you will see below, we are facing
occasional (infrequent) service disruptions: IMAP service seems
unavailable to some users.
Jun 6 12:01:25 vweb2 roundcube: <1eecb0d4> IMAP Error: Login failed for
imaptester against vmail2.noa.gr from 195.251.202.xxx. Could not connect
to ssl://vmail2.noa.gr:993: Connection rejected in
/var/webs/webmail/rcube/program/lib/Roundcube/rcube_imap.php on line 211
(POST /?_task=login&_action=login)
At that time there was no associated logged event in dovecot log. (Other
users are logging in and out.)
However, I see some warnings (I list the two of them closest to the
above event):
Jun 06 12:01:22 imap(user1)<29639>: Warning: Inotify
instance limit for user 500 (UID vmail) exceeded, disabling. Increase
/proc/sys/fs/inotify/max_user_instances
...
Jun 06 12:01:26 imap(user2)<29793>: Warning: Inotify
instance limit for user 500 (UID vmail) exceeded, disabling. Increase
/proc/sys/fs/inotify/max_user_instances
(In above log excerpts I've only modified real usernames.)
Restarting Dovecot returns things back to normal.
I have tried to use "service_count = 100" in all configured services, to
see how it goes.
Most of the config is inherited from the past (older versions) and is
not optimized. For example one can observe different "process_limit"
values for different services, for no apparent reason I am aware of.
Could anyone suggest changes and/or additions to the OS and/or Dovecot
to resolve this issue?
Any additional suggestions will also be welcome.
Thanks in advance for your kind assistance.
Here is the config (I've only changed postmaster address):
===
protocols = imap pop3 sieve lmtp
login_greeting = Dovecot NOA ICXC-NIKA
log_path = /var/log/dove.log
mail_location = maildir:~/Maildir/
mail_gid = 500
mail_uid = 500
auth_mechanisms = plain login
auth_username_format = %Ln
auth_verbose = no
auth_debug = no
mail_debug = no
disable_plaintext_auth = no
mail_plugins = quota mail_log notify
protocol imap {
imap_client_workarounds = "delay-newmail"
mail_plugins = quota imap_quota mail_log notify
mail_max_userip_connections = 400
namespace inbox {
mailbox Trash {
autoexpunge = 15d
}
}
}
protocol pop3 {
mail_max_userip_connections = 3
mail_
Re: Replicator: Panic: data stack: Out of memory
did you get this figured out / working
getting posts out of order - just wanted to make sure your ok?
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 6/6/2022 4:03 AM, Ivan Jurišić wrote:
Dana 04.06.2022 15:34, Paul Kudla (SCOM.CA Internet Services Inc.) je
napisao(la):
ok thanks for the info
from here you need to turn on full debugging and then filter the
log by
"replicat"
Now replication work when set vsz_limit in service aggregator and remove
parametar replication_dsync_parameters and
replication_full_sync_interval from my 90-replicator.conf. Now my
configuration work for replication on another mail server.
Config file for replication /etc/dovecot/conf.d/90-replicator.conf
--
service aggregator {
vsz_limit = 256M
fifo_listener replication-notify-fifo {
user = vmail
}
unix_listener replication-notify {
user = vmail
}
}
service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
mode = 0600
user = vmail
}
}
service doveadm {
inet_listener {
port = 12345
ssl = no
}
}
replication_max_conns = 100
#replication_dsync_parameters = -d -N -l 30 -U
#replication_full_sync_interval = 1 days
doveadm_port = 12345
doveadm_password = Jados82!
plugin {
mail_replica = tcp:imap.myserv2.local:12345
}
Re: Replicator: Panic: data stack: Out of memory
ok thanks for the info from here you need to turn on full debugging and then filter the log by "replicat" It is starting to look like you have a bad email somewhere causing the crash I have yet to see this however anything is possible, when you replicate anything it needs to update indexes & control files on the other side and vice versa that being said its a good place to start looking? At least see in the debug mail logs what it is trying to replicate and where it fails. Im running a large email server and in 2.3.18 there was a bug that any account over 300+ folders would fail to replicate. This appears to be fixed in 2.3.19 other then that TCP replication etc seems to work the best if you look back on the mailing lists i have uploaded a sample config of my systems which seems to work ok. Happy Saturday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 6/3/2022 10:46 PM, Ivan Jurišić wrote: Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram. I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator proces got crash. -- This message has been scanned for viruses and dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is believed to be clean.
Re: Replicator: Panic: data stack: Out of memory
0011M 3732K rpcsvc 15 29.7H 2.26% nfsd 1987 pgsql 1 200 195M47M select 5 33:21 1.84% postgres 1985 root 34 210 141M88M sigwai 14 72:22 1.32% named 1937 root 1 20027M15M select 15 491:36 0.90% python3.8 99555 root 1 20028M18M select 10 634:23 0.88% python3.8 1939 root 1 20027M15M select 1 939:47 0.87% python3.8 1988 pgsql 1 200 195M47M select 7 6:58 0.28% postgres 1989 pgsql 1 200 195M47M select 8 2:14 0.17% postgres 1964 pgsql 1 200 194M 164M select 9 10:02 0.08% postgres 85373 root 1 20014M 3644K CPU0 0 0:00 0.07% top 3150 pgsql 1 200 195M42M select 6 39:21 0.06% postgres ps -axw or ps -axww or freebsd # ps -axww PID TT STAT TIME COMMAND 0 - DLs 3788:48.94 [kernel] 1 - ILs0:05.38 /sbin/init -- 2 - DL 0:00.00 [crypto] 3 - DL 0:00.00 [crypto returns 0] 4 - DL 0:00.00 [crypto returns 1] 5 - DL 0:00.00 [crypto returns 2] 6 - DL 0:00.00 [crypto returns 3] 7 - DL 0:00.00 [crypto returns 4] 8 - DL 0:00.00 [crypto returns 5] 9 - DL 0:00.00 [crypto returns 6] 10 - DL 0:00.00 [audit] 11 - RNL 1629112:33.34 [idle] 12 - WL 180:00.70 [intr] 13 - DL 123:57.70 [geom] 14 - DL 0:00.00 [crypto returns 7] 15 - DL 0:00.00 [crypto returns 8] 16 - DL 0:00.00 [crypto returns 9] 17 - DL 0:00.00 [crypto returns 10] 18 - DL 0:00.00 [crypto returns 11] 19 - DL 0:00.00 [crypto returns 12] 20 - DL 0:00.00 [crypto returns 13] 21 - DL 0:00.00 [crypto returns 14] 22 - DL 0:00.00 [crypto returns 15] 23 - DL 0:00.00 [sequencer 00] 24 - DL 0:00.00 [cam] 25 - DL 5:42.32 [usb] 26 - DL 0:00.47 [soaiod1] 27 - DL 0:00.47 [soaiod2] 28 - DL 0:00.46 [soaiod3] 29 - DL 0:00.47 [soaiod4] 30 - DL 1714:58.15 [zfskern] 31 - DL 0:00.00 [sctp_iterator] 32 - DL12:50.77 [pf purge] 33 - DL 2:16.82 [rand_harvestq] 34 - DL29:00.62 [pagedaemon] 35 - DL 0:00.00 [vmdaemon] 36 - DL 5:25.68 [bufdaemon] 37 - DL 1:44.98 [vnlru] 38 - DL 2040:33.82 [syncer] 1657 - Is 0:01.21 /sbin/devd 1863 - Ss 0:03.44 /usr/sbin/rpcbind 1878 - Is 0:00.08 /usr/sbin/mountd -r -S 1880 - Is 0:00.27 nfsd: master (nfsd) 1882 - S 1780:23.16 nfsd: server (nfsd) 1907 - Ss10:01.06 /usr/sbin/syslogd -s 1909 - Is 0:00.55 /usr/sbin/inetd -wW -C 50 -s 500 1911 - Is 0:00.25 /usr/sbin/sshd 1955 - Is24:50.70 /usr/local/sbin/clamd 1964 - Ss10:02.28 postmaster: checkpointer(postgres) 1965 - Ss 1:38.52 postmaster: background writer(postgres) 1966 - Ss 3:48.60 postmaster: walwriter(postgres) 1967 - Ss 2:03.84 postmaster: autovacuum launcher(postgres) 1968 - Ss12:41.60 postmaster: stats collector(postgres) 1969 - Is 0:01.82 postmaster: logical replication launcher (postgres) 1974 - Ss37:19.26 postmaster: walsender pgsql 10.221.0.16(30421) (postgres) 1976 - Ss39:37.29 postmaster: walsender pgsql 10.221.0.10(64872) (postgres) 1985 - Is72:21.96 /usr/local/sbin/named -d 0 -4 1986 - Ss 426:29.15 postmaster: pgsql scom_billing 10.221.0.18(52852) (postgres) 1987 - Ss33:21.50 postmaster: pgsql scom_billing 10.221.0.18(60830) (postgres) 1988 - Ss 6:57.70 postmaster: pgsql scom_billing 10.221.0.18(34255) (postgres) 1989 - Ss 2:13.52 postmaster: pgsql scom_billing 10.221.0.18(17265) (postgres) 2073 - Ss10:12.46 /usr/local/libexec/postfix/master -w 2076 - I 0:07.82 qmgr -l -t fifo -u 2166 - Is 1:53.61 /usr/local/libexec/postfix/master -w 2168 - I 0:55.23 qmgr -l -t fifo -u 2238 - Is 1:49.77 /usr/local/libexec/postfix/master -w 2240 - I 1:01.17 qmgr -l -t fifo -u 2253 - I 0:39.34 tlsmgr -l -t unix -u 2397 - Is 0:05.58 MailScanner: starting child (perl) 2513 - Is 0:20.43 /usr/sbin/cron -s 3150 - Rs39:21.01 postmaster: walsender pgsql 10.221.0.6(1) (postgres) 3175 - Is 0:00.35 postmaster: pgsql scom_billing 10.221.0.6(10017) (postgres) 3176 - Is 0:10.80 postmaster: pgsql scom_billing 10.221.0.6(10018) (postgres) 3177 - Ss 1:10.22 postmaster: pgsql scom_billing 10.221.0.6(10019) (postgres) Happy Saturday !!! Thanks
Re: Replicator: Panic: data stack: Out of memory
just an fyi domain is registered and appears to be active so there should not be any issues with the domain .website is an actual domain (like .com, .ca etc) however i did note ZAKARIA.WEBSITE.14400 IN MX 10 ZAKARIA.WEBSITE. usually the mx record points to an actual sub domain like mail. or whatever if you are running everything on one server then this is ok its just usually better to seperate the mx record incase you want to goto a different server down the road. Domain Name: ZAKARIA.WEBSITE Registry Domain ID: D198561373-CNIC Registrar WHOIS Server: whois.ionos.com Registrar URL: https://ionos.com Updated Date: 2021-11-02T01:42:25.0Z Creation Date: 2020-08-29T09:28:59.0Z Registry Expiry Date: 2022-08-29T23:59:59.0Z Registrar: IONOS SE Registrar IANA ID: 83 Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited Registrant Organization: 1&1 Internet Limited Registrant State/Province: GLS Registrant Country: GB Registrant Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Admin Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Tech Email: Please query the RDDS service of the Registrar of Record identified in this output for information on how to contact the Registrant, Admin, or Tech contact of the queried domain name. Name Server: NS1.ZAKARIA.WEBSITE Name Server: NS2.ZAKARIA.WEBSITE ## nslookup ZAKARIA.WEBSITE Server: 10.220.0.2 Address:10.220.0.2#53 Non-authoritative answer: Name: ZAKARIA.WEBSITE Address: 213.171.210.111 Name: ZAKARIA.WEBSITE Address: 2a00:da00:1800:834c::1 ## dig mx ZAKARIA.WEBSITE ; <<>> DiG 9.14.3 <<>> mx ZAKARIA.WEBSITE ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32110 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: ba2f2ec47dfcc90f458d629b4d2855567ad8dfa57bf8 (good) ;; QUESTION SECTION: ;ZAKARIA.WEBSITE. IN MX ;; ANSWER SECTION: ZAKARIA.WEBSITE.14400 IN MX 10 ZAKARIA.WEBSITE. ;; ADDITIONAL SECTION: zakaria.website.14372 IN A 213.171.210.111 zakaria.website.14372 IN 2a00:da00:1800:834c::1 ;; Query time: 87 msec ;; SERVER: 10.220.0.2#53(10.220.0.2) ;; WHEN: Sat Jun 04 08:16:40 EDT 2022 ;; MSG SIZE rcvd: 147 Happy Saturday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 6/4/2022 6:07 AM, Marc wrote: I think it is because of the domain, obviously you should only be running a website. -Original Message- From: dovecot On Behalf Of hi@zakaria.website Sent: Saturday, 4 June 2022 11:15 To: Dovecot Subject: Re: Replicator: Panic: data stack: Out of memory On 2022-06-04 02:46, Ivan Jurišić wrote: Ok a little more help : vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g etc pending avaliable ram. I try with 524M, 1G, 2G, 4G and 8G but in any case repclicator proces got crash. Maybe there is another service process causing OOM? e.g. check clamd, antivirus DBs tend to be quite big and in updating for sometime becomes double the size due to reloading. Also, somtimes httpd service when using event worker, and its not tuned properly, it will cause the OOM crash to other service along itself.
Re: Replicator: Panic: data stack: Out of memory
Ok a little more help :
vsz_limit = 0 --> means unlimited ram for allocation, change this/try 2g
etc pending avaliable ram.
I use :
service replicator {
client_limit = 0
drop_priv_before_exec = no
idle_kill = 4294967295s
process_limit = 1
process_min_avail = 0
service_count = 0
unix_listener replicator-doveadm {
mode = 0600
user = vmail
}
vsz_limit = 8192M
}
next :
replication_max_conns = 100
think of this as a socket that gets opened everytime you connect to the
other server
100 means it will handle 100 seperate replication user requests
at a time (which is way more then sufficent)
if the connections overflows it just means the replication que will
carry on when resources opens up thus not killing the server resources.
If a que overflows because too much other stuff is going on it will just
reque and go again.
also note :
service aggregator {
process_limit = 1000
#vsz_limit = 1g
fifo_listener replication-notify-fifo {
user = vmail
group = vmail
mode = 0666
}
}
Happy Friday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 6/3/2022 5:13 AM, Ivan Jurišić wrote:
Reproduce of error (Replicator: Panic: data stack: Out of memory):
==
When replay on any mail I will got OOM.
I try add in service replicate vsz_limit = 0 but that not help (after
proces dovecot/replicator eat all ram I got OOM)
Log error:
==
Jun 03 09:38:59 Warning:
imap(ivan@myserv.local)<2533034>:
replication(ivan@myserv.local): Sync failure: Timeout in 2 secs
Jun 03 09:39:03 Panic: replicator: data stack: Out of memory when
allocating 4294967336 bytes
Jun 03 09:39:03 Error: replicator: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7feef08f0582]
-> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7feef08f069e]
-> /usr/lib/dovecot/libdovecot.so.0(+0x1022fb) [0x7feef08fd2fb] ->
/usr/lib/dovecot/libdovecot.so.0(+0x102391) [0x7feef08fd391] ->
/usr/lib/dovecot/libdovecot.so.0(+0x55589) [0x7feef0850589] ->
/usr/lib/dovecot/libdovecot.so.0(+0x54d52) [0x7feef084fd52] ->
/usr/lib/dovecot/libdovecot.so.0(+0xfb808) [0x7feef08f6808] ->
/usr/lib/dovecot/libdovecot.so.0(+0x122938) [0x7feef091d938] ->
/usr/lib/dovecot/libdovecot.so.0(+0xf76bf) [0x7feef08f26bf] ->
/usr/lib/dovecot/libdovecot.so.0(+0xf777e) [0x7feef08f277e] ->
/usr/lib/dovecot/libdovecot.so.0(buffer_append+0x61) [0x7feef08f2a21] ->
dovecot/replicator(replicator_queue_push+0x15a) [0x55953ece9b7a] ->
dovecot/replicator(+0x62b6) [0x55953ece92b6] ->
dovecot/replicator(+0x594e) [0x55953ece894e] ->
dovecot/replicator(+0x5b2c) [0x55953ece8b2c] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7feef0913529]
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132)
[0x7feef0914c12] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x50)
[0x7feef09135d0] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40)
[0x7feef0913790] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7feef0886353] -> dovecot/replicator(main+0x18d) [0x55953ece7cbd] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7feef0651d0a]
-> dovecot/replicator(_start+0x2a) [0x55953ece7d7a]
Jun 03 09:39:03 Fatal: replicator: master: service(replicator): child
2532886 killed with signal 6 (core dumps disabled -
https://dovecot.org/bugreport.html#coredumps)
My configuration:
==
# 2.3.19 (b3ad6004dc): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.19 (4eae2f79)
# OS: Linux 5.10.0-13-amd64 x86_64 Debian 11.3 ext4
# Hostname: mail.myserv.local
auth_mechanisms = plain login
auth_verbose = yes
debug_log_path = /var/log/dovecot.debug
default_client_limit = 5000
default_process_limit = 1000
default_vsz_limit = 2 G
deliver_log_format = msgid=%m, subject=%s, from=%f, size=%p(%w), %$
dict {
quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
doveadm_password = # hidden, use -P to show it
hostname = myserv.local
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *
log_path = /var/log/dovecot.log
mail_home = /var/mail/vhosts/%d/mail/%n
mail_location =
maildir:/var/mail/vhosts/%d/mail/%n:INDEX=/var/mail/vhosts/%d/indexes/%n
mail_plugins = " quota fts fts_solr notify replication"
mail_privileged_group = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext imapsieve vnd.dovecot.imapsieve
namespace inbox {
in
Re: Building dovecot-core
ok download the tarball for BOTH dovecot & pigeonhole unzip / whatever into a build src dir i use /programs/src/mail dovecot-2.3.19 for the core dovecot-2.3-pigeonhole-0.5.19 for pigeonhole make a currentconfig script file containing [13:57:42] peer1.scom.ca [paul:0] /programs/src/mail/dovecot-2.3.19.new ## cat currentconfig ./configure --with-pgsql --with-lucene (i use pgsql & lucene for the search engine) run the config (remember THIS IS THE SOURCE TARBALL NOT A BUILD) ie --> ./configure --with-pgsql --with-lucene gmake gmake install then make pigeonhole same as above currentconfig to contain ## cat currentconfig ./configure --with-dovecot=/usr/local/lib/dovecot note you need to do a chmod +x to the currentconfig file in order to run it. again like above gmake gmake install thats the basics. Happy Thursday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 6/2/2022 1:23 PM, dovecot-boun...@dovecot.org wrote: On 2022-06-02 17:03, Dmitriy Fitisov wrote: Hi all, do we have any info on building dovecot-core? INSTALL.md suggests usual way: ./configure … Needless to say, there is no .configure in root dir. I think, instructions need to be updated to correct this. Thank you. Dmitriy Hi there, I think you need to run ./autogen.sh first. Good luck. Zakaria.
Re: Restrict IMAP login, but allow Postfix SASL
You really need to database your passwd auth sasl supports pgsql / mysql you can then alter the queries by selecting flags pending the access you want to allow dovecot-pgsql.conf password_query = SELECT username as user, password FROM email_users WHERE username = '%u' and password <> 'alias' and status = True and destination = '%u' user_query = SELECT home, uid, gid FROM email_users WHERE username = '%u' and password <> 'alias' and status = True and destination = '%u' #iterate_query = SELECT user, password FROM email_users WHERE username = '%u' and password <> 'alias' and status = True and destination = '%u' iterate_query = SELECT "username" as user, domain FROM email_users WHERE status = True and alias_flag = False postfix smtpd.conf -> /usr/local/lib/sasl2/smtpd.conf #Postygres pwcheck_method: auxprop mech_list: PLAIN LOGIN auxprop_plugin: sql sql_engine: pgsql sql_hostnames: pg.scom.ca:5433 sql_database: scom_billing sql_user: sql_passwd: sql_select: SELECT password FROM email_users WHERE username = '%u@%r' and password <> 'alias' and currentcount_bad < 30 and status = True i use status to allow disabling the users ability to login but you can easily expand the select statement to include another flag for imap user(s) Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 6/1/2022 7:40 AM, dovecot-boun...@dovecot.org wrote: Hi, we have a very simple user-/passdb (like passwd) to authenticate virtual IMAP users. We also use this for Postfix authentication. Nothing special. But, we need to exclude some of the users from IMAP login. This means, some users should be allowed to send mail via Postfix (submission) and therefore authenticate via SASL against dovecot successfully, but they should not be allowed to login to their IMAP mail box. How could this be done? Thanks! -lutzn
Re: Force TCP socket disconnect on imap login failure?
iptables (linux) & pf firewall (freebsd) do drop the packets immediately
as the tables are updated.
I know this from experience as I use freebsd for the mail system's and
my asterisk voip server use linux
At the end of the day the logging has to drive the updates, the only way
to protect yourself against a brute force attack while it is happening
is to have the logging trigger a direct ip table update in the background
It is my experience that this IS extremely system resource extensive
(why i now run a seperate logging server)
even with dedicated hardware etc I found it impractical to try to do
this in real time because by the time i hit the trigger, then updated
the database and then updated pf firewall / iptables accordingly usually
the connection was over anyways.
this issue also exists in postfix where their logging does not allow a
signle line in syslog to indicate sasl user & ip address which makes it
near impossible to track bad ip's / user logins. I ended up patching
postfix sasl auth programming to add a combined line to track stuff like
this.
In ALL cases the attack is usually over before you can do anything about
it anyways.
Best to just plan for the future.
Below is a copy of the Auth penalty support which will help curve this
issue but not stop it .
It seems to be a balanced approach, postfix carries similiar config's to
acomplish the same thing.
---
from :
https://doc.dovecot.org/configuration_manual/authentication/auth_penalty/
Authentication penalty support
Dovecot anvil process tracks authentication penalties for different IPs
to slow down brute force login attempts. The algorithm works by:
First auth failure reply will be delayed for 2 seconds (this
happens even without auth penalty)
AUTH_PENALTY_INIT_SECS in src/auth/auth-penalty.h
The delay will be doubled for 4 -> 8 seconds, and then the upper
limit of 15 seconds is reached.
AUTH_PENALTY_MAX_SECS and AUTH_PENALTY_MAX_PENALTY in
src/auth/auth-penalty.h
If the IP is in login_trusted_networks (e.g. webmail), skip any
authentication penalties
If the username+password combination is the same as one of the last
10 login attempts, skip increasing authentication penalty.
CHECKSUM_VALUE_PTR_COUNT in src/anvil/penalty.c
The idea is that if a user has simply configured the password
wrong, it shouldn’t keep increasing the delay.
The username+password is tracked as the CRC32 of them, so there
is a small possibility of hash collisions
Problems:
It is still possible to do multiple auth lookups from the same IP
in parallel.
For IPv6 it currently blocks the entire /48 block, which may or may
not be what is wanted.
PENALTY_IPV6_MASK_BITS in auth-penalty.c
Authentication penalty tracking can be disabled completely with:
service anvil {
unix_listener anvil-auth-penalty {
mode = 0
}
}
Also you can have similar functionality with fail2ban.
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 5/24/2022 9:55 PM, John Hardin wrote:
On Tue, 24 May 2022, Hippo Man wrote:
I have already been doing the following for the past year or so: as
soon as
I detect (via my own, homegrown fail2ban-like log monitoring utility)
what
I deem to be attempts to log in via imap or pop3 with a dictionary
password
attack, I immediately do a DROP via iptables. Yes, this will block all
future connection attemps from the same host, but unfortunately, it
doesn't
stop the following scenario, which regularly occurs on my server ...
* Hacker connects via imap or pop3 to my server.
* Hacker makes numerous login attempts one after the other with various
passwords, and without disconnecting in between attempts. I've seen 10
and
more of these repeated attempts rapidly during a single imap or pop3
connection.
Simply using iptables to DROP or REJECT the connection does not prevent
those repeated login attempts during the original imap or pop3 session.
Again, this only prevents *future* connections via that host.
It should block all subsequent packets received from that IP address,
immediately. An in-process connection would appear (to the client) to hang.
Either there is an ACCEPT rule for related traffic somewhere in the
chain before your new DROP rule, which is matching first and allowing
the existing connection's packets through, or your DROP rule is
malformed and not actually matching the traffic.
Re: Force TCP socket disconnect on imap login failure?
closing a socket can leave a process in an undertimed state pending how the code reacts blocking in the background via iptables would just stop traffic and the process should die cleanly. programming 101, network connections at best dont like the plug being pulled once they start to talk but if the connection just dies off then it is just a network timeout error with no real harm being done. just a thought. Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 5/23/2022 9:25 PM, John Tulp wrote: i googled a little, i was just curious about your question. found a stackoverflow question which, answered, says that using gdb one can close the fd, after using lsof to find it out. oh, and your iptables command... you have the address aaa. etc with a -d, i think you mean the source ip address of the connection, -s, right ? if you want, i can provide that link. On Mon, 2022-05-23 at 17:16 -0400, Hippo Man wrote: OOPS! I incorrectly copied and pasted the iptables command in my previous message. Here is the correct iptables command: iptables -I INPUT -p tcp -m multiport --destination-port 143,993 -d aaa.bbb.ccc.ddd -j DROP This command successfully blocks *future* connections to ports 143 and 993 from that IP address, but as I mentioned, it doesn't kill the currently open connection. -- hippo...@gmail.com Take a hippopotamus to lunch today. On Mon, May 23, 2022 at 4:54 PM Hippo Man wrote: Thank you, but fail2ban doesn't do what I need. Here is why ... I have used fail2ban and also my own homegrown log monitor program for this purpose. In both cases, I can detect the failed imap logins and then cause the following command to be run ... iptables -I INPUT -p tcp --destination-port aaa.bbb.ccc.ddd -j DROP However, this does not drop connections that are existing and already open. It will only drop *future* connections from that IP address to port 143. This is why I want to kill the existing connection. Even after that "iptables" command is issued, the entity which is connected to the imap port can continue to send more and more imap commands. If I can drop the TCP connection as soon as an imap login fails and also issue that kind of "iptables" command, then the client would have to reconnect in order to retry other login attempts. Those future connections would then be successfully blocked by that iptables rule. And even if I issue a "tcpdrop" command instead of just the "iptables" command, it doesn't kill the already-open connection. It just force-blocks future connections. I'm thinking of patching the dovecot source code to create a personal version which immediately disconnects from the socket after login failure. Of course, I would prefer not to do that, if there is another way to accomplish this. -- hippo...@gmail.com Take a hippopotamus to lunch today. On Mon, May 23, 2022 at 4:24 PM Jan Hugo Prins wrote: Look at fail2ban. Should be able to do that for you. Jan Hugo On 5/23/22 21:11, Lloyd Zusman wrote: > I'm running dovecot 2.2.13 under Debian 8. > I'd like to force an immediate TCP socket disconnect > after any imap login attempt that fails. > > Right now, if invalid credentials are supplied > during an imap login, the client can keep retrying > logins with different credentials. However, I want > to prevent that from occurring by causing the socket > connection to be closed as soon as there is any > failed login attempt. > > I haven't been able to find any dovecot > configuration setting which could control this > behavior, but I'm hoping that I just missed > something. > > Thank you very much for any suggestions. > > > -- > hippo...@gmail.com > Take a hippopotamus to lunch today. >
Re: Force TCP socket disconnect on imap login failure?
"-j REJECT --reject-with tcp-reset" instead of DROP are valid ideas consider that if you update (in the background) and block the connection then dovecot (or any other process) should just block the traffic and timeout to close the connection anyways fyi ?? Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 5/24/2022 12:18 AM, Péter Márton wrote: Just for clarification (this probably won't help achieve your primary goal to reset the connections): Iptables can block future connections _and_ stop existing connections to receive (and send) packets (even the command you posted). What it can't do is closing existing connections (sending a FIN). If the example you show can not block existing connections you have somewhere before the chain a RELATED, ESTABLISHED rule with ACCEPT as target. This is a common mistake. Your fail2ban rules have to come _before_ you check for related and established connections. I never tested this, but you could try using "-j REJECT --reject-with tcp-reset" instead of DROP. Then at least a RST would be sent. Hippo Man ezt írta (időpont: 2022. máj. 23., H, 23:17): OOPS! I incorrectly copied and pasted the iptables command in my previous message. Here is the correct iptables command: iptables -I INPUT -p tcp -m multiport --destination-port 143,993 -d aaa.bbb.ccc.ddd -j DROP This command successfully blocks *future* connections to ports 143 and 993 from that IP address, but as I mentioned, it doesn't kill the currently open connection. -- hippo...@gmail.com Take a hippopotamus to lunch today. On Mon, May 23, 2022 at 4:54 PM Hippo Man wrote: Thank you, but fail2ban doesn't do what I need. Here is why ... I have used fail2ban and also my own homegrown log monitor program for this purpose. In both cases, I can detect the failed imap logins and then cause the following command to be run ... iptables -I INPUT -p tcp --destination-port aaa.bbb.ccc.ddd -j DROP However, this does not drop connections that are existing and already open. It will only drop *future* connections from that IP address to port 143. This is why I want to kill the existing connection. Even after that "iptables" command is issued, the entity which is connected to the imap port can continue to send more and more imap commands. If I can drop the TCP connection as soon as an imap login fails and also issue that kind of "iptables" command, then the client would have to reconnect in order to retry other login attempts. Those future connections would then be successfully blocked by that iptables rule. And even if I issue a "tcpdrop" command instead of just the "iptables" command, it doesn't kill the already-open connection. It just force-blocks future connections. I'm thinking of patching the dovecot source code to create a personal version which immediately disconnects from the socket after login failure. Of course, I would prefer not to do that, if there is another way to accomplish this. -- hippo...@gmail.com Take a hippopotamus to lunch today. On Mon, May 23, 2022 at 4:24 PM Jan Hugo Prins wrote: Look at fail2ban. Should be able to do that for you. Jan Hugo On 5/23/22 21:11, Lloyd Zusman wrote: I'm running dovecot 2.2.13 under Debian 8. I'd like to force an immediate TCP socket disconnect after any imap login attempt that fails. Right now, if invalid credentials are supplied during an imap login, the client can keep retrying logins with different credentials. However, I want to prevent that from occurring by causing the socket connection to be closed as soon as there is any failed login attempt. I haven't been able to find any dovecot configuration setting which could control this behavior, but I'm hoping that I just missed something. Thank you very much for any suggestions. -- hippo...@gmail.com Take a hippopotamus to lunch today.
Re: doveadm mailbox status -u p...@scom.ca -t all (syntax)
yes did thanks doveadm [-f formatter] mailbox status [-A|-u user|-F file] [-S socket_path] [-t] fields mailbox ... My apologies for being so stupid, in the cyrus days (myhead is still stuck there) the mbox name was also the username I am still seperating that in my head. Noted for future Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 5/24/2022 4:51 AM, Aki Tuomi wrote: You might want to add `INBOX` to the command? Aki On 24/05/2022 00:34 Paul Kudla (SCOM.CA Internet Services Inc.) wrote: Ok I know this is a general question and at my level I should be able to figure this out all i want is the complete status of an inbox ? # doveadm mailbox status -u p...@scom.ca -t all keeps returning doveadm mailbox status [-u |-A] [-S ] [...] ie no specific error. and no status answer just looking for default info ? -- Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca
Re: Force TCP socket disconnect on imap login failure?
#print ip
#print 'appending to list'
currentlist.append(status)
currentlist.append(ip)
except :
print 'Bad Data Skipping ...'
print
print
print'Full list Currently In Firewall ...'
#print currentlist
#sys.exit()
print 'Got the list ... Working'
print
print
blacklist = [] #This is the converted list to iptable compatable formats
for x in range (0,len(firewalldata)) : #data = ipdata from db
#Internal Sample - ['A', '10.220.0.0/16']
#DB Sample - ('A', '67.55.27.171')
y = firewalldata[x]
#print 'firewall data %s' %str(y)
#print
#print
#sys.exit()
ipaddress = str(y[1])
#print 'DB Ip Address %s' %str(ipaddress)
if ipaddress <> 'ALL' :
done = 0
#print 'IP In : %s' %str(ipaddress)
#Modify ipaddress for cidr mapping
if ipaddress.count('.') == 1 : #10.
ipaddress = ipaddress + '0.0.0/8'
done = 1
if ipaddress.count('.') == 2 and done == 0 : #10.0.
ipaddress = ipaddress + '0.0/16'
done = 1
if ipaddress.count('.') == 3 and
ipaddress[len(ipaddress)-1] == '.' and done == 0 : #10.0.0.
ipaddress = ipaddress + '0/24'
#print 'IP Out: %s' %str(ipaddress)
#Now process the tables ie update/delete/change the entries
blacklist.append(str(y[0])) #set the status
blacklist.append(str(ipaddress) ) #Set the ip block to
manage
#print 'Current List In Scom Blacklistings'
#print badlist
print 'Processing My IP Black List Entries'
for n in range (0,len(blacklist),2) : #0 - action,1 - ip block
blacklistaction = str(blacklist[n])
blacklistip = str(blacklist[n+1])
#Now go check the iptable list to see if i have an entry
#print 'Processing Entry %s for IP %s with Action %s'
%(str(n),blacklistip,blacklistaction)
#print len(currentlist)
try :
nn = currentlist.index(blacklistip)
nn = nn-1
#Is this current black list ip currently in the iptables?
iptablesaction = str(currentlist[nn])
iptablesip = str( currentlist[nn+1] )
#Do i have a matching ip block?
if blacklistip == iptablesip : #We found a matching bl
entry already in iptables.
if blacklistaction == iptablesaction : #Rule is
good as is skip
#print 'Found A Current Rule that
matches, skipping ... %s' %str(blacklistip)
del currentlist[nn+1]
del currentlist[nn]
elif ipblacklistaction <> iptablesaction : #We
have a matching block but have to update the list
DELETEIP(str(iptablesip)) #Drop the
existing ip from the tables (precautionary)
if blacklistaction == 'A' :
#print 'Adding to Accept
IPTABLES List'
ACCEPTIP(str(ipblacklistip))
elif blacklistaction == 'D' :
#print 'Adding to Drop IPTABLES
List'
DROPIP(str(ipblacklistip))
print 'Updated Mismatch IPTABLES for %s
...' %str(ipblacklistip)
del currentlist[nn+1]
del currentlist[nn]
except :
#e = sys.exc_info()[0]
#print e
#We did not find anything in the tables, add new entry
print 'Pricessing Entry : %s ' %str(n)
if blacklistaction == 'A' :
print 'Adding to Accept IPTABLES List %s'
%str(blacklistip)
ACCEPTIP(blacklistip)
elif blacklistaction == 'D' :
print 'Adding to Drop IPTABLES List %s'
%str(blacklistip)
DROPIP(blacklistip)
#print 'Updated IPTABLES with new entry %s with Action
: %s' %(blacklistip,blacklistaction)
#Ok the blacklist is god again, see if there are any left over iptables
rules that we need to delete
print len(currentlist)
if len(currentlist) <> 0 :
print 'Cleaning up %s extra iptables ' %str(len(currentlist))
for nn in range (0,len(currentlist),2) :
iptablesip = str( currentlist[nn+1] )
print 'Deleting %s from iptables' %str(iptablesip)
DELETEIP(str(iptablesip))
sys.exit()
--
Happy Tuesday !!!
Thanks - paul
Pau
Re: doveadm mailbox status -u p...@scom.ca -t all (syntax)
thanks Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 5/23/2022 6:40 PM, Patrick Domack wrote: doveadm mailbox status -t all -u 'p...@scom.ca' '*'
Re: doveadm mailbox status -u p...@scom.ca -t all (syntax)
& # doveadm mailbox status -t all -u 'p...@scom.ca' 'p...@scom.ca' doveadm(p...@scom.ca): Error: Mailbox p...@scom.ca: Failed to lookup mailbox status: Character not allowed in mailbox name: '.' messages=0 recent=0 unseen=0 vsize=0 & # doveadm mailbox status -t all 'p...@scom.ca' doveadm(root): Error: Couldn't drop privileges: User is missing UID (see mail_uid setting) sorry just can not figure it out?? Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 5/23/2022 5:34 PM, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: Ok I know this is a general question and at my level I should be able to figure this out all i want is the complete status of an inbox ? # doveadm mailbox status -u p...@scom.ca -t all keeps returning doveadm mailbox status [-u |-A] [-S ] [...] ie no specific error. and no status answer just looking for default info ?
doveadm mailbox status -u p...@scom.ca -t all (syntax)
Ok I know this is a general question and at my level I should be able to figure this out all i want is the complete status of an inbox ? # doveadm mailbox status -u p...@scom.ca -t all keeps returning doveadm mailbox status [-u |-A] [-S ] [...] ie no specific error. and no status answer just looking for default info ? -- Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca
Re: Force TCP socket disconnect on imap login failure?
Ok would like to help and I DO understand the issue at hand. It took me with custom programming 5 years of tinkering to get to what i have today. So a few questions / comments ?? Are you running an sql database or just user-db (local) to do the lookups ? sql gives you some major flexability on how to track and ignore ip blocks etc etc etc upon logins. Issues to consider (I will explain the system i wrote for SCOM.CA as I went though a lot to get here and you like most are probably experiencing the same hack attempts.) Note I am not trying to get off topic but you need to think about stuff like this through a bit. You also have to consider people trying to send through postfix as it is the same issue in reverse. Under SCOM I do the following (only meant to be a guideline): 1. i have a common syslogger running that ALL logging goes through. 2. I then have conditions against anything being logged that will trigger an event. 3. The event triggered in your case would be seeing a line like auth: sql(t...@dereilanatureinn.ca,220.194.140.110, <5H72HLPfTp/cwoxu>): unknown user pop3-login: Disconnected: Connection closed (auth failed, 1 attempts in 3 secs): user=, method=PLAIN, rip=110.44.124.224, lip=65.39.148.18 there are other conditions but you get the idea. Ok from here it starts getting complicated. The idea is to keep the rift raft out and allow good users in. Easier said then done. I track all bad logins from all bad ip addresses and then run a seperate database table that tracks that. the ip address that gets tracked lands in two places, the firewall tables (which for me are global) & the user in my database. if a user is unknown (example above) then at least the first condition will catch a bad hack attempt based on ip. Most hack attempts by ip address usually keep sending common login names (like admin, ftp, ftpuser etc etc) hoping to match to a common account. for the user (which is relative here) i let 30 attempts go by every 30 minutes and 500 per month, after which the user is blocked via auth in sql and will have to call to get unlocked. Usually the ip address is blacklisted before the username is so its not that much of a deal. afterwhich it becomes part of the user query to lock out the userfrom anywhere as they are obviously getting hacked. when an ip is doing the hacking then i count using the same formula and then blacklist it internal to all of my servers (thus a database makes it earier to track) I run freebsd and thus use pf firewall, iptables can do the same with the same info you just need to build the tables and uodate them. (i update mine every 10 minutes, i find pf does this quicker on large lables (like 10,000 blocked) ) Now for the issue at hand that you are asking about: I am sure that the c programming could be patched along the line to do exactly what you are asking, However Issues that pop up. so you hang up on the connection, they will probably just login again anyways which means without tracking the ip & username stats and updating accordingly it will really not change anything at the end of the day. In my experience I see people / servers etc constantly hacking my side and what i generally described above turned out to be the only real fix, and not even really that guarenteed to work! I do get ip's that get blacklisted by accident (i do the whole class 'c' as the 'c' block is usually all the same guy) but i get a good one maybe every few months, usually when i block it there are not many complaints after that. I know the above is complicated, fyi i track postfix's sasl auth's as well but in that case to get a username & ip address on one syslog line i had to patch the sasl auth c file to get a log entry that was useable. Postfix simply will NOT provide the info on one line. Between both of the conditions above the server's remain fairly useable and secured. the CSF firewall option below IS valid but i find you need to track IP address & username or you end up blocking stuff you dont want to also on another note IPV6 (at least in canada) is becoming a pain for isp's many cable companies, dsl providers etc are assigning an ipv6 address and then converting it to ipv4 on the way out the door from their networks using double natting ? Issue is you can have 10,000 people all sharing that same ip address and if you block it then that will prevent other 'good' people from logging in, again back to tracking the username in this case gives you an out if the hacker is just using a list obtained elsewhere on the net. Food for thought. Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 5/23/2022 3:26 PM, dovecot-boun...@dovecot.org wrote: On 2022-05-
Many Thanks to the Programmers
I am still in testing mode with 2.3.19 but the replication with the folders over 300+ seems to be fixed everything synced ok on its own within 30 minutes Again MUSH APPRECIATED ! Moving to 2.3.19 was woth it! -- Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca
Re: mail_replica in userdb?
So are you ok now I'm a postgresql kinda guy never got around to ldap as that was mostly windows based! fyi Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email p...@scom.ca On 5/23/2022 11:18 AM, Coy Hile wrote: On May 23, 2022, at 11:00 AM, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: ok there is more to replica setup user-db error seems to indicate the sql iterate it not setup or not returning stuff properly can you specify what you are using for a sql setup (if any) can help from there? Ah, on the new side, I am using LDAP, rather than SQL, setup thusly in dovecot-ldap.conf.ext (once I removed all the comments and blank lines): '/^$/d' hosts = ldap.coyhile.com dn = uid=dovecotquery,ou=people,dc=coyhile,dc=com dnpass = [redacted] base = ou=People,dc=coyhile,dc=com scope = subtree user_filter = (&(objectClass=posixAccount)(uid=%n)) pass_attrs = uid=%n,userPassword=password pass_filter = (&(objectClass=posixAccount)(uid=%n)) iterate_attrs = maildrop=user iterate_filter = (objectClass=posixAccount) -- Coy Hile coy.h...@coyhile.com
Re: mail_replica in userdb?
my apologies i read through the other posts after responding
you are using ldap ?
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 5/23/2022 11:00 AM, Paul Kudla (SCOM.CA Internet Services Inc.) wrote:
ok there is more to replica setup
user-db error seems to indicate the sql iterate it not setup or not
returning stuff properly
can you specify what you are using for a sql setup (if any)
can help from there?
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 5/23/2022 3:20 AM, Aki Tuomi wrote:
On 22/05/2022 02:32 Coy Hile wrote:
I seem to have got replication working from an older instance to a
newer one (insofar as the data are present on the new box) that I’m
looking to stand up to replace a newer box. In this case, nothing is
configured to talk to the new machine yet, so the replication is
unidirectional (old -> new).
However, I’m seeing an error like like this repeatedly in syslog:
2022-05-21T23:23:15.295252+00:00 basement-imap01 dovecot:
doveadm(h...@coyhile.com): Error: sync: User has no mail_replica in
userdb
What setting did I miss?
plugin {
mail_replica =
}
https://doc.dovecot.org/settings/plugin/replication-plugin/
Aki
Re: mail_replica in userdb?
ok there is more to replica setup
user-db error seems to indicate the sql iterate it not setup or not
returning stuff properly
can you specify what you are using for a sql setup (if any)
can help from there?
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
Email p...@scom.ca
On 5/23/2022 3:20 AM, Aki Tuomi wrote:
On 22/05/2022 02:32 Coy Hile wrote:
I seem to have got replication working from an older instance to a newer one (insofar as the data are present on the new box) that I’m looking to stand up to replace a newer box. In this case, nothing is configured to talk to the new machine yet, so the replication is unidirectional (old -> new).
However, I’m seeing an error like like this repeatedly in syslog:
2022-05-21T23:23:15.295252+00:00 basement-imap01 dovecot:
doveadm(h...@coyhile.com): Error: sync: User has no mail_replica in userdb
What setting did I miss?
plugin {
mail_replica =
}
https://doc.dovecot.org/settings/plugin/replication-plugin/
Aki
Re: Duplicate messages if message is moved when using dsync
ok - question
when you say dsync are you running that manually or as a cronjob ??
or using replication via dsync
there is a difference
replication is designed to handle mailbox changes by quing the changes
on the fly and then update in the background.
replication tracks all changes in a mailbox and then sends them one at a
time to the target server.
i run replication with tcp (not tcps or dsync)
I find (other then the 300+folder issues) that it runs pretty good in
real time.
if you are running dsync manually then yes timing on its own will be a
hit and miss and it will never sync properly if something appears during
the dsync process.
if you have the sive running as discussed that is the first step
you should setup replication so things are handled in real time.
pls advise if you have already done replication or just manually dsyncing.
Below is basic replication setup but you need to make sure various
pre-requirements are in place a head of time.
1. you need / preferably use a database for your user auth
you need to setup an iterate sql statement that returns the user
list of users to be replicated
Example (i use pgsql) :
# cat dovecot-pgsql.conf
driver = pgsql
connect = host=localhost port=5433 dbname=scom_billing user=pgsql password=
default_pass_scheme = PLAIN
password_query = SELECT username as user, password FROM email_users
WHERE username = '%u' and password <> 'alias' and status = True and
destination = '%u'
user_query = SELECT home, uid, gid FROM email_users WHERE username =
'%u' and password <> 'alias' and status = True and destination = '%u'
iterate_query = SELECT "username" as user, domain FROM email_users WHERE
status = True and alias_flag = False
#iterate_query above is what returns the user lists
2. run the command doveadm user '*' - it needs to return a list of users
that will trigger replication (after doing config above)
3. in general here is my relative parts for tcp replication
In dovecot.conf
---
mail_plugins = " virtual notify replication fts fts_lucene "
replication_dsync_parameters = -d -N -l 300 -U
dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u
plugin {
.
mail_replica = tcp:10.221.0.19:12345
#(note this is the ip address of the other server)
#it is the only config change ie server one above
#server2 would be the other server for communicating.
.
}
protocol imap {
mail_max_userip_connections = 50
mail_plugins = $mail_plugins notify replication
}
protocol pop3 {
mail_max_userip_connections = 50
mail_plugins = $mail_plugins notify replication
}
protocol imaps {
mail_max_userip_connections = 25
mail_plugins = $mail_plugins notify replication
}
protocol pop3s {
mail_max_userip_connections = 25
mail_plugins = $mail_plugins notify replication
}
replication_max_conns = 100 (optional)
replication_full_sync_interval = 1d
service replicator {
client_limit = 0
drop_priv_before_exec = no
idle_kill = 4294967295s
process_limit = 1
process_min_avail = 0
service_count = 0
vsz_limit = 8g
unix_listener replicator-doveadm {
mode = 0600
user = vmail
}
vsz_limit = 8192M
}
service aggregator {
process_limit = 1000
#vsz_limit = 1g
fifo_listener replication-notify-fifo {
user = vmail
group = vmail
mode = 0666
}
}
that should do it?
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 5/18/2022 3:39 PM, Thom Pol wrote:
Never mind. It did not work, issue still exists. I guess my network was
a little slower when I thought it worked, so the emails were moved by my
local client only after the sync had already happened between the servers.
So, back to the original message. Anyone any idea how this moving on
local client while the remote servers are still running the dsync can
work correctly without getting a duplicate?
Kind regards,
Thom Pol
On 18/05/2022 16:24, Paul Kudla (SCOM.CA Internet Services Inc.) wrote:
Your Welcome
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services<http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main?1.866.411.7266
Fax?1.888.892.7266
--
This message has been scanned for viruses and
dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
believed to be clean.
Upgrading from 2.3.18 --> 2.3.19
Ok currently running 2.3.18 Would like to upgrade to 2.3.19 (to keep current) Since these are production servers can i down grade back to 2.3.18 if nessesary? Issues at hand is there seems to be conflicting messages about the replication being fixed or not ? so would like to basically try 2.3.19 to see if it fixes some of my issues (large folder counts etc failing to replicate properly) If it does or stays the same then i am ok with that for now. if 2.3.19 causes issues can i go back to 2.3.18 as i already have scripts etc sorting out the sync issues? pls advise -- Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266
Re: dovecot Digest, Vol 229, Issue 23
Your Welcome
Happy Wednesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 5/18/2022 9:04 AM, Thom Pol wrote:
I must say I was a bit sceptical if this sieve script would resolve it,
but it seems like it totally did! Thanks a lot for your reply, hope this
helps someone else in the future as well.
Kind regards,
Thom Pol
On 16/05/2022 08:12, Paul Kudla (SCOM.CA Internet Services Inc.) wrote:
ok duplicsate emails (even across dsync, replication etc) is typically
handled via a global sieve script
I use :
# cat duplicates.sieve
require "duplicate"; # for dovecot >= 2.2.18
if duplicate {
discard;
stop;
}
for the scripts
and setup sieve to work via my dovecot.conf file
relative parts below :
protocols = imap pop3 lmtp sieve
protocol lmtp {
mail_plugins = $mail_plugins sieve
postmaster_address =moni...@scom.ca
}
protocol lda {
mail_plugins = $mail_plugins sieve
}
plugin {
.
sieve =file:~/sieve;active=~/sieve/.dovecot.sieve
sieve_duplicate_default_period = 1h
sieve_duplicate_max_period = 1d
sieve_extensions = +duplicate +notify +imapflags +vacation-seconds
sieve_global_dir = /usr/local/etc/dovecot/sieve
sieve_before = /usr/local/etc/dovecot/sieve/duplicates.sieve
.
}
service managesieve-login {
process_limit = 1000
vsz_limit = 1g
inet_listener sieve {
port = 4190
}
}
protocol sieve {
managesieve_implementation_string = Dovecot Pigeonhole
managesieve_max_line_length = 65536
}
--
note the sieve_before which handles duplictes during delivery etc.
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services<http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main?1.866.411.7266
Fax?1.888.892.7266
--
This message has been scanned for viruses and
dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
believed to be clean.
Re: Can sync/migrate all mail from remote imap account except the main "INBOX"
Assuming that the question at hand is :
My best guess at this point is the logs showing "INBOX" and
"INBOX.INBOX" , perhaps there's something about the naming scheme that
is throwing it off, but the "INBOX.INBOX.Drafts" and such are still
being handled.
Thanks again for any direction!
Darren
I went through this when migrating from syrus imap
typically when dealing with special folders, cyrus liked doing everything
INBOX.Drafts
INBOX.Set
etc
issue is it seems dovecot (only by experience) would prefer special
folders in the ROOT of the mail folder.
to acomplish this I used this in the dovecot.conf file
-
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
}
-
However dovecot will pickup on the other folders which lead to updating
the file subscriptions located in the mailbox folder of the user
What i had to do was add the namespaces
restart the dovecot
which will (or should) create extra special folders in the default locations
from there i had to move the emails manually from the old
(INBOX.INBOX.Sent - for example) to the new folders
next issue is the mail client would not allow a delete of the old folder
because it is though to be special.
in the subscriptions file (this directs what is avaliable to the client)
i had to manually remove the entry and also delete the (now empty) dir
for the old INBOX.INBOX.Sent (again for example)
you will probably have to do this for
Sent
Trash
Drafts
maybe junk??
I also noted (thunderbird for exanmple) that if you are running
replication this has to be done manually on all servers.
After all is said and done trying deleting a message (should goto trash)
however note (again thunderbird) you may have to set the Trash & Sent
folders to the new ones (why testing with deleting a message is
nessesary to make sure stuff works correctly)
Same for the Sent
Hope this helps.
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 5/13/2022 3:17 PM, Darren Mobley wrote:
Sure, thanks for the reply and suggestion [Smile]
To make testing/debugging easier, rather than using a remote provider, I
created another domain and email account on the same server with just a
few mails. I sent 3 jibberish mails to this new account, from this new
account, so they are showing in both INBOX and Sent, as well as 1 mail
from root on the CLI (mail -v w...@hellodemo.ppl
<mailto:w...@hellodemo.ppl>), These is also a draft mail saved in Drafts:
Source account file system layout:
# find /home/hellodemo/mail/hellodemo.ppl/wah/
/home/hellodemo/mail/hellodemo.ppl/wah/
/home/hellodemo/mail/hellodemo.ppl/wah/.Drafts
/home/hellodemo/mail/hellodemo.ppl/wah/.Drafts/cur
/home/hellodemo/mail/hellodemo.ppl/wah/.Drafts/cur/1652465670.M627726P3692.cent-7.darren.cpanel.net,S=341,W=353:2,S
/home/hellodemo/mail/hellodemo.ppl/wah/.Drafts/new
/home/hellodemo/mail/hellodemo.ppl/wah/.Drafts/tmp
/home/hellodemo/mail/hellodemo.ppl/wah/.Drafts/dovecot.index.log
/home/hellodemo/mail/hellodemo.ppl/wah/.Drafts/dovecot-uidlist
/home/hellodemo/mail/hellodemo.ppl/wah/.Drafts/dovecot.index.cache
/home/hellodemo/mail/hellodemo.ppl/wah/.Junk
/home/hellodemo/mail/hellodemo.ppl/wah/.Junk/cur
/home/hellodemo/mail/hellodemo.ppl/wah/.Junk/new
/home/hellodemo/mail/hellodemo.ppl/wah/.Junk/tmp
/home/hellodemo/mail/hellodemo.ppl/wah/.Junk/dovecot.index.log
/home/hellodemo/mail/hellodemo.ppl/wah/.Junk/dovecot-uidlist
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/cur
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/cur/1652465920.M799279P3808.cent-7.darren.cpanel.net,S=359,W=371:2,S
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/cur/1652465942.M469746P3832.cent-7.darren.cpanel.net,S=342,W=354:2,S
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/cur/1652465951.M751062P3906.cent-7.darren.cpanel.net,S=342,W=354:2,S
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/new
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/tmp
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/dovecot.index.log
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/dovecot-uidlist
/home/hellodemo/mail/hellodemo.ppl/wah/.Sent/dovecot.index.cache
/home/hellodemo/mail/hellodemo.ppl/wah/.Trash
/home/hellodemo/mail/hellodemo.ppl/wah/.Trash/cur
/home/hellodemo/mail/hellodemo.ppl/wah/.Trash/new
/home/hellodemo/mail/hellodemo.ppl/wah/.Trash/tmp
/home/hellodemo/mail/hellodemo.ppl/wah/.Trash/dovecot.index.log
/home/hellodemo/mail/hellodemo.ppl/w
Re: TLS renegotiation issue (CVE-2011-1473) in Dovecot
ok need some more info but in general ssl setup should be as follows. FQHN - do have have proper dns reverses setup? - this is an upstream thing for example : forwards : ## nslookup mail18.scom.ca Server: 10.220.0.2 Address:10.220.0.2#53 Name: mail18.scom.ca Address: 65.39.148.18 reverses : ## nslookup 65.39.148.18 18.148.39.65.in-addr.arpa name = sogo.scom.ca. 18.148.39.65.in-addr.arpa name = mail18.scom.ca. 18.148.39.65.in-addr.arpa name = ns2.scom.ca. 18.148.39.65.in-addr.arpa name = mail.scom.ca. Authoritative answers can be found from: it needs to be understood that the reverses are usually returned by your upstream isp and should be set accordingly, ie you will have to get them to program them. if you note above you can have several mappings for reverses next ssl rewriting (other then sni) does simply not work so well. also you should have a static ip (assuming you do) mail18 is in my reverse so this error wont be thrown. also note the server name (mail18.scom.ca) for both dovecot and postfix MUST match the certificate and dns for all to work. ssl when running a masil server should be setup with a proper ceretificate (i use a wildcard for mine), proper forwards and proper reverses. Lets Encrypt (free ssl) is not a stable way to go on a busy server. You can typically get an ssl cert (proper one) for 10~20 us? pending on the provider of the cert. also note this has to be setup properly on postfix as well as that to could throw a FQHN error if they are connecting to port 25/465/587 as well. My ssl config (example) - please note i run sni for multiple domains and certs i typically run with the dovecot defaults under 2.3.18 and it seems to work ok. # cat sni.conf #sni.conf ssl = yes verbose_ssl = yes ssl_dh = ssl_key = /programs/common/getssl.cert -c mail.hamletdevelopments.ca -q yes ssl_cert = /programs/common/getssl.cert -c mail.hamletdevelopments.ca -q yes ssl_ca = /programs/common/getssl.cert -c mail.hamletdevelopments.ca -q yes } Happy Monday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 5/13/2022 10:38 PM, Elisamuel Resto wrote: On 2022-05-13 5:02 pm, Greg Earle wrote: Hello, At work I'm running a Dovecot 2.3.15 server on a RHEL 7.9 system with OpenSSL 1.0.2k. Our IT Security people are threatening to shut it down because of this: We were notified of a possible TLS renegotiation vulnerability on [FQHN]. [Parent organization] ticket NNN is open to track efforts. We conducted a manual test on the site for TLS Renegotiation on IMAP port 993. We found that this was set to enabled. In order to remediate we will need to either: 1. Disable Renegotiation (preferred) 2. Set a max aggregated renegotiation Please remediate as soon as possible. References: https://support.f5.com/csp/article/K15278 https://nvd.nist.gov/vuln/detail/cve-2011-1473 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-1473 I did some Googling and among the results, I found a few old posts from this mailing list among them, which to summarize basically seemed to say "Yeah, we could write some code ... " but that was about it. The IT Security rep sent me a reference to an ancient Red Hat article https://access.redhat.com/articles/23543 which is hysterical - ancient history, references NSS and Tomcat, suggests changes to an add-on product (Red Hat Certificate Server) that is EOL, etc. Is there any way to mitigate this issue? (The only thing I can think of is to upgrade the Dovecot server to RHEL 8 and restrict connections to only TLSv1.3, but that ain't gonna happen overnight.) Thanks, - Greg Greg, I believe this to be a configuration error, not a dovecot problem. The output of dovecot -n (as an attachment; look it over for any data you do not want publicized) would help to suggest changes to bring you back into compliance. Regards, Elisamuel Resto
Re: Fatal Error after upgrade to 2:2.3.19-2+debian11
ok the rights can be a bit confusing at times
assuming you are running virtual users (or not)
try these one at a time, i found that when dovecot starts it will adjust
the permissions on the control files accordingly to what is set in the
examples below, also note postfix can be a variable in this but would
probably not be
I had to fiddle with stuff a lot
also dovecot i start in my rc.local (root startup)
the root user starts dovecot, it then changes everything rights wise as
stated below and then changes to user dovecot (vmail whatever) to
auctually start processing emails etc.
Again this is a pretty loose explanation but will point you in a
direction for troubleshooting.
I typically use in dovecot.conf
-
service aggregator {
process_limit = 1000
#vsz_limit = 1g
fifo_listener replication-notify-fifo {
user = vmail
group = vmail
mode = 0666
}
}
service lmtp {
process_limit=1000
vsz_limit = 512m
client_limit=1
unix_listener /usr/home/postfix.local/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service doveadm {
process_limit = 0
process_min_avail = 0
idle_kill = 0
client_limit = 1
user = vmail
inet_listener {
port = 12345
}
}
service config {
unix_listener config {
user = vmail
}
}
service anvil {
process_limit = 1
client_limit=5000
vsz_limit = 512m
unix_listener anvil {
group = vmail
mode = 0666
}
}
service auth {
process_limit = 1
client_limit=5000
vsz_limit = 1g
unix_listener auth-userdb {
mode = 0660
user = vmail
group = vmail
}
unix_listener /var/spool/postfix/private/auth {
mode = 0666
}
}
service stats {
process_limit = 1000
vsz_limit = 1g
unix_listener stats-reader {
group = vmail
mode = 0666
}
unix_listener stats-writer {
group = vmail
mode = 0666
}
}
-
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 5/16/2022 8:09 AM, Sebastian Kroczek wrote:
Hello all,
I updated the server tonight and with it Dovecot from 2.2.27 to
2:2.3.19-2+debian11. However, there seems to be a problem with the ACLs,
because since then fatal errors are logged (see core dump). I suspect
that some outdated configuration is causing this behavior, but so far I
couldn't figure out which one it could be. I also have no clue right now
how to debug this further.
Thank you very much for your help. If more information are needed, I
will of course be happy to provide them.
VG
Sebastian
Error logs =
May 16 13:33:43 Fatal: imap(us...@domain.com)<0r5YZR/fM4AfrHBI>: master:
service(imap): child 238359 killed with signal 11 (core dumped)
May 16 13:33:46 Fatal: imap(us...@domain.com): master:
service(imap): child 238386 killed with signal 11 (core dumped)
May 16 13:33:46 Fatal: imap(us...@domain.com): master:
service(imap): child 238387 killed with signal 11 (core dumped)
May 16 13:34:54 Fatal: imap(us...@domain.com)<1WS6aR/fHoAfrHBI>: master:
service(imap): child 238509 killed with signal 11 (core dumped)
May 16 13:34:54 Fatal: imap(us...@domain.com): master:
service(imap): child 238508 killed with signal 11 (core dumped)
May 16 13:35:27 Fatal: imap(us...@domain.com): master:
service(imap): child 238589 killed with signal 11 (core dumped)
May 16 13:35:27 Fatal: imap(us...@domain.com): master:
service(imap): child 238590 killed with signal 11 (core dumped)
END Error logs =
dovecot.conf ==
# 2.3.19 (b3ad6004dc): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.19 (4eae2f79)
# OS: Linux 5.10.0-14-amd64 x86_64 Debian 11.3
# Hostname: wv-imap1.wavecloud.de
auth_mechanisms = plain login
default_vsz_limit = 4 G
dict {
acl = mysql:/etc/dovecot/dovecot-dict-sql.conf
}
first_valid_gid = 5000
first_valid_uid = 5000
imap_capability = +XDOVECOT
last_valid_gid = 5000
last_valid_uid = 5000
listen = 10.10.115.XX
login_trusted_networks = 10.10.115.XX 10.10.115.XX
mail_location = maildir:~/
mail_log_prefix = "%s(%u)<%{session}>: "
mail_plugins = acl notify quota fts fts_solr virtual
maildir_stat_dirs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave
namespace {
inbox = yes
location =
mailbox Archive {
auto = subscribe
special_use = \Archive
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
auto = subscribe
s
Re: Duplicate messages if message is moved when using dsync
ok duplicsate emails (even across dsync, replication etc) is typically
handled via a global sieve script
I use :
# cat duplicates.sieve
require "duplicate"; # for dovecot >= 2.2.18
if duplicate {
discard;
stop;
}
for the scripts
and setup sieve to work via my dovecot.conf file
relative parts below :
protocols = imap pop3 lmtp sieve
protocol lmtp {
mail_plugins = $mail_plugins sieve
postmaster_address = moni...@scom.ca
}
protocol lda {
mail_plugins = $mail_plugins sieve
}
plugin {
.
sieve = file:~/sieve;active=~/sieve/.dovecot.sieve
sieve_duplicate_default_period = 1h
sieve_duplicate_max_period = 1d
sieve_extensions = +duplicate +notify +imapflags +vacation-seconds
sieve_global_dir = /usr/local/etc/dovecot/sieve
sieve_before = /usr/local/etc/dovecot/sieve/duplicates.sieve
.
}
service managesieve-login {
process_limit = 1000
vsz_limit = 1g
inet_listener sieve {
port = 4190
}
}
protocol sieve {
managesieve_implementation_string = Dovecot Pigeonhole
managesieve_max_line_length = 65536
}
--
note the sieve_before which handles duplictes during delivery etc.
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 5/15/2022 12:38 PM, Thom Pol wrote:
Hi,
Hope you are well.
We have a cluster of 2 Dovecot servers, both on v2.3.13 (89f716dc2),
using dsync to sync the messages between them.
Previously, we used TCPS to sync the messages, but after some testing,
we concluded that syncing over SSH resulted in a lot less failed syncs,
so we started using SSH.
The change has been a success, but I now notice a issue when a email
client immediately moves a messages to a separate folder while Dovecot
is syncing, where the message is seen twice in the folder (with the
exact same headers/content).
For example, I have set a filter in my email client, Thunderbird, to
immediately move all emails coming from this list to a separate folder.
When opening that folder, I do not see one, but two unread messages,
both identical to each other.
When checking the directories on the server, I see this:
mx1:
/var/vmail/example.com/joe/Maildir/.Subdir.Subdir/cur/1652615808.M190190P990486.mx2,S=19089,W=19384:2,S
/var/vmail/example.com/joe/Maildir/.Subdir.Subdir/cur/1652615811.M180050P1376677.mx1,S=19089,W=19384:2,S
mx2:
/var/vmail/example.com/joe/Maildir/.Subdir.Subdir/cur/1652615811.M981426P990530.mx2,S=19089,W=19384:2,S
/var/vmail/example.com/joe/Maildir/.Subdir.Subdir/cur/1652615808.M190190P990486.mx2,S=19089,W=19384:2,S
Note the difference: on mx1, one indicates mx1, and one mx2, while on
the other server, both indicate mx2/
Any idea (other then telling end-users not to use such filters) how we
could prevent these duplicate messages?
This is our config:
# Pigeonhole version 0.5.13 (cdd19fe3)
# OS: Linux 5.10.0-13-cloud-amd64 x86_64 Debian 11.3
# Hostname: mx1.example.com
auth_mechanisms = plain login
disable_plaintext_auth = no
dsync_remote_cmd = ssh -p 222 -l%{login} %{host} doveadm dsync-server -u%u
imap_capability = +SPECIAL-USE XLIST
listen = *,[::]
lmtp_rcpt_check_quota = yes
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_max_userip_connections = 100
mail_plugins = quota
mail_privileged_group = vmail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
separator = .
}
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
plugin {
mail_replica = remote:r...@mx2.example.com
quota = dict:user::file:/var/vmail/%d/%n/.quotausage
quota_status_nouser = DUNNO
quota_status_overquota = 552 5.2.2 Mailbox is full
quota_status_success = DUNNO
sieve = /var/vmail/%d/%n/.sieve
sieve_after = /var/vmail/%d/%n/.ispconfig.sieve
sieve_before = /var/vmail/%d/%n/.ispconfig-before.sieve
sieve_max_actions = 100
sieve_max_redirects = 25
sieve_max_script_size = 2M
}
protocols = imap pop3 lmtp
replication_max_conns = 50
service aggregator {
fifo_listener replication-notify-fifo {
mode = 0666
user = vmail
}
unix_listener replication-notify {
mode = 0666
user = vmail
}
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
user = root
}
service imap-login {
client_limit = 1000
process_limit = 512
}
service lmt
Re: Use different log files
}[51288] (997017684)
imap-login: Login: user=, method=PLAIN,
rip=172.97.231.82,
lip=65.39.148.18,
mpid=68496
mail18 05-16 07:41:57 {dovecot}[51288] (997017685)
imap-login: Login: user=, method=PLAIN,
rip=172.97.231.82,
lip=65.39.148.18,
mpid=68497
peer1 05-16 07:49:25 {su} [36623] (997022563) HISTORY:
PID=36623 UID=0 log -t p...@hiscomputer.ca
Displayed 350 Records
----
for example.
Happy Monday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 5/16/2022 5:58 AM, Cristiano Deana wrote:
Hi,
I have a mailserver with dovecot logging to syslog (by default, to
/var/log/maillog) and my MTA (postfix) is doing the same.
I use dovecot's services imap/pop3, auth and lmtp and now logs files are
hard to read because I havve all together MTA and these services.
Is it possibile to have different log with different services?
Example:
auth logging: /var/log/mail.auth
delivery: /var/log/mail.delivery and so on
Thank you
Re: how is the "Total number of known users" reported by doveadm replicator calculted?
ok assuming that you are using a database for virtual users (this is pgsql) # cat dovecot-pgsql.conf driver = pgsql connect = host=localhost port=5433 dbname=scom_billing user=pgsql password=x default_pass_scheme = PLAIN password_query = SELECT username as user, password FROM email_users WHERE username = '%u' and password <> 'alias' and status = True and destination = '%u' user_query = SELECT home, uid, gid FROM email_users WHERE username = '%u' and password <> 'alias' and status = True and destination = '%u' iterate_query = SELECT "username" as user, domain FROM email_users WHERE status = True and alias_flag = False iterate_query controls what is returned to doveadm user '*' and the replication service. in the above example i only return active accounts and skip alias accounts (fyi) adjust to your config. if it is not setup propoerly then both functions will return bad info to the replicator if the replicator picks up on a new user on its own it will add it to its own replicator db (see example below) - this is why you are getting different result count wise. once the .db file is updated it will carry that user till re-created (see below) to sync this you need to shut down all servers. origionaly before i figured this out i had to delete the .db file on all servers, touch it (aka make a blank file) and then restart the servers dovecot will set the rights automatically when you start it up again best sugestion is the get the doveadm user '*' working first as it will be the base results. see : [17:05:03] mail18.scom.ca [root:0] /usr/local/var/lib/dovecot # ll total 86 drwxr-xr-x 2 root wheel uarch4B May 13 10:52 . drwxr-xr-x 4 root wheel uarch4B Mar 8 2021 .. -rw-r--r-- 1 root wheel uarch 73B May 13 10:52 instances -rw-r--r-- 1 root wheel uarch 161K May 13 16:50 replicator.db replicator.db is a txt file that carries the current status of the replication service. example entry (carries hash info etc that tells the replicator service what happens next - ie when to pull a sync) : ditchb...@scom.ca 0 1651914641 1652433042 1652433042 0 AQAAALiUhhMjOhJiUHwAAM9Y3P8lOhJiAAEAJ8pBLXM8EmKfkQAAz1jc/yc6EmIEBgQAAABI1pY73z0SYp+RAADPWNz/KjoSYgABAJLdYwEjOhJiFCEBANHccmUlOhJiAAEAoEK+EyM6EmJQfAAAz1jc/yY6EmIAAQA9flMZFfIUYr7FAADR3HJlLDoSYgABAEZJQhMjOhJiUHwAAM9Y3P8jOhJiAAEAkErEKRnyFGK+xQAA0dxyZS06EmIAAQCYRnsA4D0SYp+RAADPWNz/KzoSYgABAHiN8xV7PBJin5EAAM9Y3P8oOhJiTQAAAFBNSIGABsI9EmKfkQAAz1jc/yk6EmIWGBYAAABX2T8x 1652433042 this db file generates the user.sync status etc doveadm replicator status '*' Happy Friday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 5/13/2022 4:26 PM, Arnaud Abélard wrote: Hello, I have a question regarding the "Total number of known users" displayed by doveadm replicator status. How is it calculated? Shouldn't it match the number of users reported by doveadm user '*' ? I have 3 servers being replicated, the "total number of known users" a lot higher than the number of users reports by doveadm user '*' why is that? Arnaud
Re: no full syncs after upgrading to dovecot 2.3.18
Ok update from my end
under 2.3.18 (have not upgraded production to 2.3.19 yet)
replication issues as stated before
however i need to note that i had to manually sync a user that was not
being listed as a replicator fail
this means i have to force a full sync between servers on all accounts
regardless of replication status
this was discovered this morning on a customers account that did not
replicate between the servers properly and thus emails were being
delivered days later because the client was accessing the other server.
its one thing to be 10 minutes late etc but a day late is not practical
again not complaining
I will load 2.3.19 on the test servers and try that and advise, also
will test for the folder count replication issue as well and advise
please note NO errors are being thrown in the debug log, it reports the
replication request, gets qued but does not complete??
Happy Thursday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 5/11/2022 12:25 AM, Cassidy B. Larson wrote:
Hi Aki,
We just installed 2.3.19, and are seeing a couple of users throwing the
"INBOX/dovecot.index reset, view is now inconsistent" and their
replicator status erroring out. Tried force-resync on the full mailbox,
but to no avail just yet. Not sure if this bug was supposedly fixed in
2.3.19?
Thanks,
Cassidy
On Thu, Apr 28, 2022 at 5:02 AM Aki Tuomi <mailto:aki.tu...@open-xchange.com>> wrote:
2.3.19 is round the corner, so not long. I cannot yet promise an
exact date but hopefully within week or two.
Aki
> On 28/04/2022 13:57 Paul Kudla (SCOM.CA <http://SCOM.CA> Internet
Services Inc.) mailto:p...@scom.ca>> wrote:
>
>
> Thanks for the update.
>
> is this for both replication issues (folders +300 etc)
>
> Just Asking - Any ETA
>
>
>
>
>
> Happy Thursday !!!
> Thanks - paul
>
> Paul Kudla
>
>
> Scom.ca Internet Services <http://www.scom.ca <http://www.scom.ca>>
> 004-1009 Byron Street South
> Whitby, Ontario - Canada
> L1N 4S3
>
> Toronto 416.642.7266
> Main 1.866.411.7266
> Fax 1.888.892.7266
>
> On 4/27/2022 9:01 AM, Aki Tuomi wrote:
> >
> > Hi!
> >
> > This is probably going to get fixed in 2.3.19, this looks like
an issue we are already fixing.
> >
> > Aki
> >
> >> On 26/04/2022 16:38 Paul Kudla (SCOM.CA <http://SCOM.CA>
Internet Services Inc.) mailto:p...@scom.ca>> wrote:
> >>
> >>
> >> Agreed there seems to be no way of posting these kinds of
issues to see
> >> if they are even being addressed or even known about moving
forward on
> >> new updates
> >>
> >> i read somewhere there is a new branch soming out but nothing
as of yet?
> >>
> >> 2.4 maybe
> >> 5.0
> >>
> >> my previous replication issues (back in feb) went unanswered.
> >>
> >> not faulting anyone, but the developers do seem to be
disconnected from
> >> issues as of late? or concentrating on other issues.
> >>
> >> I have no problem with support contracts for day to day maintence
> >> however as a programmer myself they usually dont work as the
other end
> >> relies on the latest source code anyways. Thus can not help.
> >>
> >> I am trying to take a part the replicator c programming based
on 2.3.18
> >> as most of it does work to some extent.
> >>
> >> tcps just does not work (ie 600 seconds default in the c
programming)
> >>
> >> My thoughts are tcp works ok but fails when the replicator through
> >> dsync-client.c when asked to return the folder list?
> >>
> >>
> >> replicator-brain.c seems to control the overall process and
timing.
> >>
> >> replicator-queue.c seems to handle the que file that does seem
to carry
> >> acurate info.
> >>
> >>
> >> things in the source code are documented enough to figure this
out but i
> >> am still going through all the related .h files documentation
wise which
> >> are all over the place.
> >>
> >> there is no cl
Re: IMAP connection lost while sending mail
vmail vmail uarch 131K May 6 09:08 dovecot-uidlist
-rw--- 1 vmail vmail uarch8B May 2 10:25 dovecot-uidvalidity
-r--r--r-- 1 vmail vmail uarch0B Feb 19 17:58
dovecot-uidvalidity.62117625
-rw--- 1 vmail vmail uarch 45K May 6 08:41 dovecot.index
-rw--- 1 vmail vmail uarch 1.8M May 6 10:16 dovecot.index.cache
-rw--- 1 vmail vmail uarch 26K May 6 09:09 dovecot.index.log
-rw--- 1 vmail vmail uarch 32K May 4 10:56 dovecot.index.log.2
-rw--- 1 vmail vmail uarch 2.7K May 6 06:21 dovecot.list.index
-rw--- 1 vmail vmail uarch 5.0K May 6 09:08 dovecot.list.index.log
-rw--- 1 vmail vmail uarch 2.2K May 2 10:25 dovecot.mailbox.log
-rw--- 1 vmail vmail uarch0B Feb 19 17:58 maildirfolder
drwx-- 2 vmail vmail uarch2B May 6 09:08 new
drwx-- 4 vmail vmail uarch4B Feb 27 16:44 sieve
-rw--- 1 vmail vmail uarch 364B May 2 10:25 subscriptions
drwx-- 2 vmail vmail uarch2B May 6 09:08 tmp
<<<
Notice the .Sent, .Trash folders are in the root.
this is what dovecot is probably returning in the login imap parameters
However if you are using INBOX/Sent then outlook is probably getting
confusing thus causing the error ie the box would not be accessable.
another note do you have the auto creation turned on for the imap folders ?
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
}
If not then outlook would be able to autocreate the Special Folders
I found that when migrating from cyrus imap i ended up with 2 sets of
special folders which athough a mess was explanable because of what i am
mentioning here.
if you turn on the auto create and restart dovecot chances are this
might deal with your Special Folders as they will be re-created as per
what / how outlook want to access them. (hopefully)
Another note is the subscriptions file in the maildir
# cat subscriptions
V 2
INBOX
Drafts
Sent
INBOX Archives2010
INBOX Archives2011
INBOX Archives
INBOX Archives2012
INBOX Archives2013
INBOX Archives2014
INBOX Archives2015
INBOX Archives2016
INBOX Archives2017
INBOX Archives2018
INBOX Archives2019
INBOX Archives2020
INBOX Archives2021
INBOX Junk
INBOX Notes
Trash
INBOX Archives2022
INBOX ArchivesSent 2022
INBOX Trash
this file sets the imap folder list and dir structure.
I found i had to manually fix some of these entries when moving from
cyrus to dovecot
this file is the folder list that is sent to the IMAP email client and
some times will not update correctly. (thunderbird as an example a lot
of greyed out inaccessable folders were in this file and had to be removed)
Happy Hunting
Happy Friday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 5/6/2022 10:30 AM, Philipp Ewald wrote:
Hello Paul and Kevin,
thanks for your replys!
Problem is, i am Administrator from mail Server and customer has there
own administrator.
Me and the other IT don't know whats the problem is.
I will tell them too do some more testing and maybe switching
Mailclient, Outlooks is horrible...
I will try other Ports and ask if there can clean up the mailbox (PC was
already cleaned and still not work)
On time i was there and i had no problem with my Linux + Thunderbird...
so network is working.
1 - Time it. Is it hitting a specific time and giving up?
It looks like there is instant problem, Outlook reporting problem Moving
Mail into "Sent"
2 - Does the email have any specific content?
its random problem, mostly this error happen when there are some more
recipient, but sometime problem with less recipient.
3 - Do they have anything like A/V software that might be acting to
try and proxy / scan the email?
there told me there has tested on a new installed windows with out any
"rules/extra software" applied on
Thanks i will report if we had some success.
Have a nice weekend!
Am 06.05.22 um 12:55 schrieb Paul Kudla (SCOM.CA Internet Services Inc.):
What version of outlook is your customer using?
and what internet provider (cell, dsl, cable, fiber etc) ?
anything past 2010 can be a crap shoot at best.
I have had many many customers try to use outlook because it is
fancier interface wise but outlook just does not really support imap
properly.
Last time i googled this it cam back with over 2 millions hits on
outlook imap issues. All suggestions fell back to connect using
exchange protocol's
Re: IMAP connection lost while sending mail
What version of outlook is your customer using? and what internet provider (cell, dsl, cable, fiber etc) ? anything past 2010 can be a crap shoot at best. I have had many many customers try to use outlook because it is fancier interface wise but outlook just does not really support imap properly. Last time i googled this it cam back with over 2 millions hits on outlook imap issues. All suggestions fell back to connect using exchange protocol's there was no real answer. I have been testing myself with 2019 office and it seems to work kinda. On another major note with outlook it can only handle up to maybe 1000 emails per folder or it will constantly sync and spend more time doing that then anything else. Outlook also does NOT support idle. (again echange thingy) I find thunderbird works best (even though they have issues to sort out as well here and there) I run thunderbird and can comfortably run up to 1 emails in a folder. It's a microsoft thing pushing exchange / outlook 365 against imap. This is NO Fault of Dovecot / Cyrus or any other imap server. Best suggestions for outlook look is to use Port 143 for imap (you can use ssl on port 143) Port 587 for smtp (25 is sometimes blocked by your isp) - Note on smtp connections there are some isp's blocking normal ports, i had to open up 1 on my postfix to get around at blocking smtp sends from a tethered cell phone connection for one of my customers which fixed the issue. Also outlook timeouts for both send and receive should be pushed up to a couple of minutes to cover any timing issues (although 30 seconds should be ok in most cases i have found outlook to fail with unknown reasons). One more thing about outlook is to speed up the user interface in some cases they will give you an option on the bottom of the inbox to load more messages, this should be checked as i have had several customers forget about old emails and the inbox could just simply be going past 1000 messages without any kind of warning. (see above) Hope this helps Happy Friday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 5/6/2022 3:53 AM, Philipp Ewald wrote: Hello, we have a strange problem. Costumer use Outlook to connect to our IMAP and SMTP Server. While sending Mails Outlooks reports connection lost. In Mail log there is no reason logged or something else. What i dont understand is that IMAP lose connetion while using SMTP. Outlloks reports problem by moving mails into Send dir. Is that a Outlook problem? i have enables debug logging and cant see any reason :( Network was already testet. any ideas? Kind regards Philipp
Re: no full syncs after upgrading to dovecot 2.3.18
Thanks for the update I dont push anyone when asking for updates I am a programmer by trade as well and nothing ever goes as planned prefer we all take our time and roll it out correctly then jumping the gun. Why I am trying to help elsewhere as I have gotten pretty fluid with dovecot etc and can help users out with the day to day stuff. I just can't help with ldap, never got around to that as i use pgsql databases that are replicated etc etc etc on all my configs. Again thanks for the update. Happy Thursday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 4/28/2022 7:02 AM, Aki Tuomi wrote: 2.3.19 is round the corner, so not long. I cannot yet promise an exact date but hopefully within week or two. Aki On 28/04/2022 13:57 Paul Kudla (SCOM.CA Internet Services Inc.) wrote: Thanks for the update. is this for both replication issues (folders +300 etc) Just Asking - Any ETA Happy Thursday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 4/27/2022 9:01 AM, Aki Tuomi wrote: Hi! This is probably going to get fixed in 2.3.19, this looks like an issue we are already fixing. Aki On 26/04/2022 16:38 Paul Kudla (SCOM.CA Internet Services Inc.) wrote: Agreed there seems to be no way of posting these kinds of issues to see if they are even being addressed or even known about moving forward on new updates i read somewhere there is a new branch soming out but nothing as of yet? 2.4 maybe 5.0 my previous replication issues (back in feb) went unanswered. not faulting anyone, but the developers do seem to be disconnected from issues as of late? or concentrating on other issues. I have no problem with support contracts for day to day maintence however as a programmer myself they usually dont work as the other end relies on the latest source code anyways. Thus can not help. I am trying to take a part the replicator c programming based on 2.3.18 as most of it does work to some extent. tcps just does not work (ie 600 seconds default in the c programming) My thoughts are tcp works ok but fails when the replicator through dsync-client.c when asked to return the folder list? replicator-brain.c seems to control the overall process and timing. replicator-queue.c seems to handle the que file that does seem to carry acurate info. things in the source code are documented enough to figure this out but i am still going through all the related .h files documentation wise which are all over the place. there is no clear documentation on the .h lib files so i have to walk through the tree one at a time finding relative code. since the dsync from doveadm does see to work ok i have to assume the dsync-client used to compile the replicator is at fault somehow or a call from it upstream? Thanks for your input on the other issues noted below, i will keep that in mind when disassembling the source code. No sense in fixing one thing and leaving something else behind, probably all related anyways. i have two test servers avaliable so i can play with all this offline to reproduce the issues Unfortunately I have to make a living first, this will be addressed when possible as i dont like systems that are live running this way and currently only have 5 accounts with this issue (mine included) Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 4/26/2022 9:03 AM, Reuben Farrelly wrote: I ran into this back in February and documented a reproducible test case (and sent it to this list). In short - I was able to reproduce this by having a valid and consistent mailbox on the source/local, creating a very standard empty Maildir/(new|cur|tmp) folder on the remote replica, and then initiating the replicate from the source. This consistently caused dsync to fail replication with the error "dovecot.index reset, view is now inconsistent" and sync aborted, leaving the replica mailbox in a screwed up inconsistent state. Client connections on the source replica were also dropped when this error occurred. You can see the error by enabling debug level logging if you initiate dsync manually on a test mailbox. The only workaround I found was to remove the remote Maildir and let Dovecot create the whole thing from scratch. Dovecot did not like any existing folders on the destination replica even if they were the same names as the source and completely empty. I was able to reproduce this the bare minimum of folders - just an INBOX! I have no idea if any of the developers saw my
Re: no full syncs after upgrading to dovecot 2.3.18
Thanks for the update. is this for both replication issues (folders +300 etc) Just Asking - Any ETA Happy Thursday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 4/27/2022 9:01 AM, Aki Tuomi wrote: Hi! This is probably going to get fixed in 2.3.19, this looks like an issue we are already fixing. Aki On 26/04/2022 16:38 Paul Kudla (SCOM.CA Internet Services Inc.) wrote: Agreed there seems to be no way of posting these kinds of issues to see if they are even being addressed or even known about moving forward on new updates i read somewhere there is a new branch soming out but nothing as of yet? 2.4 maybe 5.0 my previous replication issues (back in feb) went unanswered. not faulting anyone, but the developers do seem to be disconnected from issues as of late? or concentrating on other issues. I have no problem with support contracts for day to day maintence however as a programmer myself they usually dont work as the other end relies on the latest source code anyways. Thus can not help. I am trying to take a part the replicator c programming based on 2.3.18 as most of it does work to some extent. tcps just does not work (ie 600 seconds default in the c programming) My thoughts are tcp works ok but fails when the replicator through dsync-client.c when asked to return the folder list? replicator-brain.c seems to control the overall process and timing. replicator-queue.c seems to handle the que file that does seem to carry acurate info. things in the source code are documented enough to figure this out but i am still going through all the related .h files documentation wise which are all over the place. there is no clear documentation on the .h lib files so i have to walk through the tree one at a time finding relative code. since the dsync from doveadm does see to work ok i have to assume the dsync-client used to compile the replicator is at fault somehow or a call from it upstream? Thanks for your input on the other issues noted below, i will keep that in mind when disassembling the source code. No sense in fixing one thing and leaving something else behind, probably all related anyways. i have two test servers avaliable so i can play with all this offline to reproduce the issues Unfortunately I have to make a living first, this will be addressed when possible as i dont like systems that are live running this way and currently only have 5 accounts with this issue (mine included) Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 4/26/2022 9:03 AM, Reuben Farrelly wrote: I ran into this back in February and documented a reproducible test case (and sent it to this list). In short - I was able to reproduce this by having a valid and consistent mailbox on the source/local, creating a very standard empty Maildir/(new|cur|tmp) folder on the remote replica, and then initiating the replicate from the source. This consistently caused dsync to fail replication with the error "dovecot.index reset, view is now inconsistent" and sync aborted, leaving the replica mailbox in a screwed up inconsistent state. Client connections on the source replica were also dropped when this error occurred. You can see the error by enabling debug level logging if you initiate dsync manually on a test mailbox. The only workaround I found was to remove the remote Maildir and let Dovecot create the whole thing from scratch. Dovecot did not like any existing folders on the destination replica even if they were the same names as the source and completely empty. I was able to reproduce this the bare minimum of folders - just an INBOX! I have no idea if any of the developers saw my post or if the bug has been fixed for the next release. But it seemed to be quite a common problem over time (saw a few posts from people going back a long way with the same problem) and it is seriously disruptive to clients. The error message is not helpful in tracking down the problem either. Secondly, I also have had an ongoing and longstanding problem using tcps: for replication. For some reason using tcps: (with no other changes at all to the config) results in a lot of timeout messages "Error: dsync I/O has stalled, no activity for 600 seconds". This goes away if I revert back to tcp: instead of tcps - with tcp: I very rarely get timeouts. No idea why, guess this is a bug of some sort also. It's disappointing that there appears to be no way to have these sorts or problems addressed like there once was. I am not using Dovecot for commercial purposes so paying a fortune for a support contract for a high end installation just isn't goin
Re: Better not post your email password on a public mailing list, was: Re: no full syncs after upgrading to dovecot 2.3.18
thanks i love to share but sometime forget whats noted inside a config file Been meaning to change this for a while anyways. Happy Thursday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 4/27/2022 8:57 AM, Daniel Lange wrote: Am 26.04.22 um 11:36 schrieb Paul Kudla (SCOM.CA Internet Services Inc.): #imapc_host = mail.scom.ca #imapc_password = Pk554669 #imapc_user = p...@scom.ca I suggest to change that password immediately. $ openssl s_client -crlf -connect mail.scom.ca:993 CONNECTED(0003) * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN] SCOM.CA Internet Services Inc. - Dovecot ready A login p...@scom.ca Pk554669 A OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE SAVEDATE LITERAL+ NOTIFY SPECIAL-USE] Logged in A status INBOX (messages) * STATUS INBOX (MESSAGES 344) A OK Status completed (0.002 + 0.000 + 0.001 secs). ^C Kind regards, Daniel
Re: Can I set a different certificate per listen port?
options.quiet == None :
print ('\nGetting CRT Certificate : %s\n' %options.crt)
if options.csr != None :
ssl = options.csr
if options.quiet == None :
print ('\nGetting CSR Certificate : %s\n' %options.csr)
if options.inter != None :
ssl = options.inter
if options.quiet == None :
print ('\nGetting Inter Certificate : %s\n' %options.inter)
if options.pem != None :
ssl = options.pem
if options.quiet == None :
print ('\nGetting Pem Certificate : %s\n' %options.pem)
#sys.exit()
#from lib import *
#print ('Opening the Database ')
conn = psycopg2.connect(host='localhost', port = 5433,
database='db_table', user='pgsql', password='password')
pg = conn.cursor()
#print ('Connected !')
#Ok now go get the email keys
command = ("""select domain,ssl_key,ssl_cert,ssl_csr,ssl_chain from
email_ssl_certificates where domain = $$%s$$ """ %ssl)
#print (command)
pg.execute(command)
certs = pg.fetchone()
#print (certs)
#ok from here we have to decide the output ?
domain = certs[0]
if options.cert != None :
key = '#SSL Pem file (Key / Certificate / Intermediate) for
%s\n\n#Key\n\n' %domain + certs[1] + '\n\n#Certificate\n' + certs[2] +
'\n\n#Intermediate\n' + certs[4]
if options.key != None :
key = '#SSL Key file for %s\n\n' %domain + certs[1]
if options.crt != None :
key = '#SSL CERT file for %s\n\n' %domain + certs[2]
if options.csr != None :
key = '#SSL CSR Request file for %s\n\n' %domain + certs[3]
if options.inter != None :
key = '#SSL Intermediate file for %s\n\n' %domain + certs[4]
if options.pem != None :
key = '#SSL Pem (Certificate / Intermediate) file for
%s\n\n#Certificate\n\n' %domain + certs[2] + '\n\n#Intermediate\n' +
certs[4]
key = key.replace('\r','')
print (key)
conn.close()
sys.exit()
Happy Thursday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 4/28/2022 4:01 AM, Kees van Vloten wrote:
Op 28-04-2022 om 07:30 schreef Aki Tuomi:
On 27/04/2022 22:14 Kees van Vloten wrote:
Hi all,
I am trying to setup dovecot to listen to imaps on the local network and
through haproxy from the internet.
service imap-login {
inet_listener imaps {
port = 993
ssl = yes
}
inet_listener imaps_haproxy {
haproxy = yes
port = 10993
ssl = yes
}
}
Obviously the dns-name on the internet connection (10993) is different
than on the lan (993).
In the docs
(https://doc.dovecot.org/configuration_manual/dovecot_ssl_configuration/)
I found multiple options, but unfortunately none of those have the
option to distinguish per listen port.
Is there a way to setup two different certificates for the two
listeners?
- Kees
Hi!
Currently port is not supported. What we usually recommend here is
that you use haproxy to distribute connections to different local IP
addresses and use
local 127.0.0.5/32 {
ssl_cert=
Hi Aki,
Would it then look like this?
Internet -> haproxy on dmz-server -> haproxy on mailserver -> dovecot on
127.0.0.5
- Kees
Re: no full syncs after upgrading to dovecot 2.3.18
Agreed there seems to be no way of posting these kinds of issues to see if they are even being addressed or even known about moving forward on new updates i read somewhere there is a new branch soming out but nothing as of yet? 2.4 maybe 5.0 my previous replication issues (back in feb) went unanswered. not faulting anyone, but the developers do seem to be disconnected from issues as of late? or concentrating on other issues. I have no problem with support contracts for day to day maintence however as a programmer myself they usually dont work as the other end relies on the latest source code anyways. Thus can not help. I am trying to take a part the replicator c programming based on 2.3.18 as most of it does work to some extent. tcps just does not work (ie 600 seconds default in the c programming) My thoughts are tcp works ok but fails when the replicator through dsync-client.c when asked to return the folder list? replicator-brain.c seems to control the overall process and timing. replicator-queue.c seems to handle the que file that does seem to carry acurate info. things in the source code are documented enough to figure this out but i am still going through all the related .h files documentation wise which are all over the place. there is no clear documentation on the .h lib files so i have to walk through the tree one at a time finding relative code. since the dsync from doveadm does see to work ok i have to assume the dsync-client used to compile the replicator is at fault somehow or a call from it upstream? Thanks for your input on the other issues noted below, i will keep that in mind when disassembling the source code. No sense in fixing one thing and leaving something else behind, probably all related anyways. i have two test servers avaliable so i can play with all this offline to reproduce the issues Unfortunately I have to make a living first, this will be addressed when possible as i dont like systems that are live running this way and currently only have 5 accounts with this issue (mine included) Happy Tuesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 On 4/26/2022 9:03 AM, Reuben Farrelly wrote: I ran into this back in February and documented a reproducible test case (and sent it to this list). In short - I was able to reproduce this by having a valid and consistent mailbox on the source/local, creating a very standard empty Maildir/(new|cur|tmp) folder on the remote replica, and then initiating the replicate from the source. This consistently caused dsync to fail replication with the error "dovecot.index reset, view is now inconsistent" and sync aborted, leaving the replica mailbox in a screwed up inconsistent state. Client connections on the source replica were also dropped when this error occurred. You can see the error by enabling debug level logging if you initiate dsync manually on a test mailbox. The only workaround I found was to remove the remote Maildir and let Dovecot create the whole thing from scratch. Dovecot did not like any existing folders on the destination replica even if they were the same names as the source and completely empty. I was able to reproduce this the bare minimum of folders - just an INBOX! I have no idea if any of the developers saw my post or if the bug has been fixed for the next release. But it seemed to be quite a common problem over time (saw a few posts from people going back a long way with the same problem) and it is seriously disruptive to clients. The error message is not helpful in tracking down the problem either. Secondly, I also have had an ongoing and longstanding problem using tcps: for replication. For some reason using tcps: (with no other changes at all to the config) results in a lot of timeout messages "Error: dsync I/O has stalled, no activity for 600 seconds". This goes away if I revert back to tcp: instead of tcps - with tcp: I very rarely get timeouts. No idea why, guess this is a bug of some sort also. It's disappointing that there appears to be no way to have these sorts or problems addressed like there once was. I am not using Dovecot for commercial purposes so paying a fortune for a support contract for a high end installation just isn't going to happen, and this list seems to be quite ordinary for getting support and reporting bugs nowadays Reuben On 26/04/2022 7:21 pm, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: side issue if you are getting inconsistant dsyncs there is no real way to fix this in the long run. i know its a pain (already had to my self) i needed to do a full sync, take one server offline, delete the user dir (with dovecot offline) and then rsync (or somehow duplicate the main server's user
