Re: [Dovecot] 2.0 migration weirdnesses: logs and hang
On 23.8.2010, at 23.37, Kyle Wheeler wrote: >> In that trace Dovecot has sent reply to everything mutt has asked for. >> Although it looks like mutt has skipped logging some of the commands it has >> sent (LIST commands, e.g. a0002 or a0042). Perhaps it sent something else >> that Dovecot didn't reply to yet? .. Get a log with Dovecot's rawlog: >> http://wiki2.dovecot.org/Debugging/Rawlog > > I'm attaching the output of the rawlog. Could you do once more with -bt options?
Re: [Dovecot] 2.0 migration weirdnesses: logs and hang
On Friday, August 20 at 10:25 PM, quoth Timo Sirainen: On 20.8.2010, at 22.20, Kyle Wheeler wrote: On Friday, August 20 at 07:58 PM, quoth Timo Sirainen: Did you try this with IMAP client or via talking IMAP directly? I did it with mutt as my IMAP client. I'm attaching a log of the IMAP conversation. The log ends at the point that I had to kill mutt to regain control. It looks like mutt is stuck waiting for Dovecot to reply "a0049 OK Idle completed." In that trace Dovecot has sent reply to everything mutt has asked for. Although it looks like mutt has skipped logging some of the commands it has sent (LIST commands, e.g. a0002 or a0042). Perhaps it sent something else that Dovecot didn't reply to yet? .. Get a log with Dovecot's rawlog: http://wiki2.dovecot.org/Debugging/Rawlog I'm attaching the output of the rawlog. ~Kyle -- No people can be great who have ceased to be virtuous. -- Samuel Johnson, on the behavior of the British colonists in America dovecot.rawlog.tar.bz2 Description: application/tar-bz2 signature.asc Description: Digital signature
Re: [Dovecot] Messages causing an error from Solr.
On Mon, 2010-08-23 at 11:47 -0700, Erik Hetzner wrote: > > http://hg.dovecot.org/dovecot-1.2/rev/5efba9f9f0a7 > > http://hg.dovecot.org/dovecot-1.2/rev/cf0da2cd31fb > > Hi Timo, > > Unfortunately this second changeset (cf0da2cd31fb) seems to have > introduced a bug that results ever other character being dropped from > strings before they are indexed. Whops, yeah, pretty badly written patch. > Attached is a patch which fixes the problem with every > second character being dropped & results in a solr index that can be > searched for unicode characters (at least I tested it with latin > accents and with greek) Looked correct to me. Committed, thanks.
Re: [Dovecot] Messages causing an error from Solr.
At Fri, 20 Aug 2010 20:39:47 +0100, Timo Sirainen wrote: > > On Fri, 2010-08-20 at 09:53 -0700, Erik Hetzner wrote: > > > For what it’s worth, here are the “invalid XML character”s being > > complained about by Solr: > > Oh. It's not about illegal UTF8 sequences, but about some unicode > characters actually not being valid for XML. Hopefully these help: > > http://hg.dovecot.org/dovecot-1.2/rev/5efba9f9f0a7 > http://hg.dovecot.org/dovecot-1.2/rev/cf0da2cd31fb Hi Timo, Unfortunately this second changeset (cf0da2cd31fb) seems to have introduced a bug that results ever other character being dropped from strings before they are indexed. For instance, my username `egh` becomes `eh`, `spam` becomes `sa`, `drafts` becomes `dat`, etc. Furthermore I am not sure that the UTF-8 code is working as expected. Attached is a patch which fixes the problem with every second character being dropped & results in a solr index that can be searched for unicode characters (at least I tested it with latin accents and with greek) best, Erik solr_unicode.diff Description: Binary data pgp0Jy9nGY0lR.pgp Description: PGP signature
Re: [Dovecot] (Single instance) attachment storage
On Mon, 2010-07-19 at 17:29 +0100, William Blunn wrote: > > 2) If base64 attachment is in a standardized form that can be 100% > reliably converted back to its original form, it could be stored > decoded and then encoded back to original on the fly. This is now done: http://hg.dovecot.org/dovecot-2.0-sis/rev/3ef0ac874fd7 > Probably you would need to have a base64 matcher/decoder which is > smarter than normal base64 decoders and checks to make sure that all > lines (apart from the last) are encoded (a) canonically (e.g.. with no > trailing whitespace), and (b) using the same number of cells per line. Anything unexpected causes the attachment to be saved without decoding it. > Some systems finish the base64 stream with a newline (which in a > multipart manifests as a blank line between the base64 stream and the > '--' of the MIME boundary), whereas some systems finish the base64 > stream at the end of final 4-byte cell (which in a multipart manifests > as the '--' of the MIME boundary appearing on the line immediately > following the base64 encoded data). Your encoding allows for arbitrary > data between the objects, so you would have no problem store these two > cases verbatim. But something to watch out for when storing. I implemented this so that when end of base64 stream is encountered, it allows max. 1024 bytes of data after it. That data is saved in the dbox file instead of in the attachment file. So for example if the entire message body is a base64 encoded attachment but then some MTA appends a disclaimer after it, the attachment part is still saved to a separate file. I added that "max 1024 bytes after" so that if there is some weird virus/spam/whatever attachment that claims to be base64 but then actually is mostly non-base64 data, it could take less space by saving the entire part as attachment rather than only the base64 data decoded.
Re: [Dovecot] PATCH: SQL connect crash
On 8/23/10, Timo Sirainen wrote: > On Fri, 2010-08-20 at 18:27 -0300, Leandro Santi wrote: > >> Dovecot 2.0.0 auth process seems to crash while attempting to >> connect to a nonexistent SQL database. > .. >> #5 0x08062be2 in userdb_deinit (userdb=0x807fc48) at userdb.c:177 >> #6 0x0804ff4b in auths_deinit () at auth.c:213 > > Not when connecting, but when aborting such pending lookups during > deinit. Indeed, the segault triggers while cleaning up, after a failed connection attempt in my case. Subject line is somewhat misleading. Apologies for that ;) Leandro
Re: [Dovecot] 1.2.13 QRESYNC crash.
On Mon, 2010-08-23 at 17:33 +0100, David Woodhouse wrote: > And thanks for not (yet) making it reject the invalid command with the > 1:* in it I changed it in v2.0. > -- I'll need to come up with a strategy for migrating to the > 'correct' command on the client side, given that older versions of > dovecot won't accept it. > > I'll probably make the Evolution client code start off by trying the > correct command, and then retry with the bogus '1:*' if that fails. Can't you simply send 1:? Or if you want flags for messages you haven't even seen yet, 1:4294967295 should work too.
Re: [Dovecot] Authentication woes - Couldn't drop privileges: Mail access not allowed for root
On 23.08.10 15:37, Timo Sirainen wrote: > Yeah, that's how I thought it should work too. Fixed: > http://hg.dovecot.org/dovecot-2.0/rev/83f98d61bea2 I just installed Dovecot from Mercurial on the server running the dsync backup script. I can confirm that with your changes the SSH key file can be root-readable-only, which I consider a significant security improvement. Thank you! -Ralph
Re: [Dovecot] 1.2.13 QRESYNC crash.
On Mon, 2010-08-23 at 15:40 +0100, Timo Sirainen wrote: > Whops. Looks like I read the RFC a bit wrong. Fixed in v2.0 and v1.2 > hg now. I guess I should release 1.2.14 then. Could you try that the > fix works properly? At least it doesn't give any errors anymore. > http://hg.dovecot.org/dovecot-1.2/rev/7e959d397a35 Looks like it's working; thanks. Tested against 1.2.13. And thanks for not (yet) making it reject the invalid command with the 1:* in it -- I'll need to come up with a strategy for migrating to the 'correct' command on the client side, given that older versions of dovecot won't accept it. I'll probably make the Evolution client code start off by trying the correct command, and then retry with the bogus '1:*' if that fails. I don't really want to just 'fix' the client and let people get errors with older versions of Dovecot, even though QRESYNC support is optional. -- David WoodhouseOpen Source Technology Centre david.woodho...@intel.com Intel Corporation
Re: [Dovecot] Dovecot 2.0 and BSDI 4.X
On Mon, 23 Aug 2010 15:21:15 +0100 Timo Sirainen articulated: > On Mon, 2010-08-23 at 17:03 +0300, Odhiambo Washington wrote: > > > Looks like even I woke up my FreeBSD 4.0 today, Timo would be > > willing to help me make Dovecot 2.0 run on it. > > > > I think it's something to do with these hard core programmers. They > > never see failure as an option! > > If making Dovecot run on some specific OS requires only a few changes, > I'll do it. Doesn't matter how old it is. But if it's starting to look > like it would require larger changes, then I'll probably forget about > it.. Well, in my opinion, if the BSD people have all ready abandoned it, it is probably time for you to do so also. With FreeBSD-8.1 officially released, continued use of a deprecate OS makes no sense at all, assuming that the OP knows how to upgrade his installation. If he doesn't, then that is a whole new ballgame. -- Jerry ✌ dovecot.u...@seibercom.net Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __ Diplomacy is to do and say, the nastiest thing in the nicest way. Balfour
Re: [Dovecot] PATCH: SQL connect crash
On Fri, 2010-08-20 at 18:27 -0300, Leandro Santi wrote: > Dovecot 2.0.0 auth process seems to crash while attempting to > connect to a nonexistent SQL database. .. > #5 0x08062be2 in userdb_deinit (userdb=0x807fc48) at userdb.c:177 > #6 0x0804ff4b in auths_deinit () at auth.c:213 Not when connecting, but when aborting such pending lookups during deinit. That auth code is getting annoyingly complex.. Should really use a redesign some day to clean up all this horribleness. But for now, these should help: http://hg.dovecot.org/dovecot-2.0/rev/b88b207b79b2 http://hg.dovecot.org/dovecot-2.0/rev/8b85c3b89343 http://hg.dovecot.org/dovecot-2.0/rev/d2b49c7d4046
Re: [Dovecot] Over-quota bounces seeming to lack sender
On Fri, 2010-08-20 at 16:14 -0700, Dave Brenner wrote: > We're seeing an issue with 1.2.12 where over-quota rejection messages > are appearing in the logs with <> as the sender. Yes. > The sender of the > original message sees the appropriate sender (postmaster@) on the > rejection, but as the message makes its way through the postfix/amavis > chain, I can see that the sender address is missing. > > Aug 20 10:55:56 inet1 amavis[21822]: [ID 702911 mail.info] (21822-05) > Checking: rGDWxOHFPBdY <> -> Looks to me like a broken Postfix/amavis/something configuration. Not a Dovecot problem.
Re: [Dovecot] 1.2.13 QRESYNC crash.
On Sat, 2010-08-21 at 16:15 +0100, David Woodhouse wrote: > A00131 SELECT lists.bluez (QRESYNC (1154090296 1861 > (1,120,1578,2064,2226,2280,2298 1,120,12037,12523,12685,12739,12757))) > > Dovecot doesn't like that though: > > A00131 BAD Error in IMAP command SELECT: Invalid QRESYNC parameters Whops. Looks like I read the RFC a bit wrong. Fixed in v2.0 and v1.2 hg now. I guess I should release 1.2.14 then. Could you try that the fix works properly? At least it doesn't give any errors anymore. http://hg.dovecot.org/dovecot-1.2/rev/7e959d397a35
Re: [Dovecot] Dovecot 2.0 and BSDI 4.X
On Mon, 2010-08-23 at 17:03 +0300, Odhiambo Washington wrote: > Looks like even I woke up my FreeBSD 4.0 today, Timo would be willing to > help me make Dovecot 2.0 run on it. > > I think it's something to do with these hard core programmers. They never > see failure as an option! If making Dovecot run on some specific OS requires only a few changes, I'll do it. Doesn't matter how old it is. But if it's starting to look like it would require larger changes, then I'll probably forget about it..
Re: [Dovecot] DSYNC needs a lot more documentation
On Sat, 2010-08-21 at 09:51 -0700, Marc Perkel wrote: > Ok - when I type dsync at the command line it says: > > usage: dsync [-C ] [-m ] [-u ] [-frRv] >mirror | [@] | > > However the man page mentions nothing about any remote commands. I'm not really sure what more to write about it. It just needs to execute the dsync some way. > There > is a reference to ssh in one example but there isn't any kind of > overview as to how this all ties in. Does dsync pick up information from > dovecot.conf or dovecot to know where the email is an what format it is > in? Does dovecot need to be running on both ends? I added text about these and some other stuff. See updated http://wiki2.dovecot.org/Tools/Dsync > Does this run > continuously once you start it or does it need to be run once a minute? It's not continuous. > So - ServerA is running dovecot. On ServerB I want to have a live copy > so that if the drives on ServerA die I can recover on ServerB? Does it > do that? You can do that, yes. > I want to run dovecot on two servers so that if either server fails the > other seamlessly takes over and when the other comes back up they resync > as if nothing had happened. Is that possible? If so - how? Yes, as long as you call dsync for all users often enough. There's no super easy way to do this yet, see my other recent emails about this in some thread with dsync subject.. > If it just does backups, how is it different than rsync? It's faster.
Re: [Dovecot] Dovecot 2.0 and BSDI 4.X
On Sat, Aug 21, 2010 at 11:58 AM, Stan Hoeppner wrote: > The Doctor put forth on 8/20/2010 9:24 PM: > > On Thu, Aug 19, 2010 at 06:22:09PM +0100, Timo Sirainen wrote: > >> On Wed, 2010-08-18 at 23:37 -0600, The Doctor wrote: > >>> Aug 18 23:25:31 doctor dovecot: log: Error: net_accept() failed: > Invalid argument > >> > >> http://hg.dovecot.org/dovecot-2.0/rev/565f18727209 fixes this? > > Wind River discontinued BSD/OS (BSDi) in 2003, 7 years ago. This version > of > BSDi "The Doctor" is running thus hasn't seen a patch for _at least_ 7 > years. > Seven year old gcc environment? No wonder things don't work. > > Is there any real legitimate reason either of you are wasting your time > with > trying to make Dovecot work on a 7+ year old _DEAD_ operating system? > > Why haven't you moved this box up to the latest version of FreeBSD or > OpenBSD? > Looks like even I woke up my FreeBSD 4.0 today, Timo would be willing to help me make Dovecot 2.0 run on it. I think it's something to do with these hard core programmers. They never see failure as an option! This BSD/OS fellow really need to run something else. 7 year old OS - it's like running Windows (spit!) 95 in 2010. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ "If you have nothing good to say about someone, just shut up!." -- Lucky Dube
Re: [Dovecot] Authentication woes - Couldn't drop privileges: Mail access not allowed for root
On Sat, 2010-08-21 at 11:10 +0200, Ralph Seichter wrote: > I made a small mistake during setup and had one of the id_dsa files > owned by root instead of $user. This was deliberate, because the sync > script is run by root and I thought that the SSH key should thus be > owned by root aswell. Yeah, that's how I thought it should work too. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/83f98d61bea2 > The resulting error message was: > > Permission denied (publickey,keyboard-interactive). > dsync-local(foouser): Error: read() from worker server failed: EOF This is ok. > dsync-local(foouser): Fatal: dsync backup: Looks like you're trying to run > backup in wrong direction. Source is empty and destination is not. This should have been logged. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/6ef21adeb61d > Do you think one should add an example for key-based authentication to > http://wiki2.dovecot.org/Tools/Dsync ? With the above changes, I guess all that's needed for that is to add "-i id_dsa.backup" parameter to: dsync -u username mirror ssh mailu...@example.com dsync -u username
Re: [Dovecot] Dovecot 2.0 and BSDI 4.X
On Sat, 2010-08-21 at 07:08 -0600, The Doctor wrote: > > > Question how is the auth process handled in 2 that differs from 1? > > > > What does that have to do with anything? > > I am just wondering what else has changed? > > IT seems that the authentication is caught > in a non-ending loop. Why does it seem that? What exactly does it say now log when starting up? Does it answer/log anything when telneting to port 143?
Re: [Dovecot] imap, imaps, ports; config for secure-only service
On Sun, 2010-08-22 at 16:47 +0100, Ron Leach wrote:
> But I want to force secure working - in some kind of secure-only mode,
> so that internet-based users can reach the server securely. So I
> changed the protocols to:
>
> protocol = imaps
>
> with:
>
> disable_plaintext_auth = yes
>
>
> In this configuration, TB could not connect on 143, but only on 993,
> *and*, only if TB's SSL option is selected (not its TLS option). This
> was good, and bad.
You need either v2.0 or run two separate Dovecot installations with
different configs. With v2.0 you could do simply:
disable_plaintext_auth = yes
remote_ip 10.0.0.0/8 {
disable_plaintext_auth = no
}
or something like that.
Re: [Dovecot] imap, imaps, ports; config for secure-only service
On 2010-08-22 11:47 AM, Ron Leach wrote: > But I want to force secure working - in some kind of secure-only > mode, so that internet-based users can reach the server securely. > So I changed the protocols to: > I think I've disabled insecure access from any client - which is a > pity because we have one client application that is not > SSL/TLS-capable, as I mentioned before. There are 2 ways I can think of to do this... To do it with dovecot you'd use extra fields (allow_nets to be precise)... http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets Another way would be to open both ports, but limit access to the insecure port to only the desired host(s) using your firewall... ie, block access to port 143 at the firewall for all external clients... -- Best regards, Charles
