Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Timo Sirainen wrote: http://wiki.dovecot.org/TimeMovedBackwards bottom answers this. A restart isn't much better than just ignoring the time change. In point 3 (you can use clockspeed as well) you could mention chrony, too: http://chrony.sunsite.dk/ Thank you, Arno
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Matthijs Kooijman wrote: but I think it's not so trivial to handle this properly, without risking data loss (as Timo pointed out, immediately restarting is not really helping, since you'll still be running in the past. It would be interesting (as I do not know anything about the dovecot internals) why the current time is so important for dovecot. In my simple thinking the mail server has to manage a set of mails that all have a time stamp. Why is the time that the mail server is running in important for this management? Thanks, Arno
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Matthijs Kooijman wrote: In point 3 (you can use clockspeed as well) you could mention chrony, too: That page has an edit button, so feel free! Done. I hope I have not destroyed anything. Bye, Arno.
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Charles Marcus wrote: On 6/10/2009, Arno Wald (arno.w...@netcologne.de) wrote: Why is the time that the mail server is running in important for this management? No offense, but are you serious? Well, I really meant the question seriously. A server gets mails, stores them and delivers them again, regardless of the current time. The only thing that is logical for me in case of a wrong time setting is, that the retrieval time stamp will be wrong. But why has a mail server a problem in handling mails with a future time stamp? If so, I certainly hope you aren't running a public mail server. No I do not. Mail server/handling and network configuration is one of the big magics for me. --
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Timo Sirainen wrote: Hmm. I suppose I could change Dovecot master so that if no imap/pop3 processes have been created yet, it would silently ignore the clock move. Also it might be an idea to just restart dovecot instead of completely stopping it. If this happens to often in a certain time interval it still could be stopped to avoid too many restarts in a loop if there is an error situation somehow. Bye, Arno
[Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Hallo, I am running dovecot on a PC (a workstation) to have a mail client independent storage for my mails. Now I would like to have the system clock set correctly by using ntpd or ntpdate (using debian/sid). The problem is, that the PC is not online at boot time, but is set online on demand manually using pon to start the pppd later. So ntpd cannot sync the time on boot time before dovecot gets started. First I was using ntpdate that was started in ppp's if.up.d/ directory. This tool does set the time very hard instead of just slightly shifting some milliseconds. So dovecot did stop itself, reporting: Fatal: Time just moved backwards by 118 seconds. This might cause a lot of problems, so I'll just kill myself now. So I have tried ntpd as I thought it shifts the time in smaller time deltas. But this takes much time (several seconds) when booting, giving up on all configured servers, because they are not reachable when booting (as going online later manually). This turns off ntp functionality because ntpd does think that all servers are unreachable for ever. (I have tried a command dynamic in ntp.conf but this did not change anything and ntpd reports it to be obsolete.) My idea now is to not start ntpd on system boot, but only on if-up. But this brings up the same fatal error of dovecot as ntpd seems to hardly set the time, too. The only idea I have left is to stop dovecot, start ntpd and then start dovecot again on if-up. Is there a more elegant way to use dovecot and ntpd on a manually dialed in PC? Thanks, Arno
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Scott Haneda wrote: Can you explain why the system clock gets so far out of time? No, I cannot, I do not know. Is it possible that the clock is out of order? I did compare the times with another PC that is ntpd controlled. And after 1 hour the times differ for 1 second again. It seems that my PC is too fast :) After some days (I do not know when the last ntp-sync did really happen because of the problems with ntpd that I have described) my PC was about 2 minutes in the future. --
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Timo Sirainen wrote: Wonder if http://cr.yp.to/clockspeed.html would work better than ntpd? This is interesting, thank you. I will give it a try if ntpd or ntpdate will be no solution for me. But as there is no package for this in debian I would like to use the ntp stuff instead first. Arno
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
John Gateley wrote: 2 things might help: Thank you for your suggestions. This reminded me of an ntpdate option: ntpdate can be configured to change the time not in a big step on startup by using the option -B Force the time to always be slewed using the adjtime() system call, even if the measured offset is greater than +-128 ms... I try if this will help. (I just do not understand if ntpdate keeps changing the time until it is correct or if it only is changing it a bit each time it is executed.) Arno
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Pascal Volk wrote: On Debian systems I'm very happy with the OpenBSD NTP daemon. Package: openntpd This ntpd adjusts the local time in little steps. Also on startup? ntpd uses little steps while running, too. But only at startup it seems to do a big step. But as I have found in the Debian-Changelog ntpd should handle the unreachable server situation automatically in current versions. This is way the dynamic command is obsolete. Maybe I did not wait long enough when testing this. So I could try ntpd again. But good to know there is the alternative openntp From the openntp package description: Alternative packages which provide similar functionality are ntp and chrony. And chrony does what clockspeed seems to do. ;) `chronyd' determines the rate at which the computer gains or loses time, and compensates for this. Arno
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Juergen Daubert wrote: Chrony is what you are looking for, see http://chrony.sunsite.dk/ chrony does exactly look like what I need. But there is one big disadvantage when using it on manual dial up PCs: You have to configure the NTP-servers by IP-address instead of there names. I do not like to hardcode IP-addresses in a config file. (Also server-pools will not work (if I am right) where one server name returns different IP addresses each time for better load balancing or whatever.) It would be much better if chrony would look up the server addresses again when it does recieve the online state command. It seems that there is no option to turn on such a functionality. Thanks, Arno
Re: [Dovecot] dovecot and ntp: Fatal: Time just moved backwards
Arno Wald wrote: It would be much better if chrony would look up the server addresses again when it does recieve the online state command. It seems that there is no option to turn on such a functionality. For completeness, even if it is getting OT in this mailing list (Sorry): http://chrony.sunsite.dk/faq.php#question_5.2 The problem is that chronyd (currently) isn't designed in a way that allows hostname-IP address lookups during normal operation. I hope to work on this problem very soon. (How old is this entry? ;) ) The solution is to restart chrony at ppp ip-up: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=268289 My /etc/ppp/ip-up.d/chrony now is: - #!/bin/sh sleep 1 invoke-rc.d chrony restart sleep 1 ... the original unmodified script follows here ... - Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
I have running the older debian/sid dovecot 1:1.0.15-2.3 again, now with kernel 2.6.27-10.slh.1-sidux-686 and the issue seems to be fixed. So I recommend to use at least 2.6.27.10. Bye, Arno.
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
A new status report regarding this issue: Dovecot on my PC in the office is still running fine with kernel 2.6.26. Dovecot with the latest kernel 2.6.27-9.slh.1-sidux-686 on my PC at home did show the unkillable imap processes after a few minutes. Now I am running dovecot compiled without inotify support on this kernel without any problems for about 70 minutes. So I really think that the inotify stuff in kernel 2.6.27 does make the problem. (I will tell you if the imap process unexpectedly are making problems again in the current configuration.). So where are kernel issues reported? I will try to find out. Greetings, Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
Timo Sirainen wrote: One more thing you could try: Does the hang happen if you use configure --with-notify=dnotify ? I do not know if this is still interesting. But after notify=none did run for more than 3 hours without any problems, I am now testing dnotify since approximately 30 minutes without any problems, too. Ciao, Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
Timo Sirainen wrote: One more thing you could try: Does the hang happen if you use configure --with-notify=dnotify ? I first will keep the current version running for some hours to be more sure it really does not make problems. After this I can test it with dnotify. Thanks, Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
Timo Sirainen wrote: On Mon, 2008-12-15 at 12:00 +0100, Arno Wald wrote: So where are kernel issues reported? I will try to find out. Linux kernel mailing list is probably the best place. I could also write a summary mail about this and Cc it to you all who have had the problem. I would prefer this. I just have started to write a bugzilla entry on kernel.org, but as I do not know what the correct workflow for reporting issues in the kernel area is, I do not dare to press the send button yet. Greetings, Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
Timo Sirainen wrote: Or does it make more sense to try another kernel first? I guess that could also help. I started testing this with another kernel (2.6.26-6.slh.1-sidux-686). Until now (the last 15 hours) no such failing imap process did show up. So I guess it happens with 2.6.27 kernels. I will watch this during monday to be sure. btw: At another PC at home, also with a sidux-kernel 2.6.27 I had such an imap process, too, yesterday. (AMD Athlon(tm) XP 2200+) Bye, Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
Cor Bosman wrote: It seemed to be NFS related in our case. Is your Maildir directory mounted via NFS? I am using NFS-mounts, but not in connection with dovecot. Greetings, Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
David Rosenstrauch wrote: Has anyone reported this over on LKML yet? Or filed a bug? I did not yet. First I would like to test my self compiled dovecot without inotify against 2.6.27. Second I do not know where and how to report kernel issues. (Also I am a little bit afraid of the whole kernel stuff, because I do not know much about it.) Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
Timo Sirainen wrote: Anyway, Arno's ps output showed the process to be in R state, not in D It is definitely the R state. PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 6717 arno 20 0 2964 1608 1192 R 100 0.2 1158:05 imap btw: I have switched from imaps to imap protocol, because I thought this might change something. But it does not. Bye, Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
Timo Sirainen wrote: You could see if compiling Dovecot without inotify/dnotify support would help. I can't really think of anything else. I would like to try this and report the result. But there are so many configure-options that I do not know which options (and how) I should dis/enable. Could anybody give me the command line for the ./configure? That would be very kind. Or does it make more sense to try another kernel first? Thanks, Arno
Re: [Dovecot] unkillable imap process(es) with high CPU-usage
Timo Sirainen wrote: If you can't kill a process with -9, the bug is in the kernel and Do you have an idea, where and how I could report this to? (against the kernel package?). Perhaps I try an original debian kernel instead the sidux kernel first. there's nothing Dovecot can do about it. User spaces processes can't create unkillable processes unless something's broken. Although you could see if strace -p pid prints something. It's doubtful though if you can't kill the process. I have tried this already without success. The strace did not print anything and hang, without being able to be stopped by CTRL-C. Thank you for your answer, Arno
