Re: [chrony-users] Resume from suspend and default makestep configuration
On Tue, 19 May 2020, Pali Rohár wrote: On Tuesday 19 May 2020 17:36:15 Miroslav Lichvar wrote: On Tue, May 19, 2020 at 03:11:42PM +0200, Pali Rohár wrote: Also when resuming from hibernation you may have been completely powered off and also memory of system may have been modified. Plus multiOS scenario may have applied, e.g. ordinary user just "booted" windows and then turned it off and resumed linux from hibernation. I guess we would agree that ordinary user does not use any virtualisation as you described below. I don't think that's a common practice. If you suspend an OS and boot another, all kind of things can break, like corrupted swaps, etc. If you know what you are doing, fine, but don't be surprised when things break. I know that lot of people are doing it. They are not developers, sysadmins or people who watch mailing list, ... just normal users. Sure they are. So from my observation, this is common. Maybe it is less common by developers who know what can happen and break, but not uncommon by ordinary non-power users. When hibernating windows it puts special signature on NTFS filesystems and Linux's ntfs-fuse refuse to mount in R/W mode such "hibernated" NTFS filesystem. So there is no corruption of hibernated windows state. Windows does not support accessing ext4, btrfs or linux swap so there is corruption of linux fs/swap from windows. When chronyd is running, it assumes it has full control over the system clock. When you suspend and resume the OS or machine, the system clock is reset to the RTC. chronyd can see there was a forward jump, but it doesn't know what happened. systemd should know that and there could be a unit to call the chronyc reset and makestep commands if a significant offset is expected. But systemd cannot know that. It is chronyd who see that significant Sure it can. systemd knows that the system was hibernated. It did the hibernating. It does not need to look at the clock to know it was hibernated. So, systemd can stop and restart chronyd. Hibernation always causes system time problems. So, it is systemd that can ammeliorate the time problems caused by hibernation. Chrony does not know that the system was hibernated, it just sees a weird jump in time which it has not idea why it occured. jump occurred and only after it synchronize time via NTP. And until NTP daemon tell (somehow) hat this jumps occurred, systemd cannot know that during hibernation RTC clock was modified. systemd knows it was hibernated. It did it. So systemd has the responsibility of cleaning up after itself, which could include stopping and restarting chrony. This looks like a chicken and egg problem. systemd (or any other init / service system) does not know correct time after resuming system from suspend/hibernate, so it cannot check if RTC jump occurred. chronyd is But it know hibernation occured. waiting for some reset command from systemd to fix insecure system clock but systemd does not know that clock is incorrect... So the result it that system clock is incorrectly set, user has incorrect time. Sure it does. It know it hibernated and thus knows that time problems will have occured.
Re: [chrony-users] Resume from suspend and default makestep configuration
On Tuesday 19 May 2020 16:07:57 FUSTE Emmanuel wrote: > Le 19/05/2020 à 17:54, Pali Rohár a écrit : > > On Tuesday 19 May 2020 17:36:15 Miroslav Lichvar wrote: > >> On Tue, May 19, 2020 at 03:11:42PM +0200, Pali Rohár wrote: > >>> Also when resuming from hibernation you may have been completely powered > >>> off and also memory of system may have been modified. Plus multiOS > >>> scenario may have applied, e.g. ordinary user just "booted" windows and > >>> then turned it off and resumed linux from hibernation. I guess we would > >>> agree that ordinary user does not use any virtualisation as you > >>> described below. > >> I don't think that's a common practice. If you suspend an OS and boot > >> another, all kind of things can break, like corrupted swaps, etc. If > >> you know what you are doing, fine, but don't be surprised when things > >> break. > > I know that lot of people are doing it. They are not developers, > > sysadmins or people who watch mailing list, ... just normal users. > > So from my observation, this is common. Maybe it is less common by > > developers who know what can happen and break, but not uncommon by > > ordinary non-power users. > > > > When hibernating windows it puts special signature on NTFS filesystems > > and Linux's ntfs-fuse refuse to mount in R/W mode such "hibernated" NTFS > > filesystem. So there is no corruption of hibernated windows state. > > > > Windows does not support accessing ext4, btrfs or linux swap so there is > > corruption of linux fs/swap from windows. > > > >> When chronyd is running, it assumes it has full control over the > >> system clock. When you suspend and resume the OS or machine, the > >> system clock is reset to the RTC. chronyd can see there was a forward > >> jump, but it doesn't know what happened. systemd should know that and > >> there could be a unit to call the chronyc reset and makestep commands > >> if a significant offset is expected. > > But systemd cannot know that. It is chronyd who see that significant > Systemd know that you are resuming from suspend. Of course, systemd knows it. And similarly also other applications can know it. > > jump occurred and only after it synchronize time via NTP. And until NTP > Wrong. Chrony does not need to sync via NTP to see that the system clock > jumped. Ok, to be precise, chrony needs to sync via NTP to see that system clock (during suspend / hibernate) jumped with correct delay. Also what may happen is that clock does not jump, but it should. Also chronyd cannot detect it until it do sync via NTP. > > daemon tell (somehow) hat this jumps occurred, systemd cannot know that > > during hibernation RTC clock was modified. > That should not happen in normal case. But it happens, this is reason why I started this email thread. > > This looks like a chicken and egg problem. systemd (or any other init / > > service system) does not know correct time after resuming system from > > suspend/hibernate, so it cannot check if RTC jump occurred. chronyd is > RTC should be the trusted source in this case so init system, knowing In previous emails I show that generally it is not and cannot be fully trusted source. > that you are resuming from should notify ntp daemon that all is ok > (launch "chronyc reset" in the devel version). > > After that, on a "sane" computer, you could even now drop the makestep > parameter for the paranoids like me. -- Pali Rohár pali.ro...@gmail.com -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Resume from suspend and default makestep configuration
Le 19/05/2020 à 17:54, Pali Rohár a écrit : > On Tuesday 19 May 2020 17:36:15 Miroslav Lichvar wrote: >> On Tue, May 19, 2020 at 03:11:42PM +0200, Pali Rohár wrote: >>> Also when resuming from hibernation you may have been completely powered >>> off and also memory of system may have been modified. Plus multiOS >>> scenario may have applied, e.g. ordinary user just "booted" windows and >>> then turned it off and resumed linux from hibernation. I guess we would >>> agree that ordinary user does not use any virtualisation as you >>> described below. >> I don't think that's a common practice. If you suspend an OS and boot >> another, all kind of things can break, like corrupted swaps, etc. If >> you know what you are doing, fine, but don't be surprised when things >> break. > I know that lot of people are doing it. They are not developers, > sysadmins or people who watch mailing list, ... just normal users. > So from my observation, this is common. Maybe it is less common by > developers who know what can happen and break, but not uncommon by > ordinary non-power users. > > When hibernating windows it puts special signature on NTFS filesystems > and Linux's ntfs-fuse refuse to mount in R/W mode such "hibernated" NTFS > filesystem. So there is no corruption of hibernated windows state. > > Windows does not support accessing ext4, btrfs or linux swap so there is > corruption of linux fs/swap from windows. > >> When chronyd is running, it assumes it has full control over the >> system clock. When you suspend and resume the OS or machine, the >> system clock is reset to the RTC. chronyd can see there was a forward >> jump, but it doesn't know what happened. systemd should know that and >> there could be a unit to call the chronyc reset and makestep commands >> if a significant offset is expected. > But systemd cannot know that. It is chronyd who see that significant Systemd know that you are resuming from suspend. > jump occurred and only after it synchronize time via NTP. And until NTP Wrong. Chrony does not need to sync via NTP to see that the system clock jumped. > daemon tell (somehow) hat this jumps occurred, systemd cannot know that > during hibernation RTC clock was modified. That should not happen in normal case. > > This looks like a chicken and egg problem. systemd (or any other init / > service system) does not know correct time after resuming system from > suspend/hibernate, so it cannot check if RTC jump occurred. chronyd is RTC should be the trusted source in this case so init system, knowing that you are resuming from should notify ntp daemon that all is ok (launch "chronyc reset" in the devel version). After that, on a "sane" computer, you could even now drop the makestep parameter for the paranoids like me.
Re: [chrony-users] Resume from suspend and default makestep configuration
On Tuesday 19 May 2020 17:36:15 Miroslav Lichvar wrote: > On Tue, May 19, 2020 at 03:11:42PM +0200, Pali Rohár wrote: > > Also when resuming from hibernation you may have been completely powered > > off and also memory of system may have been modified. Plus multiOS > > scenario may have applied, e.g. ordinary user just "booted" windows and > > then turned it off and resumed linux from hibernation. I guess we would > > agree that ordinary user does not use any virtualisation as you > > described below. > > I don't think that's a common practice. If you suspend an OS and boot > another, all kind of things can break, like corrupted swaps, etc. If > you know what you are doing, fine, but don't be surprised when things > break. I know that lot of people are doing it. They are not developers, sysadmins or people who watch mailing list, ... just normal users. So from my observation, this is common. Maybe it is less common by developers who know what can happen and break, but not uncommon by ordinary non-power users. When hibernating windows it puts special signature on NTFS filesystems and Linux's ntfs-fuse refuse to mount in R/W mode such "hibernated" NTFS filesystem. So there is no corruption of hibernated windows state. Windows does not support accessing ext4, btrfs or linux swap so there is corruption of linux fs/swap from windows. > When chronyd is running, it assumes it has full control over the > system clock. When you suspend and resume the OS or machine, the > system clock is reset to the RTC. chronyd can see there was a forward > jump, but it doesn't know what happened. systemd should know that and > there could be a unit to call the chronyc reset and makestep commands > if a significant offset is expected. But systemd cannot know that. It is chronyd who see that significant jump occurred and only after it synchronize time via NTP. And until NTP daemon tell (somehow) hat this jumps occurred, systemd cannot know that during hibernation RTC clock was modified. This looks like a chicken and egg problem. systemd (or any other init / service system) does not know correct time after resuming system from suspend/hibernate, so it cannot check if RTC jump occurred. chronyd is waiting for some reset command from systemd to fix insecure system clock but systemd does not know that clock is incorrect... So the result it that system clock is incorrectly set, user has incorrect time. -- Pali Rohár pali.ro...@gmail.com -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Resume from suspend and default makestep configuration
On Tue, May 19, 2020 at 03:11:42PM +0200, Pali Rohár wrote: > Also when resuming from hibernation you may have been completely powered > off and also memory of system may have been modified. Plus multiOS > scenario may have applied, e.g. ordinary user just "booted" windows and > then turned it off and resumed linux from hibernation. I guess we would > agree that ordinary user does not use any virtualisation as you > described below. I don't think that's a common practice. If you suspend an OS and boot another, all kind of things can break, like corrupted swaps, etc. If you know what you are doing, fine, but don't be surprised when things break. When chronyd is running, it assumes it has full control over the system clock. When you suspend and resume the OS or machine, the system clock is reset to the RTC. chronyd can see there was a forward jump, but it doesn't know what happened. systemd should know that and there could be a unit to call the chronyc reset and makestep commands if a significant offset is expected. -- Miroslav Lichvar -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Resume from suspend and default makestep configuration
Le 19/05/2020 à 16:11, Pali Rohár a écrit : > On Tuesday 19 May 2020 13:40:18 FUSTE Emmanuel wrote: >> Le 19/05/2020 à 15:11, Pali Rohár a écrit : >> >>> In past I lot of time seen problem that Windows stored system time in >>> local timezone to RTC, then computer was rebooted to Linux which reads >>> system time from RTC in UTC and saw incorrect time. Installing NTP >>> daemon fixed this problem. And then after reboot Windows time was >>> shifted and after few seconds/minutes it synchronized it again against >>> Windows time server. >> A better workaround: just intruct linux that RTC is in localtimezone and >> not UTC and it would have worked. > I remember that this setup did not work in one case: when linux system > was booted prior booting windows system after DST change. Time was > shifted two times, once by linux, once by windows as windows did not > know that it should do it... > Emmanuel. Yes , the problem of shadow states. Only one could control the RTC, simultaneously (VM) or asynchronously (Multiboot). Your hibernation image is a shadow state too. Emmanuel.
Re: [chrony-users] Resume from suspend and default makestep configuration
On Tuesday 19 May 2020 13:40:18 FUSTE Emmanuel wrote: > Le 19/05/2020 à 15:11, Pali Rohár a écrit : > > On Tuesday 19 May 2020 12:42:28 FUSTE Emmanuel wrote: > >> Le 19/05/2020 à 13:30, Pali Rohár a écrit : > >>> On Tuesday 19 May 2020 11:10:01 FUSTE Emmanuel wrote: > Le 19/05/2020 à 12:29, Pali Rohár a écrit : > > On Monday 18 May 2020 13:45:04 FUSTE Emmanuel wrote: > >> Le 18/05/2020 à 13:15, Pali Rohár a écrit : > >>> On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: > Hello Pali, > > Le 18/05/2020 à 12:37, Pali Rohár a écrit : > > The main problem is when system is put into suspend or hibernate > > state. > > > > In my opinion resuming from suspend / hibernate state should be > > handled > > in the same way as (re)starting chronyd. You do not know what may > > happened during sleep. > Yes and in case of needed workaround, it should be done at the system > level, not chrony. > A job for systemd. > >>> Hello! Sorry for a stupid question, but what has systemd in common > >>> with > >>> chronyd? Why should systemd care about chronyd time synchronization? > >> Nothing. > >> But it is to your "process manager" being systemd, sysvinit pile of > >> scripts or whatever to restart or notify chrony, it has do do > >> housekeeping anyway for other things when you suspend/resume. > > Hm... I remember that in past it was needed to blacklist broken daemons, > > software and kernel modules which did not work correctly during S3 or > > hibernate state. It was in some pm scripts utils... > > > > But I thought that these days are already passed and software can deal > > with fact that machine may be put into suspend or hibernate state. > > > > So what you are suggesting is to put chronyd daemon into list of broken > > software (which needs to be stopped prior suspend / resume)? > > > > It does not make sense for me as the immediate step after putting > > software or kernel module into such "blacklist" was to inform upstream > > authors of that daemon or kernel module they it is broken / incompatible > > with suspend state and it should be fixed. > > > > That "blacklist" was just workaround for buggy software and not > > permanent solution. > No not chrony, but the machine which change RTC on your back : buggy Bios > >>> Sorry, but I have not caught this line. Blacklist contained list of > >>> buggy software, daemons and kernel modules which had to be (in past) > >>> stopped / unloaded prior system went to S3 and started / (re)loaded > >>> after system resumed. So obviously putting "buggy Bios" into blacklist > >>> not only does not make sense, but also it did nothing. In that > >>> particular case chronyd had to be put into that blacklist of buggy > >>> software as it as you described is chronyd which needs to be stopped / > >>> started... But as I said this was used in past when buggy software and > >>> kernel modules were there when they was not able to correctly handle S3 > >>> state. > >> I said the machine not chrony. > >> Please I'm not native English, but this conversation became more and > >> more like a trooling one. > >> Blacklist are black list, this is a generic term as you point out. > > Sorry for that. Lets call it just list. If you want to somehow use > > machine in that list, then you probably need tuple > > and teach scripts around to read that list as tuple and restart > > "software" if "machine" matches string of current machine on which it is > > running. > Yes and software in this case is "software that provide time sync" > > > > I'm saying that in past this was just list of "buggy" software and > > kernel modules which needs to be restarted during S3. It was not some > > smart structure where you was able to define rules like "if you are > > running on machine ABC then restart software CDE". And this is I guess > > what you want to achieve by putting machine on list. > > > >> Exactly as networkmanager, ifupdown scripts, systemd-networkd > >> reload/restart some network services when interfaces/tunnels/vpn are > >> upped/downed. > > This is something totally different. all those mentioned "services" are > > just independent part of system which manages network connections. > > > > chronyd is there to manage time synchronization. > It was an "imaged comparison" for event driven config change. > The event in the suspend vs time case, the event is only know and > should be managed by your init system not by your time daemon. > > > And as I pointed there are existing problems that UEFI/BIOS firmware > > changes RTC clock without good reason which results in completely > > wrong > > system clock. > > > Could well be identified by blacklist at the udev/systemd level for >
Re: [chrony-users] Resume from suspend and default makestep configuration
Le 19/05/2020 à 15:11, Pali Rohár a écrit : > On Tuesday 19 May 2020 12:42:28 FUSTE Emmanuel wrote: >> Le 19/05/2020 à 13:30, Pali Rohár a écrit : >>> On Tuesday 19 May 2020 11:10:01 FUSTE Emmanuel wrote: Le 19/05/2020 à 12:29, Pali Rohár a écrit : > On Monday 18 May 2020 13:45:04 FUSTE Emmanuel wrote: >> Le 18/05/2020 à 13:15, Pali Rohár a écrit : >>> On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: Hello Pali, Le 18/05/2020 à 12:37, Pali Rohár a écrit : > The main problem is when system is put into suspend or hibernate > state. > > In my opinion resuming from suspend / hibernate state should be > handled > in the same way as (re)starting chronyd. You do not know what may > happened during sleep. Yes and in case of needed workaround, it should be done at the system level, not chrony. A job for systemd. >>> Hello! Sorry for a stupid question, but what has systemd in common with >>> chronyd? Why should systemd care about chronyd time synchronization? >> Nothing. >> But it is to your "process manager" being systemd, sysvinit pile of >> scripts or whatever to restart or notify chrony, it has do do >> housekeeping anyway for other things when you suspend/resume. > Hm... I remember that in past it was needed to blacklist broken daemons, > software and kernel modules which did not work correctly during S3 or > hibernate state. It was in some pm scripts utils... > > But I thought that these days are already passed and software can deal > with fact that machine may be put into suspend or hibernate state. > > So what you are suggesting is to put chronyd daemon into list of broken > software (which needs to be stopped prior suspend / resume)? > > It does not make sense for me as the immediate step after putting > software or kernel module into such "blacklist" was to inform upstream > authors of that daemon or kernel module they it is broken / incompatible > with suspend state and it should be fixed. > > That "blacklist" was just workaround for buggy software and not > permanent solution. No not chrony, but the machine which change RTC on your back : buggy Bios >>> Sorry, but I have not caught this line. Blacklist contained list of >>> buggy software, daemons and kernel modules which had to be (in past) >>> stopped / unloaded prior system went to S3 and started / (re)loaded >>> after system resumed. So obviously putting "buggy Bios" into blacklist >>> not only does not make sense, but also it did nothing. In that >>> particular case chronyd had to be put into that blacklist of buggy >>> software as it as you described is chronyd which needs to be stopped / >>> started... But as I said this was used in past when buggy software and >>> kernel modules were there when they was not able to correctly handle S3 >>> state. >> I said the machine not chrony. >> Please I'm not native English, but this conversation became more and >> more like a trooling one. >> Blacklist are black list, this is a generic term as you point out. > Sorry for that. Lets call it just list. If you want to somehow use > machine in that list, then you probably need tuple > and teach scripts around to read that list as tuple and restart > "software" if "machine" matches string of current machine on which it is > running. Yes and software in this case is "software that provide time sync" > > I'm saying that in past this was just list of "buggy" software and > kernel modules which needs to be restarted during S3. It was not some > smart structure where you was able to define rules like "if you are > running on machine ABC then restart software CDE". And this is I guess > what you want to achieve by putting machine on list. > >> Exactly as networkmanager, ifupdown scripts, systemd-networkd >> reload/restart some network services when interfaces/tunnels/vpn are >> upped/downed. > This is something totally different. all those mentioned "services" are > just independent part of system which manages network connections. > > chronyd is there to manage time synchronization. It was an "imaged comparison" for event driven config change. The event in the suspend vs time case, the event is only know and should be managed by your init system not by your time daemon. > And as I pointed there are existing problems that UEFI/BIOS firmware > changes RTC clock without good reason which results in completely > wrong > system clock. > Could well be identified by blacklist at the udev/systemd level for applying or not the workaround (restart chrony or launch a chronyc command at resume) >>> Could you describe in details what do you mean by blacklist? Which udev >>> blacklist you mean and what should be put into that
Re: [chrony-users] Resume from suspend and default makestep configuration
On Tuesday 19 May 2020 12:42:28 FUSTE Emmanuel wrote: > Le 19/05/2020 à 13:30, Pali Rohár a écrit : > > On Tuesday 19 May 2020 11:10:01 FUSTE Emmanuel wrote: > >> Le 19/05/2020 à 12:29, Pali Rohár a écrit : > >>> On Monday 18 May 2020 13:45:04 FUSTE Emmanuel wrote: > Le 18/05/2020 à 13:15, Pali Rohár a écrit : > > On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: > >> Hello Pali, > >> > >> Le 18/05/2020 à 12:37, Pali Rohár a écrit : > >>> The main problem is when system is put into suspend or hibernate > >>> state. > >>> > >>> In my opinion resuming from suspend / hibernate state should be > >>> handled > >>> in the same way as (re)starting chronyd. You do not know what may > >>> happened during sleep. > >> Yes and in case of needed workaround, it should be done at the system > >> level, not chrony. > >> A job for systemd. > > Hello! Sorry for a stupid question, but what has systemd in common with > > chronyd? Why should systemd care about chronyd time synchronization? > Nothing. > But it is to your "process manager" being systemd, sysvinit pile of > scripts or whatever to restart or notify chrony, it has do do > housekeeping anyway for other things when you suspend/resume. > >>> Hm... I remember that in past it was needed to blacklist broken daemons, > >>> software and kernel modules which did not work correctly during S3 or > >>> hibernate state. It was in some pm scripts utils... > >>> > >>> But I thought that these days are already passed and software can deal > >>> with fact that machine may be put into suspend or hibernate state. > >>> > >>> So what you are suggesting is to put chronyd daemon into list of broken > >>> software (which needs to be stopped prior suspend / resume)? > >>> > >>> It does not make sense for me as the immediate step after putting > >>> software or kernel module into such "blacklist" was to inform upstream > >>> authors of that daemon or kernel module they it is broken / incompatible > >>> with suspend state and it should be fixed. > >>> > >>> That "blacklist" was just workaround for buggy software and not > >>> permanent solution. > >> No not chrony, but the machine which change RTC on your back : buggy Bios > > Sorry, but I have not caught this line. Blacklist contained list of > > buggy software, daemons and kernel modules which had to be (in past) > > stopped / unloaded prior system went to S3 and started / (re)loaded > > after system resumed. So obviously putting "buggy Bios" into blacklist > > not only does not make sense, but also it did nothing. In that > > particular case chronyd had to be put into that blacklist of buggy > > software as it as you described is chronyd which needs to be stopped / > > started... But as I said this was used in past when buggy software and > > kernel modules were there when they was not able to correctly handle S3 > > state. > I said the machine not chrony. > Please I'm not native English, but this conversation became more and > more like a trooling one. > Blacklist are black list, this is a generic term as you point out. Sorry for that. Lets call it just list. If you want to somehow use machine in that list, then you probably need tuple and teach scripts around to read that list as tuple and restart "software" if "machine" matches string of current machine on which it is running. I'm saying that in past this was just list of "buggy" software and kernel modules which needs to be restarted during S3. It was not some smart structure where you was able to define rules like "if you are running on machine ABC then restart software CDE". And this is I guess what you want to achieve by putting machine on list. > > > Exactly as networkmanager, ifupdown scripts, systemd-networkd > reload/restart some network services when interfaces/tunnels/vpn are > upped/downed. > >>> This is something totally different. all those mentioned "services" are > >>> just independent part of system which manages network connections. > >>> > >>> chronyd is there to manage time synchronization. > >> It was an "imaged comparison" for event driven config change. > >> The event in the suspend vs time case, the event is only know and > >> should be managed by your init system not by your time daemon. > >> > >>> And as I pointed there are existing problems that UEFI/BIOS firmware > >>> changes RTC clock without good reason which results in completely > >>> wrong > >>> system clock. > >>> > >> Could well be identified by blacklist at the udev/systemd level for > >> applying or not the workaround (restart chrony or launch a chronyc > >> command at resume) > > Could you describe in details what do you mean by blacklist? Which udev > > blacklist you mean and what should be put into that blacklist? I have > > not caught this part. > Faulty systems could be identified by DMI/ACPI strings and quirk applied. > >>> And
Re: [chrony-users] Resume from suspend and default makestep configuration
Le 19/05/2020 à 13:30, Pali Rohár a écrit : > On Tuesday 19 May 2020 11:10:01 FUSTE Emmanuel wrote: >> Le 19/05/2020 à 12:29, Pali Rohár a écrit : >>> On Monday 18 May 2020 13:45:04 FUSTE Emmanuel wrote: Le 18/05/2020 à 13:15, Pali Rohár a écrit : > On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: >> Hello Pali, >> >> Le 18/05/2020 à 12:37, Pali Rohár a écrit : >>> The main problem is when system is put into suspend or hibernate state. >>> >>> In my opinion resuming from suspend / hibernate state should be handled >>> in the same way as (re)starting chronyd. You do not know what may >>> happened during sleep. >> Yes and in case of needed workaround, it should be done at the system >> level, not chrony. >> A job for systemd. > Hello! Sorry for a stupid question, but what has systemd in common with > chronyd? Why should systemd care about chronyd time synchronization? Nothing. But it is to your "process manager" being systemd, sysvinit pile of scripts or whatever to restart or notify chrony, it has do do housekeeping anyway for other things when you suspend/resume. >>> Hm... I remember that in past it was needed to blacklist broken daemons, >>> software and kernel modules which did not work correctly during S3 or >>> hibernate state. It was in some pm scripts utils... >>> >>> But I thought that these days are already passed and software can deal >>> with fact that machine may be put into suspend or hibernate state. >>> >>> So what you are suggesting is to put chronyd daemon into list of broken >>> software (which needs to be stopped prior suspend / resume)? >>> >>> It does not make sense for me as the immediate step after putting >>> software or kernel module into such "blacklist" was to inform upstream >>> authors of that daemon or kernel module they it is broken / incompatible >>> with suspend state and it should be fixed. >>> >>> That "blacklist" was just workaround for buggy software and not >>> permanent solution. >> No not chrony, but the machine which change RTC on your back : buggy Bios > Sorry, but I have not caught this line. Blacklist contained list of > buggy software, daemons and kernel modules which had to be (in past) > stopped / unloaded prior system went to S3 and started / (re)loaded > after system resumed. So obviously putting "buggy Bios" into blacklist > not only does not make sense, but also it did nothing. In that > particular case chronyd had to be put into that blacklist of buggy > software as it as you described is chronyd which needs to be stopped / > started... But as I said this was used in past when buggy software and > kernel modules were there when they was not able to correctly handle S3 > state. I said the machine not chrony. Please I'm not native English, but this conversation became more and more like a trooling one. Blacklist are black list, this is a generic term as you point out. > Exactly as networkmanager, ifupdown scripts, systemd-networkd reload/restart some network services when interfaces/tunnels/vpn are upped/downed. >>> This is something totally different. all those mentioned "services" are >>> just independent part of system which manages network connections. >>> >>> chronyd is there to manage time synchronization. >> It was an "imaged comparison" for event driven config change. >> The event in the suspend vs time case, the event is only know and >> should be managed by your init system not by your time daemon. >> >>> And as I pointed there are existing problems that UEFI/BIOS firmware >>> changes RTC clock without good reason which results in completely wrong >>> system clock. >>> >> Could well be identified by blacklist at the udev/systemd level for >> applying or not the workaround (restart chrony or launch a chronyc >> command at resume) > Could you describe in details what do you mean by blacklist? Which udev > blacklist you mean and what should be put into that blacklist? I have > not caught this part. Faulty systems could be identified by DMI/ACPI strings and quirk applied. >>> And what is the faulty system? >> Citing yourself : >> >> "as I pointed there are existing problems that UEFI/BIOS firmware >> changes RTC clock without good reason" > Ok. Main problem is that there is no way how to identify such broken > firmwares. So definition is now nice and clear but basically useless as > it does not say anything how to find or identify such faulty systems. Yes that is the generic problem of faulty hw/devices/firmware, they are faulty but not on purpose. The kernel is full of theses lists. And they are build by hand with users/developers feedback but you know that in the Bt too world isn't it ? > >>> I think this is something general and not related to particular machine. >>> I guess under specific conditions it may happen on any system. >>> See for example /lib/udev/hwdb.d/60-sensor.hwdb for some
Re: [chrony-users] Resume from suspend and default makestep configuration
On Tuesday 19 May 2020 11:10:01 FUSTE Emmanuel wrote: > Le 19/05/2020 à 12:29, Pali Rohár a écrit : > > On Monday 18 May 2020 13:45:04 FUSTE Emmanuel wrote: > >> Le 18/05/2020 à 13:15, Pali Rohár a écrit : > >>> On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: > Hello Pali, > > Le 18/05/2020 à 12:37, Pali Rohár a écrit : > > The main problem is when system is put into suspend or hibernate state. > > > > In my opinion resuming from suspend / hibernate state should be handled > > in the same way as (re)starting chronyd. You do not know what may > > happened during sleep. > Yes and in case of needed workaround, it should be done at the system > level, not chrony. > A job for systemd. > >>> Hello! Sorry for a stupid question, but what has systemd in common with > >>> chronyd? Why should systemd care about chronyd time synchronization? > >> Nothing. > >> But it is to your "process manager" being systemd, sysvinit pile of > >> scripts or whatever to restart or notify chrony, it has do do > >> housekeeping anyway for other things when you suspend/resume. > > Hm... I remember that in past it was needed to blacklist broken daemons, > > software and kernel modules which did not work correctly during S3 or > > hibernate state. It was in some pm scripts utils... > > > > But I thought that these days are already passed and software can deal > > with fact that machine may be put into suspend or hibernate state. > > > > So what you are suggesting is to put chronyd daemon into list of broken > > software (which needs to be stopped prior suspend / resume)? > > > > It does not make sense for me as the immediate step after putting > > software or kernel module into such "blacklist" was to inform upstream > > authors of that daemon or kernel module they it is broken / incompatible > > with suspend state and it should be fixed. > > > > That "blacklist" was just workaround for buggy software and not > > permanent solution. > No not chrony, but the machine which change RTC on your back : buggy Bios Sorry, but I have not caught this line. Blacklist contained list of buggy software, daemons and kernel modules which had to be (in past) stopped / unloaded prior system went to S3 and started / (re)loaded after system resumed. So obviously putting "buggy Bios" into blacklist not only does not make sense, but also it did nothing. In that particular case chronyd had to be put into that blacklist of buggy software as it as you described is chronyd which needs to be stopped / started... But as I said this was used in past when buggy software and kernel modules were there when they was not able to correctly handle S3 state. > > > >> Exactly as networkmanager, ifupdown scripts, systemd-networkd > >> reload/restart some network services when interfaces/tunnels/vpn are > >> upped/downed. > > This is something totally different. all those mentioned "services" are > > just independent part of system which manages network connections. > > > > chronyd is there to manage time synchronization. > It was an "imaged comparison" for event driven config change. > The event in the suspend vs time case, the event is only know and > should be managed by your init system not by your time daemon. > > > > > And as I pointed there are existing problems that UEFI/BIOS firmware > > changes RTC clock without good reason which results in completely wrong > > system clock. > > > Could well be identified by blacklist at the udev/systemd level for > applying or not the workaround (restart chrony or launch a chronyc > command at resume) > >>> Could you describe in details what do you mean by blacklist? Which udev > >>> blacklist you mean and what should be put into that blacklist? I have > >>> not caught this part. > >> Faulty systems could be identified by DMI/ACPI strings and quirk applied. > > And what is the faulty system? > Citing yourself : > > "as I pointed there are existing problems that UEFI/BIOS firmware > changes RTC clock without good reason" Ok. Main problem is that there is no way how to identify such broken firmwares. So definition is now nice and clear but basically useless as it does not say anything how to find or identify such faulty systems. > > > > > I think this is something general and not related to particular machine. > > I guess under specific conditions it may happen on any system. > > > >> See for example /lib/udev/hwdb.d/60-sensor.hwdb for some laptop sensors. > >> We could add an attribute to the RTC if it matche some vendor/bios > >> version/model etc... to put in the hwdb (the blacklist) > >> A udev rule will assign this attribute to the RTC if you are running on > >> a known buggy system. > >> A script could do anything you want at suspend/resume time in > >> /lib/systemd/system-sleep if your RTC has the offended attribute (see > >> systemd-sleep man page). > >> Or better, a unit run at resume time could do anything too. > >> The hwdb
Re: [chrony-users] Resume from suspend and default makestep configuration
Le 19/05/2020 à 12:29, Pali Rohár a écrit : > On Monday 18 May 2020 13:45:04 FUSTE Emmanuel wrote: >> Le 18/05/2020 à 13:15, Pali Rohár a écrit : >>> On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: Hello Pali, Le 18/05/2020 à 12:37, Pali Rohár a écrit : > The main problem is when system is put into suspend or hibernate state. > > In my opinion resuming from suspend / hibernate state should be handled > in the same way as (re)starting chronyd. You do not know what may > happened during sleep. Yes and in case of needed workaround, it should be done at the system level, not chrony. A job for systemd. >>> Hello! Sorry for a stupid question, but what has systemd in common with >>> chronyd? Why should systemd care about chronyd time synchronization? >> Nothing. >> But it is to your "process manager" being systemd, sysvinit pile of >> scripts or whatever to restart or notify chrony, it has do do >> housekeeping anyway for other things when you suspend/resume. > Hm... I remember that in past it was needed to blacklist broken daemons, > software and kernel modules which did not work correctly during S3 or > hibernate state. It was in some pm scripts utils... > > But I thought that these days are already passed and software can deal > with fact that machine may be put into suspend or hibernate state. > > So what you are suggesting is to put chronyd daemon into list of broken > software (which needs to be stopped prior suspend / resume)? > > It does not make sense for me as the immediate step after putting > software or kernel module into such "blacklist" was to inform upstream > authors of that daemon or kernel module they it is broken / incompatible > with suspend state and it should be fixed. > > That "blacklist" was just workaround for buggy software and not > permanent solution. No not chrony, but the machine which change RTC on your back : buggy Bios > >> Exactly as networkmanager, ifupdown scripts, systemd-networkd >> reload/restart some network services when interfaces/tunnels/vpn are >> upped/downed. > This is something totally different. all those mentioned "services" are > just independent part of system which manages network connections. > > chronyd is there to manage time synchronization. It was an "imaged comparison" for event driven config change. The event in the suspend vs time case, the event is only know and should be managed by your init system not by your time daemon. > > And as I pointed there are existing problems that UEFI/BIOS firmware > changes RTC clock without good reason which results in completely wrong > system clock. > Could well be identified by blacklist at the udev/systemd level for applying or not the workaround (restart chrony or launch a chronyc command at resume) >>> Could you describe in details what do you mean by blacklist? Which udev >>> blacklist you mean and what should be put into that blacklist? I have >>> not caught this part. >> Faulty systems could be identified by DMI/ACPI strings and quirk applied. > And what is the faulty system? Citing yourself : "as I pointed there are existing problems that UEFI/BIOS firmware changes RTC clock without good reason" > > I think this is something general and not related to particular machine. > I guess under specific conditions it may happen on any system. > >> See for example /lib/udev/hwdb.d/60-sensor.hwdb for some laptop sensors. >> We could add an attribute to the RTC if it matche some vendor/bios >> version/model etc... to put in the hwdb (the blacklist) >> A udev rule will assign this attribute to the RTC if you are running on >> a known buggy system. >> A script could do anything you want at suspend/resume time in >> /lib/systemd/system-sleep if your RTC has the offended attribute (see >> systemd-sleep man page). >> Or better, a unit run at resume time could do anything too. >> The hwdb abstraction is not need if it is a local hack and should be >> properly defined with the hwdb/udev/systemd developers. > This database is for describing hardware differences or issues. > > But above problem with time synchronization is general and hardware > independent. You can simulate same issue on your machine. > > Just put your computer into hibernation. Then boot from liveUSB some > Linxu distribution and change RTC time. Turn off liveUSB and boot your > hibernated system. And you should be in same situation as I described. Yes but this is like shooting yourself in your feet. If you want to be robust in this case and all others, then by default you must restart ANY time sync daemon in the resume callback of your init system, being ntpd or chrony, systemd or sysvinit or upstart or anything else. But it is problematic as Miroslav point out as you potentially start to trust any anonymous time source more than your own RTC. The actual makestep value is a sane default for all the majority of sane machine with standard usecase. For broken
Re: [chrony-users] Resume from suspend and default makestep configuration
On Monday 18 May 2020 13:45:04 FUSTE Emmanuel wrote: > Le 18/05/2020 à 13:15, Pali Rohár a écrit : > > On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: > >> Hello Pali, > >> > >> Le 18/05/2020 à 12:37, Pali Rohár a écrit : > >>> The main problem is when system is put into suspend or hibernate state. > >>> > >>> In my opinion resuming from suspend / hibernate state should be handled > >>> in the same way as (re)starting chronyd. You do not know what may > >>> happened during sleep. > >> Yes and in case of needed workaround, it should be done at the system > >> level, not chrony. > >> A job for systemd. > > Hello! Sorry for a stupid question, but what has systemd in common with > > chronyd? Why should systemd care about chronyd time synchronization? > Nothing. > But it is to your "process manager" being systemd, sysvinit pile of > scripts or whatever to restart or notify chrony, it has do do > housekeeping anyway for other things when you suspend/resume. Hm... I remember that in past it was needed to blacklist broken daemons, software and kernel modules which did not work correctly during S3 or hibernate state. It was in some pm scripts utils... But I thought that these days are already passed and software can deal with fact that machine may be put into suspend or hibernate state. So what you are suggesting is to put chronyd daemon into list of broken software (which needs to be stopped prior suspend / resume)? It does not make sense for me as the immediate step after putting software or kernel module into such "blacklist" was to inform upstream authors of that daemon or kernel module they it is broken / incompatible with suspend state and it should be fixed. That "blacklist" was just workaround for buggy software and not permanent solution. > Exactly as networkmanager, ifupdown scripts, systemd-networkd > reload/restart some network services when interfaces/tunnels/vpn are > upped/downed. This is something totally different. all those mentioned "services" are just independent part of system which manages network connections. chronyd is there to manage time synchronization. > >>> And as I pointed there are existing problems that UEFI/BIOS firmware > >>> changes RTC clock without good reason which results in completely wrong > >>> system clock. > >>> > >> Could well be identified by blacklist at the udev/systemd level for > >> applying or not the workaround (restart chrony or launch a chronyc > >> command at resume) > > Could you describe in details what do you mean by blacklist? Which udev > > blacklist you mean and what should be put into that blacklist? I have > > not caught this part. > Faulty systems could be identified by DMI/ACPI strings and quirk applied. And what is the faulty system? I think this is something general and not related to particular machine. I guess under specific conditions it may happen on any system. > See for example /lib/udev/hwdb.d/60-sensor.hwdb for some laptop sensors. > We could add an attribute to the RTC if it matche some vendor/bios > version/model etc... to put in the hwdb (the blacklist) > A udev rule will assign this attribute to the RTC if you are running on > a known buggy system. > A script could do anything you want at suspend/resume time in > /lib/systemd/system-sleep if your RTC has the offended attribute (see > systemd-sleep man page). > Or better, a unit run at resume time could do anything too. > The hwdb abstraction is not need if it is a local hack and should be > properly defined with the hwdb/udev/systemd developers. This database is for describing hardware differences or issues. But above problem with time synchronization is general and hardware independent. You can simulate same issue on your machine. Just put your computer into hibernation. Then boot from liveUSB some Linxu distribution and change RTC time. Turn off liveUSB and boot your hibernated system. And you should be in same situation as I described. > If raised to the systemd developers, systemd-sleep / resume could take > care directly and fire an appropriate target with a formally defined > attribute in the hwdb. > What to do with this target could be configurable and default to time > daemon restart. > I'm not a systemd/udev/hwdb expert/develloper, but I think this is a > good track and deserve a discussion with them. > > Anyway, the level to tackle the problem is not chrony and the proper > level for managing the problem is the init/process manager. Hwdb/udev is > "a" way to share the faulty systems information across "init" ecosystem. > Information that is usefully not only for chrony. > > Emmanuel. -- Pali Rohár pali.ro...@gmail.com -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Resume from suspend and default makestep configuration
Le 18/05/2020 à 13:15, Pali Rohár a écrit : > On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: >> Hello Pali, >> >> Le 18/05/2020 à 12:37, Pali Rohár a écrit : >>> The main problem is when system is put into suspend or hibernate state. >>> >>> In my opinion resuming from suspend / hibernate state should be handled >>> in the same way as (re)starting chronyd. You do not know what may >>> happened during sleep. >> Yes and in case of needed workaround, it should be done at the system >> level, not chrony. >> A job for systemd. > Hello! Sorry for a stupid question, but what has systemd in common with > chronyd? Why should systemd care about chronyd time synchronization? Nothing. But it is to your "process manager" being systemd, sysvinit pile of scripts or whatever to restart or notify chrony, it has do do housekeeping anyway for other things when you suspend/resume. Exactly as networkmanager, ifupdown scripts, systemd-networkd reload/restart some network services when interfaces/tunnels/vpn are upped/downed. >>> And as I pointed there are existing problems that UEFI/BIOS firmware >>> changes RTC clock without good reason which results in completely wrong >>> system clock. >>> >> Could well be identified by blacklist at the udev/systemd level for >> applying or not the workaround (restart chrony or launch a chronyc >> command at resume) > Could you describe in details what do you mean by blacklist? Which udev > blacklist you mean and what should be put into that blacklist? I have > not caught this part. Faulty systems could be identified by DMI/ACPI strings and quirk applied. See for example /lib/udev/hwdb.d/60-sensor.hwdb for some laptop sensors. We could add an attribute to the RTC if it matche some vendor/bios version/model etc... to put in the hwdb (the blacklist) A udev rule will assign this attribute to the RTC if you are running on a known buggy system. A script could do anything you want at suspend/resume time in /lib/systemd/system-sleep if your RTC has the offended attribute (see systemd-sleep man page). Or better, a unit run at resume time could do anything too. The hwdb abstraction is not need if it is a local hack and should be properly defined with the hwdb/udev/systemd developers. If raised to the systemd developers, systemd-sleep / resume could take care directly and fire an appropriate target with a formally defined attribute in the hwdb. What to do with this target could be configurable and default to time daemon restart. I'm not a systemd/udev/hwdb expert/develloper, but I think this is a good track and deserve a discussion with them. Anyway, the level to tackle the problem is not chrony and the proper level for managing the problem is the init/process manager. Hwdb/udev is "a" way to share the faulty systems information across "init" ecosystem. Information that is usefully not only for chrony. Emmanuel.
Re: [chrony-users] Resume from suspend and default makestep configuration
On Monday 18 May 2020 10:45:02 FUSTE Emmanuel wrote: > Hello Pali, > > Le 18/05/2020 à 12:37, Pali Rohár a écrit : > > The main problem is when system is put into suspend or hibernate state. > > > > In my opinion resuming from suspend / hibernate state should be handled > > in the same way as (re)starting chronyd. You do not know what may > > happened during sleep. > Yes and in case of needed workaround, it should be done at the system > level, not chrony. > A job for systemd. Hello! Sorry for a stupid question, but what has systemd in common with chronyd? Why should systemd care about chronyd time synchronization? > > And as I pointed there are existing problems that UEFI/BIOS firmware > > changes RTC clock without good reason which results in completely wrong > > system clock. > > > Could well be identified by blacklist at the udev/systemd level for > applying or not the workaround (restart chrony or launch a chronyc > command at resume) Could you describe in details what do you mean by blacklist? Which udev blacklist you mean and what should be put into that blacklist? I have not caught this part. -- Pali Rohár pali.ro...@gmail.com -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Resume from suspend and default makestep configuration
Hello Pali, Le 18/05/2020 à 12:37, Pali Rohár a écrit : > The main problem is when system is put into suspend or hibernate state. > > In my opinion resuming from suspend / hibernate state should be handled > in the same way as (re)starting chronyd. You do not know what may > happened during sleep. Yes and in case of needed workaround, it should be done at the system level, not chrony. A job for systemd. > And as I pointed there are existing problems that UEFI/BIOS firmware > changes RTC clock without good reason which results in completely wrong > system clock. > Could well be identified by blacklist at the udev/systemd level for applying or not the workaround (restart chrony or launch a chronyc command at resume) Emmanuel.
Re: [chrony-users] Resume from suspend and default makestep configuration
On Monday 11 May 2020 10:52:49 Miroslav Lichvar wrote: > On Sat, May 09, 2020 at 01:30:29AM +0200, Pali Rohár wrote: > > I would suggest to either change default configuration to 'makestep 1 -1' > > or assuming that after resuming from suspend / hibernate, chrony should > > behave like it was restarted. And therefore suck forward jump detection > > done by clock update would be treated as in first clock update -- > > slewing would not be used and instead clock would jump. > > > > What do you think about it? It is possible to change it? So default > > chrony configuration would be suitable also for desktop / laptop users? > > By default, chronyd doesn't make any steps, except for a leap second > if not supported by the system. Most distributions have a default > config that allows a small number of steps, possibly based on one of > the provided examples. I'd not recommend changing that to unlimited > number of steps as the vast majority of computers don't need that. > It should be enabled only in specific cases when really needed and the > implications are understood. > > One issue with allowing steps at any time is that it may break > applications that don't handle backward steps. Another issue is that > it allows a MITM attacker to inject arbitrary offsets to the clock at > any time. With a limited makestep that window is limited to a short > time after the boot, or package upgrade. When I take my laptop to an > untrusted network, I don't want people there to be able to step my > clock 50 years ahead to break TLS certificates for example. Ideally, > when not using authentication, no steps should ever be allowed. For a > default configuration that would probably be unreasonable. Hello Miroslav! I understand your security concern and I agree that during time when system is running, that unlimited number of steps is not needed. The main problem is when system is put into suspend or hibernate state. In my opinion resuming from suspend / hibernate state should be handled in the same way as (re)starting chronyd. You do not know what may happened during sleep. And as I pointed there are existing problems that UEFI/BIOS firmware changes RTC clock without good reason which results in completely wrong system clock. -- Pali Rohár pali.ro...@gmail.com -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Resume from suspend and default makestep configuration
On Sat, May 09, 2020 at 01:30:29AM +0200, Pali Rohár wrote: > I would suggest to either change default configuration to 'makestep 1 -1' > or assuming that after resuming from suspend / hibernate, chrony should > behave like it was restarted. And therefore suck forward jump detection > done by clock update would be treated as in first clock update -- > slewing would not be used and instead clock would jump. > > What do you think about it? It is possible to change it? So default > chrony configuration would be suitable also for desktop / laptop users? By default, chronyd doesn't make any steps, except for a leap second if not supported by the system. Most distributions have a default config that allows a small number of steps, possibly based on one of the provided examples. I'd not recommend changing that to unlimited number of steps as the vast majority of computers don't need that. It should be enabled only in specific cases when really needed and the implications are understood. One issue with allowing steps at any time is that it may break applications that don't handle backward steps. Another issue is that it allows a MITM attacker to inject arbitrary offsets to the clock at any time. With a limited makestep that window is limited to a short time after the boot, or package upgrade. When I take my laptop to an untrusted network, I don't want people there to be able to step my clock 50 years ahead to break TLS certificates for example. Ideally, when not using authentication, no steps should ever be allowed. For a default configuration that would probably be unreasonable. -- Miroslav Lichvar -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
