RE: Leap second tonight
Not being a time geek, since Cisco's were called out for being wild jitter-mongers... how much jitter are we talking about? Clock is synchronized, stratum 2, nominal freq is 250. Hz, actual freq is 249.9989 Hz, precision is 2**18 reference time is CD6A7CD4.45A9BB00 (19:47:32.272 UTC Tue Mar 17 2009) clock offset is 2.0581 msec, root delay is 29.62 msec root dispersion is 6.81 msec, peer dispersion is 3.30 msec Are we talking about +/- 30 seconds, or a problem bounded by +/- 30 msec? I've actually been gathering some statistics on this using Munin (http://munin.projects.linpro.no/) on my linux server. There's currently 10 ntp servers being monitored and one of them is a 7600-series Cisco, which is handling quite a bit of traffic (CPU load around 20%). Here are the Munin graphs for it http://dx.fi/alt/ntp/7600.png (times in Finnish time, UTC+2). In comparison, here are the same graphs for time1.mikes.fi (a stratum-2 clock provided by the Finnish Centre for metrology and accreditation) http://dx.fi/alt/ntp/time1.mikes.fi.png and for Netnods stratum-1 clock in Stockholm http://dx.fi/alt/ntp/ntp1.sth.netnod.se.png Best regards, -- Tero Toikkanen Nebula Oy
Re: Leap second tonight
On Dec 31, 2008, at 15:28, Kevin Oberman wrote: We use CDMA clocks and last leap second it took weeks for all of the cell sites to adjust the last one. As a result, I have set all of our clocks for manual leap second and set them to adjust tonight at midnight (UTC).I'll take a look in about 35 minutes and see how it worked. Chiming in a little late here ... Over at the NTP Pool we had about 9% of the servers not handle the leap second accurately; starting at midnight UTC. After an hour (so between 01:00 and 02:00) it was down to about 3%; a couple hours later down to about 1% of our servers (a few dozen)[1]. Most of those got in order within 24-48 hours.Interestingly the few who didn't get corrected within a few days were, tada: CDMA clocks. To stay vaguely NANOG on-topic: I believe at least some of our ~1700 NTP servers are routers; so I'm guessing they handled the leap second alright. Sounds like a RISKS lesson: Don't use side-effects of a tool for something critical. (If I understand it right then CDMA uses accurate time because it needs accurate frequency; not because it cares what time it is). Also: Who came up with having the leap second on New Year!? Clearly not someone with any operational experience. - ask [1] http://fortytwo.ch/mailman/pipermail/timekeepers/2009/004619.html and http://fortytwo.ch/mailman/pipermail/timekeepers/2009/004623.html -- http://develooper.com/ - http://askask.com/
Re: Leap second tonight
On Tue, Mar 17, 2009 at 1:07 AM, Ask Bjørn Hansen a...@develooper.comwrote: On Dec 31, 2008, at 15:28, Kevin Oberman wrote: We use CDMA clocks and last leap second it took weeks for all of the cell sites to adjust the last one. As a result, I have set all of our clocks for manual leap second and set them to adjust tonight at midnight (UTC).I'll take a look in about 35 minutes and see how it worked. Chiming in a little late here ... Oh, quiet. After all, what's 6.5 million seconds or so between friends? -j -- Jamie Rishaw // .com.a...@j - reverse it. ish. [Impressive C-level Title Here], arpa / arpa labs
Re: Leap second tonight
From: =?ISO-8859-1?Q?Ask_Bj=F8rn_Hansen?= a...@develooper.com Date: Mon, 16 Mar 2009 23:07:42 -0700 On Dec 31, 2008, at 15:28, Kevin Oberman wrote: We use CDMA clocks and last leap second it took weeks for all of the cell sites to adjust the last one. As a result, I have set all of our clocks for manual leap second and set them to adjust tonight at midnight (UTC).I'll take a look in about 35 minutes and see how it worked. Chiming in a little late here ... Over at the NTP Pool we had about 9% of the servers not handle the leap second accurately; starting at midnight UTC. After an hour (so between 01:00 and 02:00) it was down to about 3%; a couple hours later down to about 1% of our servers (a few dozen)[1]. Most of those got in order within 24-48 hours.Interestingly the few who didn't get corrected within a few days were, tada: CDMA clocks. To stay vaguely NANOG on-topic: I believe at least some of our ~1700 NTP servers are routers; so I'm guessing they handled the leap second alright. Routers as ntp servers. Yuck! Routers route well, but they treat time as a low priority job and jitter on Cisco routers is simply terrible. Junipers do better, but are still a poor time server. Sounds like a RISKS lesson: Don't use side-effects of a tool for something critical. (If I understand it right then CDMA uses accurate time because it needs accurate frequency; not because it cares what time it is). As I understand it, they need both time and frequency to do cell hand-offs cleanly, but, as long as all towers in a carrier's market are showing the same time, it really does not matter if they do the leap second. Endrun Technologies, who make our clocks, ship them configured for manual leap seconds because so many cell operators are pretty casual about the leap second thing, but that means that the people using the clocks need to be aware that they need to be told when a leap second is coming and that, in turn, means the they must know a bit about leap seconds and must have read the manual. No surprise that a lot of CDMA clocks missed the leap second.
Re: Leap second tonight
On Tue, 17 Mar 2009 08:06:51 PDT, Kevin Oberman said: Routers as ntp servers. Yuck! Routers route well, but they treat time as a low priority job and jitter on Cisco routers is simply terrible. Junipers do better, but are still a poor time server. They may suck for being a Stratum-1/2 server, but even the most jittery Cisco is still far and away good enough to serve up a ntpdate so that an end-user PC-class machine is in the right minute. pgpZCAIpOHmB3.pgp Description: PGP signature
Re: Leap second tonight
On Tue, 17 Mar 2009, valdis.kletni...@vt.edu wrote: They may suck for being a Stratum-1/2 server, but even the most jittery Cisco is still far and away good enough to serve up a ntpdate so that an end-user PC-class machine is in the right minute. As long as the end-user is made aware that the accuracy of said NTP clock is +/- 30.000 seconds (or whatever jitter might exist). Seems kind of ridiculous to use an NTP source that is, for many purposes, wildly inaccurate. For my purposes, wildly is more than +/- 0.1 seconds. Trying to troubleshoot a problem, network or server, where the timestamps on each server/router/device vary inconsistently, is like walking on broken fluorescent bulbs -- painful and dangerous to one's health. --- Peter Beckman Internet Guy beck...@angryox.com http://www.angryox.com/ ---
RE: Leap second tonight
As long as the end-user is made aware that the accuracy of said NTP clock is +/- 30.000 seconds (or whatever jitter might exist). Seems kind of ridiculous to use an NTP source that is, for many purposes, wildly inaccurate. For my purposes, wildly is more than +/- 0.1 seconds. Trying to troubleshoot a problem, network or server, where the timestamps on each server/router/device vary inconsistently, is like walking on broken fluorescent bulbs -- painful and dangerous to one's health. Not being a time geek, since Cisco's were called out for being wild jitter-mongers... how much jitter are we talking about? Clock is synchronized, stratum 2, nominal freq is 250. Hz, actual freq is 249.9989 Hz, precision is 2**18 reference time is CD6A7CD4.45A9BB00 (19:47:32.272 UTC Tue Mar 17 2009) clock offset is 2.0581 msec, root delay is 29.62 msec root dispersion is 6.81 msec, peer dispersion is 3.30 msec Are we talking about +/- 30 seconds, or a problem bounded by +/- 30 msec? Deepak Jain AiNET
RE: Leap second tonight
A report from a DHCP/DNS appliance vendor here: Several customers have reported a complete lock-up of their Proteus system around the beginning of January 1st 2009. We believe that we have traced this to a problem in the underlying kernel and NTP and the handling of the date change associated with 2008 being a Leap Year and therefore having 366 days. Several conditions must be met to trigger this problem: 1. The Proteus was originally installed as v2.1.x or earlier. 2. NTP is enabled as a client with 2 or more external source servers defined. 3. There is a discrepancy in the times reported back by these other NTP servers. There is no correction available at this time, and the resolution is to power cycle the system, after which it will run fine. If you experienced a similar problem at the indicated time, please submit a trouble ticket so that we can confirm that this occurred on your system. I don't know what the underlying OS is. Frank -Original Message- From: Kevin Day [mailto:toa...@dragondata.com] Sent: Wednesday, December 31, 2008 4:42 PM To: NANOG list Subject: Leap second tonight Just a reminder that there's a leap second tonight. Last time I watched for what happened on 01/01/2006, there was a little bit of chaos: http://markmail.org/message/cpoj3jw5onzhhjkr?q=%22kevin+day%22+leap+second+r eminder+nanogpage=1refer=cnkxb3iv7sls5axu I've been told that some of the causes of these problems are fixed on any reasonably recent ntp distribution, but just in case, you might wanna keep an eye out if you're seeing any weirdness. The worst damage I'd heard from anyone after that event was their clock being significantly off for several hours. -- Kevin
Re: Leap second tonight
On Jan 5, 2009, at 11:30 AM, Adrian Chadd wrote: This begs the question - how the heck do timekeepers and politicians get away with last minute time changes? Surely there's -some- pushback from technology related interest groups to try and get more than four weeks warning? :) Having been involved in the leap second business, I can tell you that Daniel Gambis strictly follows the rules, which are Bulletin C is mailed every six months, either to announce a time step in UTC or to confirm that there will be no time step at the next possible date. If you want more lead time warning, pay attention to the LOD graph in http://hpiers.obspm.fr/eop-pc/ The long term LOD offset is about 1 msec now. That means that every day, Earth time and atomic time will drift off by 1 msec. Since there are 1000 msec in the second, and since the rule is that a leap second is chosen when the difference (UT1−UTC) approaches 0.9 seconds, projected out to the next period, and since the strong preference is to have leap seconds in January, you can generally figure out what will happen before Daniel announces it. For example, in one year the offset should be ~ 400 msec, so I will informally predict another leap second in January, 2011, not 2010. Keep watching that graph. Anyone who is dealing with Leap Second code should keep in mind that negative leap seconds (i.e., no second # 59, instead of an extra second called 60) are a distinct possibility. It all depends on the weather at the core mantle boundary - note that the LOD offset was almost 3 msec not too long ago. Regards Marshall Adrian On Mon, Jan 05, 2009, Frank Bulk wrote: A report from a DHCP/DNS appliance vendor here: Several customers have reported a complete lock-up of their Proteus system around the beginning of January 1st 2009. We believe that we have traced this to a problem in the underlying kernel and NTP and the handling of the date change associated with 2008 being a Leap Year and therefore having 366 days. Several conditions must be met to trigger this problem: 1. The Proteus was originally installed as v2.1.x or earlier. 2. NTP is enabled as a client with 2 or more external source servers defined. 3. There is a discrepancy in the times reported back by these other NTP servers. There is no correction available at this time, and the resolution is to power cycle the system, after which it will run fine. If you experienced a similar problem at the indicated time, please submit a trouble ticket so that we can confirm that this occurred on your system. I don't know what the underlying OS is. Frank -Original Message- From: Kevin Day [mailto:toa...@dragondata.com] Sent: Wednesday, December 31, 2008 4:42 PM To: NANOG list Subject: Leap second tonight Just a reminder that there's a leap second tonight. Last time I watched for what happened on 01/01/2006, there was a little bit of chaos: http://markmail.org/message/cpoj3jw5onzhhjkr?q=%22kevin+day%22+leap+second+r eminder+nanogpage=1refer=cnkxb3iv7sls5axu I've been told that some of the causes of these problems are fixed on any reasonably recent ntp distribution, but just in case, you might wanna keep an eye out if you're seeing any weirdness. The worst damage I'd heard from anyone after that event was their clock being significantly off for several hours. -- Kevin -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: Leap second tonight
On 05/01/2009 6:01, Nick Hilliard n...@foobar.org wrote: [...] But seriously. Leap seconds occur every couple of years, either on July 30th and Dec 31. Sometimes both. And sometimes every consecutive year for a couple of years on the run. It's theoretically possible for leap seconds to be introduced at the end of March and September. It's never happened but it might, so I suppose software should allow for the possibility. Regards, Leo
Re: Leap second tonight
Adrian Chadd wrote: This begs the question - how the heck do timekeepers and politicians get away with last minute time changes? Surely there's -some- pushback from technology related interest groups to try and get more than four weeks warning? :) Adrian The first notice I found was dated July 4th 2008 http://tycho.usno.navy.mil/bulletinc2008.html
Re: Leap second tonight
On Tue, Jan 06, 2009 at 01:30:51AM +0900, Adrian Chadd wrote: This begs the question - how the heck do timekeepers and politicians get away with last minute time changes? Surely there's -some- pushback from technology related interest groups to try and get more than four weeks warning? :) Try six months. NTP itself sets the leap indicator by 28 days prior to the leap and clears it before the end of the following day, so in theory the appliance itself had at least 4 weeks notice and the rest of us had an additional five months. IERS announces a pending leap second six months in advance. The announcement for this one was dated July 4th. System vendors have only had 37 years since the first leap second to figure this out; please be patient. However, I can't excuse them for bugs surrounding the final day of a leap year. The Julian calendar is not exactly a new phenomenon. --msa
RE: Leap second tonight
It's theoretically possible for leap seconds to be introduced at the end of March and September. As I recall, NTP supports leap seconds every month, for which there is a prediction that even this would be insufficient at some point in this millennium (depending, of course, on the actual rotation speed). There have been on again/off again talks to abolish the leap second for quite a number of years. Gary
Re: Leap second tonight
On Mon, Jan 05, 2009, Nick Hilliard wrote: Notice for the leap second was issued on July 4 2008. http://hpiers.obspm.fr/iers/bul/bulc/bulletinc.36 Wow, how'd I miss that, I wonder? :) I'm just angry at the jack moves pulled by last minute timezone changes back in Australia, and the massively stupid repercussions seen throughout chunks of IT (incl. network auditing setups I had to poke at the time.) I'll add handling second == 60 to the list of things I should check for in my code. Thanks. :) Adrian
Re: Leap second tonight
This begs the question - how the heck do timekeepers and politicians get away with last minute time changes? Surely there's -some- pushback from technology related interest groups to try and get more than four weeks warning? :) Adrian On Mon, Jan 05, 2009, Frank Bulk wrote: A report from a DHCP/DNS appliance vendor here: Several customers have reported a complete lock-up of their Proteus system around the beginning of January 1st 2009. We believe that we have traced this to a problem in the underlying kernel and NTP and the handling of the date change associated with 2008 being a Leap Year and therefore having 366 days. Several conditions must be met to trigger this problem: 1. The Proteus was originally installed as v2.1.x or earlier. 2. NTP is enabled as a client with 2 or more external source servers defined. 3. There is a discrepancy in the times reported back by these other NTP servers. There is no correction available at this time, and the resolution is to power cycle the system, after which it will run fine. If you experienced a similar problem at the indicated time, please submit a trouble ticket so that we can confirm that this occurred on your system. I don't know what the underlying OS is. Frank -Original Message- From: Kevin Day [mailto:toa...@dragondata.com] Sent: Wednesday, December 31, 2008 4:42 PM To: NANOG list Subject: Leap second tonight Just a reminder that there's a leap second tonight. Last time I watched for what happened on 01/01/2006, there was a little bit of chaos: http://markmail.org/message/cpoj3jw5onzhhjkr?q=%22kevin+day%22+leap+second+r eminder+nanogpage=1refer=cnkxb3iv7sls5axu I've been told that some of the causes of these problems are fixed on any reasonably recent ntp distribution, but just in case, you might wanna keep an eye out if you're seeing any weirdness. The worst damage I'd heard from anyone after that event was their clock being significantly off for several hours. -- Kevin -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: Leap second tonight
Adrian Chadd wrote: This begs the question - how the heck do timekeepers and politicians get away with last minute time changes? Surely there's -some- pushback from technology related interest groups to try and get more than four weeks warning? :) ? Notice for the leap second was issued on July 4 2008. http://hpiers.obspm.fr/iers/bul/bulc/bulletinc.36 Nick
Re: Leap second tonight
Adrian Chadd wrote: Wow, how'd I miss that, I wonder? :) I would recommend lodging a complaint to the relevant authorities. That's sure to help. But seriously. Leap seconds occur every couple of years, either on July 30th and Dec 31. Sometimes both. And sometimes every consecutive year for a couple of years on the run. If your code insists on exactly 60 seconds in all minutes, or 86400 seconds in a hour, your code is wrong. You need to take this into account, because leap seconds are actually pretty common occurrences. I'm just angry at the jack moves pulled by last minute timezone changes back in Australia, and the massively stupid repercussions seen throughout chunks of IT (incl. network auditing setups I had to poke at the time.) I'll add handling second == 60 to the list of things I should check for in my code. Thanks. :) Yeah, last minute declarations are annoying. Like the british government's mad-cap idea to change VAT for the first time in donkey's years from 17.5% to 15% with 7 days warning. Now that's screwed up. Nick
Re: Leap second tonight
I've gleened from this thread that: * everyone uses UTC, or should, because UTC is a uniform time scale, except for those leap seconds * UTC is sourced from the frequence of a radio emission from cesium atoms which are extremely constant * UTC can get out of whack with the rotation of the earth around the sun, because our rotation is not uniform, but is calculated rather than measured (well, sort of) * UTC is TAI plus leap seconds. In 1972, when leap seconds were first introduced, UTC was TAI - 10 seconds. UTC is now TAI - 34 seconds. TAI ticks exactly as fast as UTC, ignoring leap second adjustments. * UTC is slower than UT1 by about 1ms per day. * On 12/31/2008 UTC was (-) 591.8ms behind UT1. On 1/1/2008 UTC was 407.1ms ahead of UT1. * Leap seconds are applied to UTC every few years to remain in line with UT1, the time based on the rotation of the earth around the sun. * GMT is used to imply UT1, but sometimes UTC, but really GMT is just massively confusing and you shouldn't use it, either in conversation or in your servers/routers, because nobody is really sure without reading a lot of documentation what GMT means for each manufacturer/OS/software. * When writing code regarding dates and times, know that any year may have 366 days, and any minute may have 61 seconds. * When in doubt, Dr. Daniel Gambis is always right. Beckman On Mon, 5 Jan 2009, Marshall Eubanks wrote: Having been involved in the leap second business, I can tell you that Daniel Gambis strictly follows the rules, which are Bulletin C is mailed every six months, either to announce a time step in UTC or to confirm that there will be no time step at the next possible date. --- Peter Beckman Internet Guy beck...@angryox.com http://www.angryox.com/ ---
Re: Leap second tonight
On 1/5/2009 at 1:19 PM, Peter Beckman beck...@angryox.com wrote: I've gleened from this thread that: * everyone uses UTC, or should, because UTC is a uniform time scale, except for those leap seconds Local time is totally appropriate in some circumstances, but it is pretty much always defined as just an offset to UTC. UT1 is important when you are doing astronomical observations or depend on such things. * UTC is sourced from the frequence of a radio emission from cesium atoms which are extremely constant There is nothing too special about cesium. It's properties and the properties of the particular transition are convenient from an engineering standpoint. * UTC can get out of whack with the rotation of the earth around the sun, because our rotation is not uniform, but is calculated rather than measured (well, sort of) No its not about years, that's what February 29th is for, it's about days. As the Earth rotates (there's a rotation versus revolution nomenclature), the sun appears to move around it daily. This is the solar day. At a certain point, at a certain location, the sun is at the highest it will be in the sky for that rotation. This is solar noon. The time between two consecutive noons at any location is not constant mostly due to the Earth's orbit being elliptical and the tilt of the Earth's axis. But for any place on Earth, the mean solar day, the average of the solar day over the year, is the same. If the Earth was a solid sphere rotating at a constant speed, that would be the end of the story. But it's not and for a variety of reasons, the rotation of the Earth changes with time (mostly slows). This causes the mean solar day to get longer. * UTC is TAI plus leap seconds. In 1972, when leap seconds were first introduced, UTC was TAI - 10 seconds. UTC is now TAI - 34 seconds. TAI ticks exactly as fast as UTC, ignoring leap second adjustments. * UTC is slower than UT1 by about 1ms per day. That's a very tricky sentence. I think that the mean solar day right now is about 86400.002 s long. The mean solar day was actually exactly 86400 s long sometime around 1820. * On 12/31/2008 UTC was (-) 591.8ms behind UT1. On 1/1/2008 UTC was 407.1ms ahead of UT1. * Leap seconds are applied to UTC every few years to remain in line with UT1, the time based on the rotation of the earth around the sun. A time based on the rotation of the Earth.(period) As mentioned above, it doesn't really have anything directly to do with Earth's location in its orbit. * GMT is used to imply UT1, but sometimes UTC, but really GMT is just massively confusing and you shouldn't use it, either in conversation or in your servers/routers, because nobody is really sure without reading a lot of documentation what GMT means for each manufacturer/OS/software. * When writing code regarding dates and times, know that any year may have 366 days, and any minute may have 61 seconds. Also remember that an administrator or automated clock recalibration may take you forward or back in time any arbitrary amount. * When in doubt, Dr. Daniel Gambis is always right. Very, very few should not defer to his judgement on such matters.
Re: Leap second tonight
Peter Beckman wrote: * GMT is used to imply UT1, but sometimes UTC, but really GMT is just massively confusing and you shouldn't use it, either in conversation or in your servers/routers, because nobody is really sure without reading a lot of documentation what GMT means for each manufacturer/OS/software. WET/WEST is a little more precise and less confusing than GMT/BST (or IST if you're of a more celtic nature, although this confuses people living on the Indian subcontinent) although in tz-land, GMT has a specific and pretty consistent definition. But it is generally a bad idea to use it. People get confused. * When writing code regarding dates and times, know that any year may have 366 days, and any minute may have 61 seconds. Or 59 seconds in the case of a negative leap second. Nick
Re: Leap second tonight
On Mon, Jan 5, 2009 at 4:19 PM, Peter Beckman beck...@angryox.com wrote: * UTC can get out of whack with the rotation of the earth around the sun, because our rotation is not uniform, but is calculated rather than measured (well, sort of) As Crist Clark points out, leap seconds are about the Earth's rotation about its own axis, not its revolution (orbit) around the sun. Atomic clocks are much more consistent and uniform than the Earth's rotation. If we don't correct for the differences somehow, eventually wall clock time would get out of sync with the day/night cycle. In $LARGE_NUMBER years, 1:00 PM would be in the middle of the night. Earth's orbit and the Julian calendar have the same problem, which is why we have leap days. Otherwise, the calendar would get out of sync with the seasons. * When writing code regarding dates and times, know that any year may have 366 days, and any minute may have 61 seconds. I wouldn't even define it that narrowly. We might end up with 62 or 63 or 57 seconds in a minute, or 364 or 367 days in a year. Internally, store everything using a fixed-unit offset (e.g., traditional Unix time, i.e., seconds from 1 Jan 1970). Use OS provided facilities to translate to and from human-friendly representations, and thus make it the OS's problem. (If the OS is your problem... sucks to be you. You'll need lots of tables of historic, idiosyncratic clock/calendar changes to get it right.) For program timers (timeouts, etc.), don't use wall clock time at all, since the wall clock may be wrong, and the admin may fix it, yielding time travel. Most OSes provide something like a seconds since boot value, which should always monotonically increase (unless you're running Windows 95, heh), regardless of what the admin is doing to confuse matters. From the other side of the coin: As an admin, avoid advancing the wall clock in large steps, or going backwards at all. If you must do either, do it in single-user mode, or whatever your platform's equivalent is. Don't assume the programmers got it right. Another programmer tip: When storing dates and times in a file, database, etc., and you have to care about multiple timezones: Store at least three of UTC, local time, calculated difference, logical local timezone. The extra information lets one figure out after-the-fact what the timezone tables on the system said when the user entered the information. When the gov'mint monkeys with the time zones, there's always a lag between official announcement and local implementation. Without knowing what the time zone tables said, it can be hard to know if you should apply a correction later. -- Ben
Re: Leap second tonight
On Thu, 1 Jan 2009, Simon Lockhart wrote:
My Oracle boxes that rebooted were running RAC (version 10G R2), too. Another
Solaris 10 box running the same version of Oracle, but not RAC, did not reboot.
Looks rather like an Oracle 10 RAC bug.
It's a known bug in Oracle 10. When the time is set backwards the system
reboots. I don't have the bug id at hand but there is one, and a patch.
Either patch or don't run NTP on Oracle servers.
-
typedef struct me_s {
char name[] = { Thomas Habets };
char email[] = { tho...@habets.pp.se };
char kernel[]= { Linux };
char *pgpKey[] = { http://www.habets.pp.se/pubkey.txt; };
char pgp[] = { A8A3 D1DD 4AE0 8467 7FDE 0945 286A E90A AD48 E854 };
char coolcmd[] = { echo '. ./_. ./_'_;. ./_ };
} me_t;
Re: Leap second tonight
On Wed Dec 31, 2008 at 04:53:57PM -0800, Wil Schultz wrote: At which point my Solaris 10 v490's reboot in unison, lovely. Anyone else see anything interesting? I had a couple of Oracle servers (Solaris 10) reboot a couple of minutes just before the leap second. All my other Solaris 10 boxes appear to have stayed up fine. Simon
Re: Leap second tonight
On Thu, Jan 1, 2009 at 04:15, Simon Lockhart si...@slimey.org wrote: On Wed Dec 31, 2008 at 04:53:57PM -0800, Wil Schultz wrote: At which point my Solaris 10 v490's reboot in unison, lovely. Anyone else see anything interesting? I had a couple of Oracle servers (Solaris 10) reboot a couple of minutes just before the leap second. All my other Solaris 10 boxes appear to have stayed up fine. Have either of you determined if this was a OS reboot and not a bios reset? -Jim P.
Re: Leap second tonight
On Thu Jan 01, 2009 at 04:29:35AM -0500, Jim Popovitch wrote: Have either of you determined if this was a OS reboot and not a bios reset? I've been trawling through all the logfiles I can find on the box, and I see normal entries up until 23:59:xx, and then the next entry is stuff restarting. Could well be a BIOS/LOM reset, but it's odd that the only two boxes affected were my Oracle servers. Simon
Re: Leap second tonight
All of my Solaris 10 boxes stayed up with the exception of the Oracle 10g RAC boxes. db1:~ wschultz$ uname -a SunOS db1 5.10 Generic_137111-01 sun4u sparc SUNW,Sun-Fire-V490 A friend of mine had his RAC boxes reboot as well, similar configuration. I've poured through the logs and see normal activity until the reboot, nothing suspicious and no reason for the reboot. Seems to be specific to Solaris 10 running RAC on this end. -wil On Jan 1, 2009, at 2:36 AM, Simon Lockhart wrote: On Thu Jan 01, 2009 at 04:29:35AM -0500, Jim Popovitch wrote: Have either of you determined if this was a OS reboot and not a bios reset? I've been trawling through all the logfiles I can find on the box, and I see normal entries up until 23:59:xx, and then the next entry is stuff restarting. Could well be a BIOS/LOM reset, but it's odd that the only two boxes affected were my Oracle servers. Simon
Re: Leap second tonight
On Thu Jan 01, 2009 at 07:58:21AM -0800, Wil Schultz wrote: All of my Solaris 10 boxes stayed up with the exception of the Oracle 10g RAC boxes. My Oracle boxes that rebooted were running RAC (version 10G R2), too. Another Solaris 10 box running the same version of Oracle, but not RAC, did not reboot. Looks rather like an Oracle 10 RAC bug. Simon
Re: Leap second tonight
What FS do you have on your RAC? And do you have a log of console? We do not have 10g R2 on Solaris, but with the random issues we have seen. Oracle could have fenced each other off. We noticed that some errors do not show up anywhere than console, so we run conserver and dump all console out to log. --Original Message-- From: Simon Lockhart Sender: To: Wil Schultz Cc: NANOG list Sent: Jan 1, 2009 8:13 AM Subject: Re: Leap second tonight On Thu Jan 01, 2009 at 07:58:21AM -0800, Wil Schultz wrote: All of my Solaris 10 boxes stayed up with the exception of the Oracle 10g RAC boxes. My Oracle boxes that rebooted were running RAC (version 10G R2), too. Another Solaris 10 box running the same version of Oracle, but not RAC, did not reboot. Looks rather like an Oracle 10 RAC bug. Simon
Re: Leap second tonight
Jon Meek wrote: My Solaris 10 boxes are all happy (and did not reboot). I monitor NTP on a number of devices, including one router. The router was off by one second for a while, but is OK after an hour. Everything else was fine immediately. In 2005, our CDMA clock got the leap second between 15:08 and 15:38 EST creating some issues due to disagreement with the (too few) GPS clocks. Jon On Wed, Dec 31, 2008 at 7:53 PM, Wil Schultz wschu...@bsdboy.com wrote: At which point my Solaris 10 v490's reboot in unison, lovely. Anyone else see anything interesting? -wil I run a bunch of Slackware Linux boxes of varying versions. As best as I can tell, at or around 00:00 UTC all of my Slackware 12.0 boxes crashed with a kernel panic. I don't think it is ntpd because it is the same version as on 12.1 boxes (4.2.4p0) that did not crash. It may be the kernel: 2.6.21.5 Anyone else experience similar or was this coincidental and I have other issues... Steve -- -- Steven Saner ssa...@hubris.net Voice: 316-858-3000 Director of Network Operations Fax: 316-858-3001 Hubris Communicationshttp://www.hubris.net
Re: Leap second tonight
Once upon a time, Steven Saner ssa...@hubris.net said: I run a bunch of Slackware Linux boxes of varying versions. As best as I can tell, at or around 00:00 UTC all of my Slackware 12.0 boxes crashed with a kernel panic. I don't think it is ntpd because it is the same version as on 12.1 boxes (4.2.4p0) that did not crash. It may be the kernel: 2.6.21.5 Anyone else experience similar or was this coincidental and I have other issues... I had one (out of many, including about a half dozen identically configured) Red Hat Enterprise Linux 4 systems hang at the leap second. There have been some messages on the NTP list referencing posts on a Debian list about leap second crashes, and there's a post on /. about a similar problem with Fedora 8. -- Chris Adams cmad...@hiwaay.net Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
Leap second tonight
Just a reminder that there's a leap second tonight. Last time I watched for what happened on 01/01/2006, there was a little bit of chaos: http://markmail.org/message/cpoj3jw5onzhhjkr?q=%22kevin+day%22+leap+second+reminder+nanogpage=1refer=cnkxb3iv7sls5axu I've been told that some of the causes of these problems are fixed on any reasonably recent ntp distribution, but just in case, you might wanna keep an eye out if you're seeing any weirdness. The worst damage I'd heard from anyone after that event was their clock being significantly off for several hours. -- Kevin
Re: Leap second tonight
INTERNATIONAL EARTH ROTATION AND REFERENCE SYSTEMS SERVICE (IERS) SERVICE INTERNATIONAL DE LA ROTATION TERRESTRE ET DES SYSTEMES DE REFERENCE SERVICE DE LA ROTATION TERRESTRE OBSERVATOIRE DE PARIS 61, Av. de l'Observatoire 75014 PARIS (France) Tel. : 33 (0) 1 40 51 22 26 FAX : 33 (0) 1 40 51 22 91 e-mail: services.i...@obspm.fr http://hpiers.obspm.fr/eop-pc Paris, 4 July 2008 Bulletin C 36 To authorities responsible for the measurement and distribution of time UTC TIME STEP on the 1st of January 2009 A positive leap second will be introduced at the end of December 2008. The sequence of dates of the UTC second markers will be: 2008 December 31, 23h 59m 59s 2008 December 31, 23h 59m 60s 2009 January 1, 0h 0m 0s The difference between UTC and the International Atomic Time TAI is: from 2006 January 1, 0h UTC, to 2009 January 1 0h UTC : UTC-TAI = - 33s from 2009 January 1, 0h UTC, until further notice : UTC-TAI = - 34s Leap seconds can be introduced in UTC at the end of the months of December or June, depending on the evolution of UT1-TAI. Bulletin C is mailed every six months, either to announce a time step in UTC or to confirm that there will be no time step at the next possible date. Daniel GAMBIS Head Earth Orientation Center of IERS Observatoire de Paris, France
Re: Leap second tonight
Since leap seconds apply to UTC, won't the leap second be in about 22 minutes? -jasper On 1/01/2009, at 11:41 AM, Kevin Day wrote: Just a reminder that there's a leap second tonight. Last time I watched for what happened on 01/01/2006, there was a little bit of chaos: http://markmail.org/message/cpoj3jw5onzhhjkr?q=%22kevin+day%22+leap+second+reminder+nanogpage=1refer=cnkxb3iv7sls5axu I've been told that some of the causes of these problems are fixed on any reasonably recent ntp distribution, but just in case, you might wanna keep an eye out if you're seeing any weirdness. The worst damage I'd heard from anyone after that event was their clock being significantly off for several hours. -- Kevin -- Jasper Bryant-Greene Network Engineer, Unleash ddi: +64 3 978 1222 mob: +64 21 129 9458
Re: Leap second tonight
On Wed, Dec 31, 2008 at 04:41:39PM -0600, Kevin Day wrote: I've been told that some of the causes of these problems are fixed on any reasonably recent ntp distribution, but just in case, you might wanna keep an eye out if you're seeing any weirdness. The worst damage I'd heard from anyone after that event was their clock being significantly off for several hours. One note, if you're using ntpd along with an HF receiver and the CHU reference driver, you'll either need to manually retune your receiver to 7850 kHz or update your ntpd. As of approximately one hour ago, CHU has moved from 7335 kHz, where it has been for several decades up to 7850 kHz due to increasing shortwave broadcast interference. Also note that many reference clocks, including GPS derived ones, do not handle leap seconds correctly, so it may be a while before your reference clocks stabilize. Happy New Year! --msa
Re: Leap second tonight
From: Kevin Day toa...@dragondata.com Date: Wed, 31 Dec 2008 16:41:39 -0600 Just a reminder that there's a leap second tonight. Last time I watched for what happened on 01/01/2006, there was a little bit of chaos: http://markmail.org/message/cpoj3jw5onzhhjkr?q=%22kevin+day%22+leap+second+reminder+nanogpage=1refer=cnkxb3iv7sls5axu I've been told that some of the causes of these problems are fixed on any reasonably recent ntp distribution, but just in case, you might wanna keep an eye out if you're seeing any weirdness. The worst damage I'd heard from anyone after that event was their clock being significantly off for several hours. While I hope people are not still running NTP versions too old to handle leap seconds correctly, but that is only a small part of the problem. If the stratum 1 reference clocks don't handle leap seconds correctly, there is no way for NTP to deal with it. We use CDMA clocks and last leap second it took weeks for all of the cell sites to adjust the last one. As a result, I have set all of our clocks for manual leap second and set them to adjust tonight at midnight (UTC).I'll take a look in about 35 minutes and see how it worked. I've heard reports of various GPS clocks not dealing with the leap second correctly, as well. Fortunately, not too many need this sort of accuracy, but those of us who do need to be prepared. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: ober...@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 pgpb4CY7IEfZG.pgp Description: PGP signature
Re: Leap second tonight
bash-2.05b# date Thu Jan 1 00:59:58 CET 2009 bash-2.05b# date Thu Jan 1 00:59:59 CET 2009 bash-2.05b# date Thu Jan 1 00:59:60 CET 2009 bash-2.05b# date Thu Jan 1 01:00:00 CET 2009 bash-2.05b# date Thu Jan 1 01:00:01 CET 2009 bash-2.05b# -P
Re: Leap second tonight
Re: Leap second tonight
At which point my Solaris 10 v490's reboot in unison, lovely. Anyone else see anything interesting? -wil On Dec 31, 2008, at 4:01 PM, Peter Lothberg wrote: bash-2.05b# date Thu Jan 1 00:59:58 CET 2009 bash-2.05b# date Thu Jan 1 00:59:59 CET 2009 bash-2.05b# date Thu Jan 1 00:59:60 CET 2009 bash-2.05b# date Thu Jan 1 01:00:00 CET 2009 bash-2.05b# date Thu Jan 1 01:00:01 CET 2009 bash-2.05b# -P
Re: Leap second tonight
My Solaris 10 boxes are all happy (and did not reboot). I monitor NTP on a number of devices, including one router. The router was off by one second for a while, but is OK after an hour. Everything else was fine immediately. In 2005, our CDMA clock got the leap second between 15:08 and 15:38 EST creating some issues due to disagreement with the (too few) GPS clocks. Jon On Wed, Dec 31, 2008 at 7:53 PM, Wil Schultz wschu...@bsdboy.com wrote: At which point my Solaris 10 v490's reboot in unison, lovely. Anyone else see anything interesting? -wil
RE: Leap second tonight
It looks like clepsydra hasn't been updated: address ref clock st when poll reach delay offsetdisp -~192.5.41.40 .USNO.1 194 1024 37741.15.1938.2 -~130.207.244.240 .GPS. 168 1024 37723.1 11.09 1.3 ~127.127.7.1 127.127.7.1 75364 377 0.00.00 0.0 +~198.60.22.240.GPS. 1 436 1024 37765.77.0932.5 ~204.123.2.5 .GPS. 1 182 1024 37780.4 1011.424.9 +~67.128.71.76 .CDMA.1 178 1024 37716.8 10.8579.2 +~18.26.4.105 .CDMA.1 562 1024 377 8.47.90 235.8 -~209.81.9.7 .GPS. 1 170 1024 17779.9 16.96 170.8 *~209.51.161.238 .CDMA.1 1019 1024 377 3.9 10.13 2.5 -~204.152.184.72 .GPS. 1 697 1024 37775.01.81 3.2 +~18.145.0.30 .PSC. 1 1737 1024 37610.08.52 297.1 Quick, someone call DEC. Seriously, clepsydra is one of the older ntp servers out there, and hasn't dealt with the leap second correctly. -Original Message- From: Kevin Day [mailto:toa...@dragondata.com] Sent: Wednesday, December 31, 2008 5:42 PM To: NANOG list Subject: Leap second tonight Just a reminder that there's a leap second tonight. Last time I watched for what happened on 01/01/2006, there was a little bit of chaos: http://markmail.org/message/cpoj3jw5onzhhjkr?q=%22kevin+day%22+leap+second+reminder+nanogpage=1refer=cnkxb3iv7sls5axu I've been told that some of the causes of these problems are fixed on any reasonably recent ntp distribution, but just in case, you might wanna keep an eye out if you're seeing any weirdness. The worst damage I'd heard from anyone after that event was their clock being significantly off for several hours. -- Kevin
Re: Leap second tonight
On Wed, 31 Dec 2008 16:53:57 -0800 Wil Schultz wschu...@bsdboy.com wrote: At which point my Solaris 10 v490's reboot in unison, lovely. Solaris? Or ZuneOS? (See http://www.nytimes.com/2009/01/01/technology/personaltech/01zune.html) --Steve Bellovin, http://www.cs.columbia.edu/~smb
