Dear Lev, thanks for this! Can you please post links to the GitHub & Medium articles? Sorry for my ignorance. Best Regards! Mark
On Saturday, January 1, 2022, Lev Maximov <lev.maxi...@gmail.com> wrote: > I've dual-published the article on github and medium under the title 'A > comprehensive guide to NumPy data types'. > > Thank you all for your help and happy New Year! > > Best regards, > Lev > > On Sun, Jan 2, 2022 at 12:31 AM Stefano Miccoli <stefano.micc...@polimi.it> > wrote: > >> First of all, happy new 2022 UTC year! >> >> Let my add just a very brief note to this discussion: I opened >> https://github.com/numpy/numpy/issues/20675 which addresses the >> shortcomings of the current documentation, which is in my opinion not >> sufficiently explicit in stating the datetime64 semantics. It is true that >> these are largely consistent with python ‘datetime.datetime’, but ‘explicit >> is better than implicit’. If nobody objects I will then open a doc-only PR >> adding a very short paragraph to the docs trying to explain the points >> discussed here. >> >> As what regards how much time UTC gained from 1970-01-01 up to today, you >> are right, it’s about 29 s. The UTC timescale was officially born in 1963 >> but it can be traced back at least up to 1956/1958, see >> https://github.com/skyfielders/python-skyfield/issues/679 where this is >> discussed with reference to the timescales implemented in python-skyfield. >> >> Stefano >> >> On 31 Dec 2021, at 13:27, numpy-discussion-requ...@python.org wrote: >> >> Hey, Stefano! >> >> The level of being pedantic is absolutely acceptable. >> >> I don't question any of your arguments. They are all perfectly valid. >> >> Except that I'd rather say it is ~29 seconds if measuring against 1970. >> Leap seconds were introduced in 1972 and there were >> a total of 27 seconds since then, but TAI time was ticking since 1958 and >> gained 10 seconds by 1970 so it is approximately 0.83 second per year at >> which gives approx 28.67 sec between today and 1970. >> So 1970 is a bad choice of epoch if you want to introduce a >> leap-second-aware datetime. >> In GPS time they chose 1980. In TAI it is 1958, but that is somewhat >> worse than 1980 because it is not immediately clear how to perform the >> conversion timestamp<->timedelta between 1958 and 1970. >> >> Something like 'proleptic gps time' would be needed to estimate the >> number of leap seconds in the years before 1972 when they were introduced. >> Or maybe to limit the leap-second timescale >> to start at 1972 and not to accept any timestamps before that date. >> >> The system that ignores the existence of the leap seconds has a right to >> exist. >> But it just has limited applicability. >> >> np.datetime64 keeps time as a delta between the moment in time and a >> predefined epoch. >> Which standard does it use to translate this delta to human-readable time >> in years, >> months, and so on? >> >> If it is UTC, then it must handle times like 2016-12-31 23:59:60, because >> it is a valid UTC timestamp. >> >>> np.datetime64('2016-12-31 12:59:60') >> Traceback (most recent call last): >> File "<stdin>", line 1, in <module> >> ValueError: Seconds out of range in datetime string "2016-12-31 12:59:60" >> >> Datetime also fails (so far) to handle it: >> >>> dt(2016,12,31,23,59,60) >> Traceback (most recent call last): >> File "<stdin>", line 1, in <module> >> ValueError: second must be in 0..59 >> >> But `time` works. Well, at least it doesn't raise an exception: >> >>> t = time.struct_time((2016,12,31,12,59,60,0,0,0)); t >> time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=12, >> tm_min=59, tm_sec=60, tm_wday=0, tm_yday=0, tm_isdst=0) >> >>> time.asctime(t) >> 'Mon Dec 31 12:59:60 2016' >> >>> time.gmtime(calendar.timegm(t)) >> time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, >> tm_sec=0, tm_wday=6, tm_yday=1, tm_isdst=0) >> >> Imagine a user that decides which library to use to store some (life >> critical!) measurements taken every 100 ms. He looks at NumPy datetime64, >> reads that it is capable of handling attosecods, and decides that it is a >> perfect fit. Now imagine that on 31 Dec 2022 the World Government decided >> to inject a leap second. The system will receive the announcement from the >> NTC servers and will >> prepare to replay this second twice. As soon as this moment chimes in >> he'll run into a ValueError, which he won't notice because he's celebrating >> a New Year :) And guess whom he'll blame? ;) >> >> Actually the humanity has already got used to replaying timespans twice. >> It happens every year in the countries that observe daylight saving time. >> And the solution is to use a more linear scale than local time, namely, >> UTC. But now turns out that UTC is not linear enough and it also has >> certain timespans happening twice. >> >> The solution once again is use a _really_ linear time which is TAI. I >> think python 'time' library did a right thing to introduce time.CLOCK_TAI, >> after all. >> >> Astropy handles the UTC scale properly though: >> >>> t = Time('2016-12-31 23:59:60') >> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000> >> >>> t0 = Time('2016-12-31 23:59:59') >> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:59.000> >> >>> delta = t-t0 >> <TimeDelta object: scale='tai' format='jd' value=1.1574074074038876e-05> >> >>> delta.sec >> 0.9999999999969589 >> >>> t0 + delta >> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000> >> >> So the solution for that particular person with regular intervals of time >> is to use astropy. I mention it in the article. >> I made some corrections to the text. I'd be grateful if you had a look >> and pointed me to the particular sentences >> that need improvement. >> >> Best regards, >> Lev >> >> >> _______________________________________________ >> NumPy-Discussion mailing list -- numpy-discussion@python.org >> To unsubscribe send an email to numpy-discussion-le...@python.org >> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ >> Member address: lev.maxi...@gmail.com >> > -- Mark Mikofski, PhD (2005) *Fiat Lux*
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
