Dear Mark, They are GitHub <https://axil.github.io/a-comprehensive-guide-to-numpy-data-types.html> and Medium <https://betterprogramming.pub/a-comprehensive-guide-to-numpy-data-types-8f62cb57ea83?sk=a417246f0cf9e24aca734525711299d3> ('friend link').
Best regards, Lev On Sun, Jan 2, 2022 at 9:43 AM Dr. Mark Alexander Mikofski PhD < mikof...@berkeley.edu> wrote: > Dear Lev, thanks for this! Can you please post links to the GitHub & > Medium articles? Sorry for my ignorance. Best Regards! Mark > > On Saturday, January 1, 2022, Lev Maximov <lev.maxi...@gmail.com> wrote: > >> I've dual-published the article on github and medium under the title 'A >> comprehensive guide to NumPy data types'. >> >> Thank you all for your help and happy New Year! >> >> Best regards, >> Lev >> >> On Sun, Jan 2, 2022 at 12:31 AM Stefano Miccoli < >> stefano.micc...@polimi.it> wrote: >> >>> First of all, happy new 2022 UTC year! >>> >>> Let my add just a very brief note to this discussion: I opened >>> https://github.com/numpy/numpy/issues/20675 which addresses the >>> shortcomings of the current documentation, which is in my opinion not >>> sufficiently explicit in stating the datetime64 semantics. It is true that >>> these are largely consistent with python ‘datetime.datetime’, but ‘explicit >>> is better than implicit’. If nobody objects I will then open a doc-only PR >>> adding a very short paragraph to the docs trying to explain the points >>> discussed here. >>> >>> As what regards how much time UTC gained from 1970-01-01 up to today, >>> you are right, it’s about 29 s. The UTC timescale was officially born in >>> 1963 but it can be traced back at least up to 1956/1958, see >>> https://github.com/skyfielders/python-skyfield/issues/679 where this is >>> discussed with reference to the timescales implemented in python-skyfield. >>> >>> Stefano >>> >>> On 31 Dec 2021, at 13:27, numpy-discussion-requ...@python.org wrote: >>> >>> Hey, Stefano! >>> >>> The level of being pedantic is absolutely acceptable. >>> >>> I don't question any of your arguments. They are all perfectly valid. >>> >>> Except that I'd rather say it is ~29 seconds if measuring against 1970. >>> Leap seconds were introduced in 1972 and there were >>> a total of 27 seconds since then, but TAI time was ticking since 1958 >>> and gained 10 seconds by 1970 so it is approximately 0.83 second per year >>> at which gives approx 28.67 sec between today and 1970. >>> So 1970 is a bad choice of epoch if you want to introduce a >>> leap-second-aware datetime. >>> In GPS time they chose 1980. In TAI it is 1958, but that is somewhat >>> worse than 1980 because it is not immediately clear how to perform the >>> conversion timestamp<->timedelta between 1958 and 1970. >>> >>> Something like 'proleptic gps time' would be needed to estimate the >>> number of leap seconds in the years before 1972 when they were introduced. >>> Or maybe to limit the leap-second timescale >>> to start at 1972 and not to accept any timestamps before that date. >>> >>> The system that ignores the existence of the leap seconds has a right to >>> exist. >>> But it just has limited applicability. >>> >>> np.datetime64 keeps time as a delta between the moment in time and a >>> predefined epoch. >>> Which standard does it use to translate this delta to human-readable >>> time in years, >>> months, and so on? >>> >>> If it is UTC, then it must handle times like 2016-12-31 23:59:60, >>> because it is a valid UTC timestamp. >>> >>> np.datetime64('2016-12-31 12:59:60') >>> Traceback (most recent call last): >>> File "<stdin>", line 1, in <module> >>> ValueError: Seconds out of range in datetime string "2016-12-31 12:59:60" >>> >>> Datetime also fails (so far) to handle it: >>> >>> dt(2016,12,31,23,59,60) >>> Traceback (most recent call last): >>> File "<stdin>", line 1, in <module> >>> ValueError: second must be in 0..59 >>> >>> But `time` works. Well, at least it doesn't raise an exception: >>> >>> t = time.struct_time((2016,12,31,12,59,60,0,0,0)); t >>> time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=12, >>> tm_min=59, tm_sec=60, tm_wday=0, tm_yday=0, tm_isdst=0) >>> >>> time.asctime(t) >>> 'Mon Dec 31 12:59:60 2016' >>> >>> time.gmtime(calendar.timegm(t)) >>> time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0, >>> tm_sec=0, tm_wday=6, tm_yday=1, tm_isdst=0) >>> >>> Imagine a user that decides which library to use to store some (life >>> critical!) measurements taken every 100 ms. He looks at NumPy datetime64, >>> reads that it is capable of handling attosecods, and decides that it is a >>> perfect fit. Now imagine that on 31 Dec 2022 the World Government decided >>> to inject a leap second. The system will receive the announcement from the >>> NTC servers and will >>> prepare to replay this second twice. As soon as this moment chimes in >>> he'll run into a ValueError, which he won't notice because he's celebrating >>> a New Year :) And guess whom he'll blame? ;) >>> >>> Actually the humanity has already got used to replaying timespans twice. >>> It happens every year in the countries that observe daylight saving time. >>> And the solution is to use a more linear scale than local time, namely, >>> UTC. But now turns out that UTC is not linear enough and it also has >>> certain timespans happening twice. >>> >>> The solution once again is use a _really_ linear time which is TAI. I >>> think python 'time' library did a right thing to introduce time.CLOCK_TAI, >>> after all. >>> >>> Astropy handles the UTC scale properly though: >>> >>> t = Time('2016-12-31 23:59:60') >>> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000> >>> >>> t0 = Time('2016-12-31 23:59:59') >>> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:59.000> >>> >>> delta = t-t0 >>> <TimeDelta object: scale='tai' format='jd' value=1.1574074074038876e-05> >>> >>> delta.sec >>> 0.9999999999969589 >>> >>> t0 + delta >>> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000> >>> >>> So the solution for that particular person with regular intervals of >>> time is to use astropy. I mention it in the article. >>> I made some corrections to the text. I'd be grateful if you had a look >>> and pointed me to the particular sentences >>> that need improvement. >>> >>> Best regards, >>> Lev >>> >>> >>> _______________________________________________ >>> NumPy-Discussion mailing list -- numpy-discussion@python.org >>> To unsubscribe send an email to numpy-discussion-le...@python.org >>> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ >>> Member address: lev.maxi...@gmail.com >>> >> > > -- > Mark Mikofski, PhD (2005) > *Fiat Lux* > > _______________________________________________ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: lev.maxi...@gmail.com >
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
