Dear Lev, thanks for this! Can you please post links to the GitHub & Medium
articles? Sorry for my ignorance. Best Regards! Mark

On Saturday, January 1, 2022, Lev Maximov <lev.maxi...@gmail.com> wrote:

> I've dual-published the article on github and medium under the title 'A
> comprehensive guide to NumPy data types'.
>
> Thank you all for your help and happy New Year!
>
> Best regards,
> Lev
>
> On Sun, Jan 2, 2022 at 12:31 AM Stefano Miccoli <stefano.micc...@polimi.it>
> wrote:
>
>> First of all, happy new 2022 UTC year!
>>
>> Let my add just a very brief note to this discussion: I opened
>> https://github.com/numpy/numpy/issues/20675 which addresses the
>> shortcomings of the current documentation, which is in my opinion not
>> sufficiently explicit in stating the datetime64 semantics. It is true that
>> these are largely consistent with python ‘datetime.datetime’, but ‘explicit
>> is better than implicit’. If nobody objects I will then open a doc-only PR
>> adding a very short paragraph to the docs trying to explain the points
>> discussed here.
>>
>> As what regards how much time UTC gained from 1970-01-01 up to today, you
>> are right, it’s about 29 s. The UTC timescale was officially born in 1963
>> but it can be traced back at least up to 1956/1958, see
>> https://github.com/skyfielders/python-skyfield/issues/679 where this is
>> discussed with reference to the timescales implemented in python-skyfield.
>>
>> Stefano
>>
>> On 31 Dec 2021, at 13:27, numpy-discussion-requ...@python.org wrote:
>>
>> Hey, Stefano!
>>
>> The level of being pedantic is absolutely acceptable.
>>
>> I don't question any of your arguments. They are all perfectly valid.
>>
>> Except that I'd rather say it is ~29 seconds if measuring against 1970.
>> Leap seconds were introduced in 1972 and there were
>> a total of 27 seconds since then, but TAI time was ticking since 1958 and
>> gained 10 seconds by 1970 so it is approximately 0.83 second per year at
>> which gives approx 28.67 sec between today and 1970.
>> So 1970 is a bad choice of epoch if you want to introduce a
>> leap-second-aware datetime.
>> In GPS time they chose 1980. In TAI it is 1958, but that is somewhat
>> worse than 1980 because it is not immediately clear how to perform the
>> conversion timestamp<->timedelta between 1958 and 1970.
>>
>> Something like 'proleptic gps time' would be needed to estimate the
>> number of leap seconds in the years before 1972 when they were introduced.
>> Or maybe to limit the leap-second timescale
>> to start at 1972 and not to accept any timestamps before that date.
>>
>> The system that ignores the existence of the leap seconds has a right to
>> exist.
>> But it just has limited applicability.
>>
>> np.datetime64 keeps time as a delta between the moment in time and a
>> predefined epoch.
>> Which standard does it use to translate this delta to human-readable time
>> in years,
>> months, and so on?
>>
>> If it is UTC, then it must handle times like 2016-12-31 23:59:60, because
>> it is a valid UTC timestamp.
>> >>> np.datetime64('2016-12-31 12:59:60')
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> ValueError: Seconds out of range in datetime string "2016-12-31 12:59:60"
>>
>> Datetime also fails (so far) to handle it:
>> >>> dt(2016,12,31,23,59,60)
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> ValueError: second must be in 0..59
>>
>> But `time` works. Well, at least it doesn't raise an exception:
>> >>> t = time.struct_time((2016,12,31,12,59,60,0,0,0)); t
>> time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=12,
>> tm_min=59, tm_sec=60, tm_wday=0, tm_yday=0, tm_isdst=0)
>> >>> time.asctime(t)
>> 'Mon Dec 31 12:59:60 2016'
>> >>> time.gmtime(calendar.timegm(t))
>> time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=1, tm_min=0,
>> tm_sec=0, tm_wday=6, tm_yday=1, tm_isdst=0)
>>
>> Imagine a user that decides which library to use to store some (life
>> critical!) measurements taken every 100 ms. He looks at NumPy datetime64,
>> reads that it is capable of handling attosecods, and decides that it is a
>> perfect fit. Now imagine that on 31 Dec 2022 the World Government decided
>> to inject a leap second. The system will receive the announcement from the
>> NTC servers and will
>> prepare to replay this second twice. As soon as this moment chimes in
>> he'll run into a ValueError, which he won't notice because he's celebrating
>> a New Year :) And guess whom he'll blame? ;)
>>
>> Actually the humanity has already got used to replaying timespans twice.
>> It happens every year in the countries that observe daylight saving time.
>> And the solution is to use a more linear scale than local time, namely,
>> UTC. But now turns out that UTC is not linear enough and it also has
>> certain timespans happening twice.
>>
>> The solution once again is use a _really_ linear time which is TAI. I
>> think python 'time' library did a right thing to introduce time.CLOCK_TAI,
>> after all.
>>
>> Astropy handles the UTC scale properly though:
>> >>> t = Time('2016-12-31 23:59:60')
>> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000>
>> >>> t0 = Time('2016-12-31 23:59:59')
>> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:59.000>
>> >>> delta = t-t0
>> <TimeDelta object: scale='tai' format='jd' value=1.1574074074038876e-05>
>> >>> delta.sec
>> 0.9999999999969589
>> >>> t0 + delta
>> <Time object: scale='utc' format='iso' value=2016-12-31 23:59:60.000>
>>
>> So the solution for that particular person with regular intervals of time
>> is to use astropy. I mention it in the article.
>> I made some corrections to the text. I'd be grateful if you had a look
>> and pointed me to the particular sentences
>> that need improvement.
>>
>> Best regards,
>> Lev
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list -- numpy-discussion@python.org
>> To unsubscribe send an email to numpy-discussion-le...@python.org
>> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
>> Member address: lev.maxi...@gmail.com
>>
>

-- 
Mark Mikofski, PhD (2005)
*Fiat Lux*
_______________________________________________
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-le...@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: arch...@mail-archive.com

Reply via email to