[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Ok, I fixed Python 3.4 and 3.5 too: fromtimestamp() and utcfromtimestamp() now 
uses also ROUND_HALF_EVEN rounding mode, as timedelta constructor. I explained 
in Misc/NEWS that the change was made to respect the property:

(datetime(1970,1,1) + timedelta(seconds=t)) == datetime.utcfromtimestamp(t)

Thanks Alexander Belopolsky & Tim Peters for your explanation on rounding. It's 
not a simple problem :-)

Thanks Tommaso Barbugli for the bug report: it's now fixed on all (maintained) 
Python 3 versions: 3.4, 3.5 and 3.6. The fix will be part of Python 3.5.1.

--
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset ee1cf1b188d2 by Victor Stinner in branch '3.4':
Issue #23517: Fix rounding in fromtimestamp() and utcfromtimestamp() methods
https://hg.python.org/cpython/rev/ee1cf1b188d2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Alexander: can you please review attached round_half_even_py34.patch? It's the 
minimum patch to fix datetime/timedelta rounding for Python 3.4 (and 3.5).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

Given Victor's reluctance to get this in to 3.5.0, this can't even be marked as 
a "deferred blocker" anymore.  Demoting to normal priority.

--
priority: deferred blocker -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

Given Victor's reluctance to get this in to 3.5.0, this can't even be marked as 
a "deferred blocker" anymore.  Demoting to normal priority.

--
priority: deferred blocker -> normal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Removed file: 
http://bugs.python.org/file40349/timedelta_round_half_up_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

I wish people wouldn't remove old patches.  There's no harm in leaving them, 
and it may be a useful historical record.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file40362/round_half_up_py34-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file40365/combined_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Removed file: 
http://bugs.python.org/file40364/fromtimestamp_round_half_up_py34-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

I modified Python 3.6 to use ROUND_HALF_EVEN rounding mode in 
datetime.timedelta, datetime.datetime.fromtimestamp(), 
datetime.datetime.utcfromtimestamp().

round_half_even_py34.patch: Backport this change to Python 3.4. I tried to 
write a minimal patch only changing datetime, not pytime.

--
Added file: http://bugs.python.org/file40414/round_half_even_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset 171d5590ebc3 by Victor Stinner in branch 'default':
Issue #23517: fromtimestamp() and utcfromtimestamp() methods of
https://hg.python.org/cpython/rev/171d5590ebc3

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> I wish people wouldn't remove old patches.  There's no harm in leaving them, 
> and it may be a useful historical record.

It became hard to read this issue, it has a long history. My old patches for 
Python 3.4 were for ROUND_HALF_UP, but it was then decided to use 
ROUND_HALF_EVEN. Technically, patches are not "removed" but "unlinked" from the 
issue: you can get from the history at the bottom of this page.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

Is it appropriate to make this change as a "bug fix", in already-released 
versions of Python?  Would you be happy or sad if you updated your Python from 
3.x.y to 3.x.y+1 and the rounding method used when converting floats to 
datetime stamps changed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Larry> Well, for now I assume it really truly genuinely isn't going in 3.5.0.

This is an unfortunate outcome.

Larry> I suppose we can debate about 3.4.x and 3.5.1 later

It is even more unfortunate that the question of whether this regression is a 
bug or not is up for debate again.

Victor> The code will be new, I'm not condifent enough to push it immedialty 
into python 3.5.

I don't understand why any new code is needed to fix a regression.  All you 
need to do is to revert a few chunks of 1e9cc1a03365 where the regression was 
introduced.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

If the guy doing the work says "don't merge it in 3.5.0", and we're at the 
final release candidate before 3.5.0 final ships, and we don't even have a 
patch that everyone likes yet... it does seem like it's not going to happen for 
3.5.0.  Unfortunate perhaps but that's the situation we're in.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> All you need to do is to revert a few chunks of 1e9cc1a03365 where the 
> regression was introduced.

Hum, please don't revert this change. I spent a lot of days to write 
pytime.c/.h.

My patches add more unit tests to datetime, to test the exact rounding mode.

> Well, for now I assume it really truly genuinely isn't going in 3.5.0.  I 
> suppose we can debate about 3.4.x and 3.5.1 later, once we have a fix 
> everybody is happy with.

The rounding mode of microseconds only impact a very few people. The rounding 
mode changed in 3.3, again in 3.4 and again in 3.5. This issue is the first bug 
report. So I think it's fine to modify 3.4.x and 3.5.x.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Victor> please don't revert this change.

I did not suggest reverting the entire commit.  The change that affects 
fromdatetime() is just

-us = round(frac * 1e6)
+us = int(frac * 1e6)

in datetime.py.  It is probably more involved in _datetimemodule.c, but cannot 
be that bad.  You can still keep pytime.c/.h.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Mark Dickinson]

Mark Dickinson added the comment:

FWIW, count me as +1 on roundTiesToEven, mostly just for consistency.  It's 
easier to remember that pretty much everything in Python 3 does 
round-ties-to-even (round function, string formatting, float literal 
evaluations, int-to-float conversion, Fraction-to-float conversion, ...) than 
to have to remember that there's a handful of corner cases where we do 
roundTiesToAway instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

Well, for now I assume it really truly genuinely isn't going in 3.5.0.  I 
suppose we can debate about 3.4.x and 3.5.1 later, once we have a fix everybody 
is happy with.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

Okay, this is literally the only thing rc3 is waiting on now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

I understand that I have to implement a new rounding mode. The code will be
new, I'm not condifent enough to push it immedialty into python 3.5. IMHO a
buggy rounding mode is worse than keeping the current rounding mode. The
rounding mode changed in python 3.3. There is no urgency to fix it.

I will change python 3.6, then 3.4 and 3.5.1.
Le 7 sept. 2015 07:33, "Larry Hastings"  a écrit :

>
> Larry Hastings added the comment:
>
> Okay, this is literally the only thing rc3 is waiting on now.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

I suspect we're not fixing this in 3.4, so I'm removing 3.4 from the version 
list.

--
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

Well, I'm already holding up rc3 on one other issue, might as well fix this 
too.  Can somebody make me a pull request?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Universal consensus on ROUND_HALF_EVEN, yes.

I would like to see it repaired for 3.5.0, but that's just me saying so without 
offering to do a single damned thing to make it happen ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

So is this considered broken enough that I need to accept a fix for 3.5.0?  And 
has a consensus been reached about exactly what that fix would be?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset d755f75019c8 by Victor Stinner in branch 'default':
Issue #23517: Skip a datetime test on Windows
https://hg.python.org/cpython/rev/d755f75019c8

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Victor, sorry if I muddied the waters here:  Alex & I do agree nearest/even 
must be used.  It's documented for timedelta already, and the 
seconds-from-the-epoch invariant Alex showed is at least as important to 
preserve as round-tripping.

Alex, agreed about the overall mean in the example, but expect that continuing 
to exchange contrived examples isn't really a major life goal for either of us 
;-)  Thanks for playing along.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> Alex & I do agree nearest/even must be used.

Ok, so Python 3.6 should be updated to use ROUND_HALF_EVEN, and then
updated patches should be backported to Python 3.4 and 3.5. Right?

Right now, Python 3.6 uses ROUND_HALF_UP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

You may find it easier to start with a patch for 3.5 which is the only time 
sensitive task.  I'll be happy to review your code in whatever form you find it 
easier to submit, but I believe in hg it is easier to forward port than to 
backport.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

> It doesn't matter who's consuming the rounding of a binary
> float to decimal microseconds

That is true, but what does matter is who is producing the incoming floats.   
Consider an extreme case of a timer that ticks twice a microsecond and gives 
you results in a timespec struct style sec, nsec pair.  If you convert those to 
timedeltas with timedelta(0, sec, nsec/1000), you will see half of nsec/1000 
floats ending in .5 and half ending with .0.  If you have a large sample of 
(sec, nsec) measurements, the mean of corresponding timedeltas will be 0.25µs 
larger with the half-up rounding than the actual mean of your samples.

Is round-half-to-even a panacea?  Of course not.  In an extreme case if all 
your measurements are 1.5µs - it will not help at all, but in a more realistic 
sample you are likely to have an approximately equal number of even and odd µs 
and the Dutch rounding will affect the mean much less than round-half-up.

As you said, for most uses none of this matters, but we are not discussing a 
change to timedelta(0, s) rounding here.  All I want from this issue is to keep 
the promise that we make in the docs:

On the POSIX compliant platforms, utcfromtimestamp(timestamp) is equivalent to 
the following expression:

datetime(1970, 1, 1) + timedelta(seconds=timestamp)

https://docs.python.org/3/library/datetime.html#datetime.datetime.utcfromtimestamp

Tim, while we have this entertaining theoretical dispute, I am afraid we are 
leaving Victor confused about what he has to do in his patch.  I think we agree 
that fromtimestamp() should use Dutch rounding.  We only disagree why.

If this is the case, please close this thread with a clear advise to use 
round-half-to-even and we can continue discussing the rationale privately or in 
a different forum.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

That's great, Victor!  Another person trying the code with their own critical 
eyes would still be prudent.  Two days ago you wrote:

> This part of Python (handling timestamps, especially
> the rounding mode) is complex, I prefer to check for
> all buildbots and wait for some feedback from users
> (wait at least 2 weeks).

It's not entirely clear why that switched to "So I'm confident on the change." 
in 12 days short of 2 weeks ;-)

I have no reason to doubt your confidence.  Just saying some independent 
checking is prudent (but I can't do it at this time).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

I'll try to find the time to kick the tires on this patch this weekend.

--
assignee:  -> belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Alexander Belopolsky added the comment:
> I'll try to find the time to kick the tires on this patch this weekend.

Cool! Keep me in touch ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Larry, I appreciate the vote of confidence, but I'm ill-equipped to help at the 
patch level:  I'm solely on Windows, and (long story) don't even have a C 
compiler at the moment.  The patch(es) are too broad and delicate to be sure of 
without kicking the tires (running contrived examples).

So I would sub-delegate to Alexander and/or Mark.  They understand the issues 
too.  I was just the most annoying about insisting it get fixed ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

2015-09-04 17:52 GMT+02:00 Tim Peters :
> That's great, Victor!  Another person trying the code with their own critical 
> eyes would still be prudent.

Sure!

> It's not entirely clear why that switched to "So I'm confident on the 
> change." in 12 days short of 2 weeks ;-)

He he. 2 days ago, the buildbots were broken for various reasons. I
fixed a lot of issues (unrelated to this rounding mode issue), so I
now got the confirmation that the test pass on all platforms.

> I have no reason to doubt your confidence.  Just saying some independent 
> checking is prudent (but I can't do it at this time).

Sorry if I wasn't clear. I'm confident, but not enough to not wait for
a review :-)

--

Usually, I don't wait for a review simply because there are too few
reviewers :-( I spent the last 3 years to work alone on the funnny
_PyTime C API project. I started to write an article to tell this
journey ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> Larry, I appreciate the vote of confidence, but I'm ill-equipped to help at 
> the patch level:  (...)  The patch(es) are too broad and delicate to be sure 
> of without kicking the tires (running contrived examples).

Well, the patches change how timedelta, .fromtimestamp() and 
.utcfromtimestamp() round the number of microseconds. It's a deliberate choice 
since it was decided that the current rounding mode is a bug, and not a feature 
:-)

The code is well tested. There are unit tests on how numbers are rounded for: 
timedelta, .(utc)fromtimestamp(), and even the C private API _PyTime. The code 
is (almost) the same in default and was validated on various platforms. So I'm 
confident on the change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Changes by Alexander Belopolsky :


--
stage:  -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Victor,

Do I understand correctly that this is already committed in 3.4 - 3.6 
development branches and we just need to decide whether to cherry-pick this fix 
to 3.5rc?

Is the "review" link up-to date?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

It looks like this patch violates fromtimestamp(s) == EPOCH + 
timedelta(seconds=s) invariant:

Python 3.6.0a0 (default:73911e6c97c8, Sep  4 2015, 13:14:12)
>>> E = datetime(1970,1,1,tzinfo=timezone.utc)
>>> s = -1/2**7
>>> datetime.fromtimestamp(s, timezone.utc) == E + timedelta(seconds=s)
False

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Can someone check 3.3?  I believe that was the release where we tried to get 
various rounding issues right.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

FYI, that invariant failed for me just now under the released 3.4.3 too:

Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import *
>>> E = datetime(1970,1,1,tzinfo=timezone.utc)
>>> s = -1/2**7
>>> datetime.fromtimestamp(s, timezone.utc)
datetime.datetime(1969, 12, 31, 23, 59, 59, 992187, 
tzinfo=datetime.timezone.utc)
>>> E + timedelta(seconds=s)
datetime.datetime(1969, 12, 31, 23, 59, 59, 992188, 
tzinfo=datetime.timezone.utc)

It's an exactly-tied rounding case for the exactly-representable-in-binary s = 
-0.0078125.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Oh, it looks like my implementation of ROUND_HALF_UP is wrong.
Negative numbers are not correctly rounded :-/ I'm working on a fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Alex, if you like, I'll take the blame for the rounding method - I did express 
a preference for it here:

http://bugs.python.org/issue23517#msg249309

When I looked at the code earlier, the round-half-up implementation looked good 
to me (floor(x+0.5) if x >= 0 else ceil(x-0.5)).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> I did not quite understand why you've chosen ROUND_HALF_UP over 
> ROUND_HALF_EVEN, but as long as fromtimestamp() uses the same rounding as 
> timedelta() - I am happy.

Not only Tim prefers this rounding mode, Python 2.7 also uses the same mode, 
and the original poster basically said that Python 3 doesn't behave like Python 
2. So... the most obvious rounding mode is the same than Python 2, 
ROUND_HALF_UP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Mark Lawrence]

Mark Lawrence added the comment:

Python 3.3.5 (v3.3.5:62cf4e77f785, Mar  9 2014, 10:35:05) [MSC v.1600 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import *
>>> E = datetime(1970,1,1,tzinfo=timezone.utc)
>>> s = -1/2**7
>>> datetime.fromtimestamp(s, timezone.utc)
datetime.datetime(1969, 12, 31, 23, 59, 59, 992187, 
tzinfo=datetime.timezone.utc)
>>> E + timedelta(seconds=s)
datetime.datetime(1969, 12, 31, 23, 59, 59, 992187, 
tzinfo=datetime.timezone.utc)

FWIW Windows 10.

--
nosy: +BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Victor,

I did not quite understand why you've chosen ROUND_HALF_UP over 
ROUND_HALF_EVEN, but as long as fromtimestamp() uses the same rounding as 
timedelta() - I am happy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset 3c29d05c0710 by Victor Stinner in branch 'default':
Issue #23517: Fix implementation of the ROUND_HALF_UP rounding mode in
https://hg.python.org/cpython/rev/3c29d05c0710

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

.. and "[Round half to even] is the default rounding mode used in IEEE 754 
computing functions and operators."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Goodness.  It's the properties of "randomly chosen decimals" that have nothing 
to do with timestamps ;-)  timestamps aren't random, so "statistical bias" is 
roughly meaningless in this context.  I gave a specific example before of how 
nearest/even destroys obvious regularities in a _sequence_ of timestamps, where 
half-up preserves as much of the input regularity as possible.  That's worth 
more than a million abstract "head arguments" on Wikipedia.

But it doesn't make a lick of real difference either way.  We're rounding to 
microseconds, and there are only 64 "fractional parts" where the methods could 
_possibly_ deliver different results:  those of the form i/128 for i in 
range(1, 128, 2).  All and only those are exactly representable in base 2, and 
require exactly 7 decimal digits "after the decimal point" to express in 
decimal, _and_ end with "5" in decimal.  Half end with "25" while the other 
half with "75".  So Alex's 1/128 is one of the only 32 possible fractional 
parts where it makes a difference.  We systematically force all these cases to 
even, and dare think that's _not_ "biased"?  Half-up would leave half the 
results even and half odd, exactly the same as the _input_ odd/even 
distribution of the 6th digit.  And preserve the input strict alternation 
between even and odd in the 6th digit.  nearest/even destroys all of that.

Except that, I agree, there's no arguing with "Dutch rounding" ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

> Half-up leaves them all 5 microseconds apart,

When only looking at the decimal digit in the 6th place after rounding.
Which is all I did look at ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Removed file: 
http://bugs.python.org/file40363/fromtimestamp_round_half_up_py34-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Added file: 
http://bugs.python.org/file40364/fromtimestamp_round_half_up_py34-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> Is the "review" link up-to date?

Oh, I wrote a patch serie, but Rietveld doesn't know the dependencies between 
my patches...

So I attached combined_py34.patch which combines the 3 patches into a single 
one, hard to reviewer, but it should work with Rietveld.

--
Added file: http://bugs.python.org/file40365/combined_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Here is what Wikipedia has to say on the subject:

"""
Round half to even .. This method treats positive and negative values 
symmetrically, and is therefore free of sign bias. More importantly, for 
reasonable distributions of y values, the expected (average) value of the 
rounded numbers is the same as that of the original numbers. However, this rule 
will introduce a towards-zero bias when y − 0.5 is even, and a towards-infinity 
bias for when it is odd.
"""
https://en.wikipedia.org/wiki/Rounding#Round_half_to_even

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

It would be nice to hear from Mark Dickinson on this.  In Python 3, we took a 
much more systematic approach to rounding than a rather haphazard Python 2. For 
example, the rounding mode for timedelta(0, float_seconds) is not specified in 
Python 2, but it is in Python 3:


https://docs.python.org/2/library/datetime.html#datetime.timedelta
https://docs.python.org/3/library/datetime.html#datetime.timedelta

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file40362/round_half_up_py34-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Removed file: 
http://bugs.python.org/file40351/fromtimestamp_round_half_up_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Added file: 
http://bugs.python.org/file40363/fromtimestamp_round_half_up_py34-2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file40350/round_half_up_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Yes, it would be good to hear from Mark.  When I first saw this report, I 
checked to see whether he was on the nosy list.  He is, but is apparently busy 
elsewhere.

My opinions haven't changed:  nearest/even is unnatural for rounding times 
("sometimes up, sometimes down" grates against the nature of monotonically 
non-decreasing timestamps), but it doesn't make a lick of real difference.  If 
the inconsequential choice was documented, then sure, that it _was_ documented 
makes it consequential, so that's the one that must be used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> My opinions haven't changed:  nearest/even is unnatural for rounding times 
> ("sometimes up, sometimes down" grates against the nature of monotonically 
> non-decreasing timestamps),

By the way, why does Python use ROUND_HALF_EVEN for round()? It also feels 
unnatural to me!

>>> round(0.5)
0
>>> round(1.5)
2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Victor, there are good "theoretical" reasons for using half/even rounding in 
_general_ use.  But this bug report isn't the place to get into them.  Here it 
just should be enough to note that the IEEE 754 floating point standard 
_requires_ half/even to be the default rounding mode.  Slowly but surely, all 
the world's non-business computer applications are moving toward that.  Python 
is just one of 'em.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

> By the way, why does Python use ROUND_HALF_EVEN for round()?

ROUND_HALF_EVEN does not introduce statistical bias in your floating point 
data.  With this choice a randomly chosen decimal has an equal chance of being 
rounded up or down.  I think one can prove the same for a binary float being 
converted to a decimal with a fixed number of places after dot.  The builtin 
round() is a unique beast and I would have to ask Mark to know what its 
statistical properties are, but in general round-half-to-even is justified by 
the desire of not introducing a statistical bias and I don't think the natural 
asymmetry of the time line matters in this argument.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

.. and here is an unbeatable argument: "this variant of the round-to-nearest 
method is also called unbiased rounding, convergent rounding, statistician's 
rounding, **Dutch rounding**, Gaussian rounding, odd–even rounding, or bankers' 
rounding."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Ok, thanks for the explanation :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

> timestamps aren't random, so "statistical bias" is roughly meaningless in 
> this context.

I agree.  I don't think I made any arguments about timestamps specifically 
other than a consistency with timedeltas.  In the later case, I think all the 
usual arguments for half-to-even tiebreaker apply.

Let's just leave it at "When in doubt - use Dutch rounding."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Bah.  It doesn't matter who's consuming the rounding of a binary float to 
decimal microseconds:  there are only 32 possible fractional parts where 
nearest/even and half-up deliver different results.  half-up preserves 
properties of these specific inputs that nearest/even destroys.  These inputs 
themselves have no bias - they're utterly uniformly spaced.

Not only does nearest/even _introduce_ bias on these inputs by destroying these 
properties, it doesn't even preserve the spacing between them.  Half-up leaves 
them all 5 microseconds apart, while nearest/even creates a bizarre "sometimes 
4 microseconds apart, sometimes 6" output spacing out of thin air.

So it's not a question of "when in doubt" to me, it's a question of "live up to 
what the docs already say".  Although, again, it doesn't make a lick of real 
difference.  That's why we'll never stop arguing about it ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

I will happy delegate to Tim Peters whether or not this should be fixed in 
3.5.0, or whether it should wait until 3.5.1 or even 3.6.

Tim, ball's in your court!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Added file: http://bugs.python.org/file40350/round_half_up_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Backport to Python 3.4 splitted in 3 patches:

* (1) timedelta_round_half_up_py34.patch, backport changeset 0eb8c182131e: 
"datetime.timedelta constructor now rounds microseconds to nearest with ties 
going away from zero (ROUND_HALF_UP)".

* (2) round_half_up.patch: add _PyTime_ROUND_HALF_UP rounding mode to the 
_PyTime API

* (3) fromtimestamp_round_half_up.patch, backport changeset bf634dfe076f: 
"fromtimestamp() and utcfromtimestamp() methods of datetime.datetime now round 
microseconds to nearest with ties going away from zero (ROUND_HALF_UP)"

--
keywords: +patch
Added file: http://bugs.python.org/file40349/timedelta_round_half_up_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

Changes by STINNER Victor :


Added file: 
http://bugs.python.org/file40351/fromtimestamp_round_half_up_py34.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset bf634dfe076f by Victor Stinner in branch 'default':
Issue #23517: fromtimestamp() and utcfromtimestamp() methods of
https://hg.python.org/cpython/rev/bf634dfe076f

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Larry Hasting wrote:
>> too late for 3.5.0
> How's that?

Well, for example... my change broke all buildbots.

I don't think that it's good idea to rush to fix Python 3.5 :-) This part of 
Python (handling timestamps, especially the rounding mode) is complex, I prefer 
to check for all buildbots and wait for some feedback from users (wait at least 
2 weeks).

I reverted my change, another function must be changed:

$ python2 -c 'import datetime; print(datetime.timedelta(microseconds=0.5))'
0:00:00.01
$ python3 -c 'import datetime; print(datetime.timedelta(microseconds=0.5))'
0:00:00

datetime.timedelta must also use the ROUND_HALF_UP method, as Python 2, instead 
of ROUND_HALF_EVEN (datetime.py uses the round() function).

$ python2 -c 'import datetime; print(datetime.timedelta(microseconds=1.5))'
0:00:00.02
$ python3 -c 'import datetime; print(datetime.timedelta(microseconds=1.5))'
0:00:00.02

I have to rework my patch to use ROUND_HALF_UP in datetime.timedelta(), 
datetime.datetime.fromtimestamp() and datetime.datetime.utcfromtimestamp(), and 
update test_datetime.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset 30454ef98e81 by Victor Stinner in branch 'default':
Backed out changeset b690bf218702
https://hg.python.org/cpython/rev/30454ef98e81

New changeset 700303850cd7 by Victor Stinner in branch 'default':
Issue #23517: Fix _PyTime_ObjectToDenominator()
https://hg.python.org/cpython/rev/700303850cd7

New changeset 03c97bb04cd2 by Victor Stinner in branch 'default':
Issue #23517: Reintroduce unit tests for the old PyTime API since it's still
https://hg.python.org/cpython/rev/03c97bb04cd2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset 0eb8c182131e by Victor Stinner in branch 'default':
Issue #23517: datetime.timedelta constructor now rounds microseconds to nearest
https://hg.python.org/cpython/rev/0eb8c182131e

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset 59185ef69166 by Victor Stinner in branch 'default':
Issue #23517: test_time, skip a test checking a corner case on floating point
https://hg.python.org/cpython/rev/59185ef69166

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset df074eb2a5be by Victor Stinner in branch 'default':
Issue #23517: Try to fix test_time on "x86 Ubuntu Shared 3.x" buildbot
https://hg.python.org/cpython/rev/df074eb2a5be

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset abeb625b20c2 by Victor Stinner in branch 'default':
Issue #23517: Add "half up" rounding mode to the _PyTime API
https://hg.python.org/cpython/rev/abeb625b20c2

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Ok, I fixed the issue in Python 3.6. Example with the initial message:

$ python2.7 -c 'import datetime; 
print(datetime.datetime.utcfromtimestamp(1424817268.274).microsecond); 
print(datetime.datetime.utcfromtimestamp(-1424817268.274).microsecond)'
274000
726000

$ python3.6 -c 'import datetime; 
print(datetime.datetime.utcfromtimestamp(1424817268.274).microsecond); 
print(datetime.datetime.utcfromtimestamp(-1424817268.274).microsecond)'
274000
726000

I wrote:
"On Python < 3.3, datetime.datetime.fromtimestamp(float) doesn't use exactly 
ROUND_HALF_EVEN, but it looks more to "round half away from zero" (the decimal 
module doesn't seem to support this exact rounding method)."

I was wrong: it's decimal.ROUND_HALF_UP in fact.

I will backport the change to Python 3.4 and 3.5. Since this issue was defined 
as a bugfix, it should be fixed in Python 3.5.1 (too late for 3.5.0).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Larry Hastings]

Larry Hastings added the comment:

> too late for 3.5.0

How's that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset b690bf218702 by Victor Stinner in branch 'default':
Issue #23517: datetime.datetime.fromtimestamp() and
https://hg.python.org/cpython/rev/b690bf218702

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

 IMHO we should only modify the rounding method used by
 datetime.datetime.fromtimestamp() and
 datetime.datetime.utcfromtimestamp(), other functions
 use the right rounding method.

Fine by me.  How about today? ;-)

The regression reported here must get repaired.  nearest/even is generally 
favored when there's a choice.

I personally _prefer_ add-a-half-and-chop in time contexts that need rounding, 
because it's more uniform.  That is, picturing a context that rounds to 1 digit 
for clarity, using a decimal system, with a uniformly spaced sequence of inputs 
on the first line, then a line with add-a-half-and-chop results, and then a 
line with nearest/even results:

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 input
0.0 1.0 1.0 2.0 2.0 3.0 3.0 4.0 add-a-half-and-chop
0.0 0.0 1.0 2.0 2.0 2.0 3.0 4.0 nearest/even

From the last (nearest/even) line, you'd never guess that the inputs were 
uniformly spaced; in the second line, you would.  nearest/even's in a tie, 
sometimes go up, sometimes go down is, IMO, unnatural in this context.

But it doesn't make a lick of real difference to timestamp functions.  We're 
not working in decimal, and people aren't going to be staring at hex patterns 
in output.  So I'd pick whichever is easier to implement.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

  x = float.fromhex('0x1.38f312b1b36bdp-1')
  x
 0.6112295
  round(x, 6)
 0.611229
  timedelta(0, x).microseconds
 611230

 but I no longer remember whether we concluded that
 timedelta got it wrong or round or both or neither. :-)

Here you go:

 import decimal
 decimal.Decimal(x)
Decimal('0.611229498116351207499974407255649566650390625')

That's the exact value you're actually using.  What's correct depends on 
what's intended.

round(x, 6) actually rounds to

 decimal.Decimal(round(x, 6))
0.61122900222968310481519438326358795166015625

and that's fine.  timedelta's result does not match what using infinite 
precision would deliver, but I couldn't care much less ;-)

The real lesson to take from all this, when you design your own killer 
language, is that using a binary floating point type for timestamps comes with 
many costs and surprises.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Changes by Alexander Belopolsky alexander.belopol...@gmail.com:


--
title: datetime.utcfromtimestamp parses timestamps incorrectly - 
datetime.utcfromtimestamp rounds results incorrectly

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

I wish we could use the same algorithm in datetime.utcfromtimestamp as we use 
in float to string conversion.  This may allow the following chain of 
conversions to round trip in most cases:

float literal - float - datetime - seconds.microseconds string

--
nosy: +belopolsky

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

 I wish we could use the same algorithm in
 datetime.utcfromtimestamp as we use in float
 to string conversion.  This may allow the
 following chain of conversions to round trip in most cases:

 float literal - float - datetime - seconds.microseconds string

I don't follow.  float-string produces the shortest string that reproduces the 
float exactly.  Any flavor of changing a timestamp to a microsecond-precision 
datetime is essentially converting a float * 1e6 to an integer - there doesn't 
seem to be a coherent concept of shortest integer that could apply.  We have 
to fill every bit a datetime has.

A variant of the code I posted could be good enough:  take the result we get 
now (truncate float*1e6).  Also add 1 ulp to the float and do that again.  If 
the results are the same, we're done.  If the results are different, and the 
difference is 1, take the second result.  Else keep the first result.  What 
this means is that we're rounding up if and only if the original is so close 
to the boundary that the tiniest possible amount of floating-point noise is all 
that's keeping it from giving a different result - but also that the float has 
enough bits to represent a 1-microsecond difference (which is true of current 
times, but in a couple centuries will become false).

But that's all nuts compared to just rounding float*1e6 to the nearest int, 
period.  There's nothing wrong with doing that.  Truncating is propagating the 
tiniest possible binary fp representation error all the way into the 
microseconds.  It would be defensible _if_ we were using base-10 floats (in 
which representation error doesn't occur for values expressed _in_ base 10).  
But we're not.  Truncating a base-2 float _as if_ it were a base-10 float is 
certain to cause problems.  Like the one this report is about ;-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

I probably misremembered a different issue.  See msg194311.

 timedelta(seconds=0.6112295) == timedelta(seconds=1)*0.6112295
False

I thought the problem there was that the same float was converted to one 
decimal by str() and to a different decimal by timedelta.  But now it looks 
like it was something else.

Does your algorithm guarantee that any float that is displayed with 6 decimal 
places or less will convert to a datetime or timedelta with microseconds 
matching the fractional part?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

OK, I looked at the wrong place.  Here is the correct example:

 x = float.fromhex('0x1.38f312b1b36bdp-1')
 x
0.6112295
 round(x, 6)
0.611229
 timedelta(0, x).microseconds
611230

but I no longer remember whether we concluded that timedelta got it wrong or 
round or both or neither. :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

 Does your algorithm guarantee that any float that
 is displayed with 6 decimal places or less will
 convert to a datetime or timedelta with microseconds
 matching the fractional part?

No algorithm can, for datetimes far enough in the future (C doubles just plain 
run out of enough bits).

Apart from negative timestamps (which I didn't consider - they just blow up on 
my platform :-) ), the intent is to do the best that _can_ be done.

But _proving_ things in this area isn't simple, and there's no need for it:  
check in a change to round the thing, and be done with it.  If Victor wants to 
rework rounding again, that's fine, but only under a _requirement_ that this 
particular bug remain fixed.  His change created the problem, and it's still 
languishing half a year after being reported - there's little sense in 
continuing to wait for him to do something about it.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

Hi, I'm trying to write the rationale of the changes that I wrote in pytime.h 
in Python 3.3-3.5. The rounding of microseconds or nanoseconds is tricky. The 
code changed a lot and we used and we are still using various rounding methods 
depending on the case...

Alexander Belopolsky wrote:
 I believe the correct mode is ROUND_HALF_EVEN.  This is the mode used by 
 the builtin round() function: (...)

Right, round(float) and round(decimal.Decimal) uses the ROUND_HALF_EVEN 
rounding method.

On Python  3.3, datetime.datetime.fromtimestamp(float) doesn't use exactly 
ROUND_HALF_EVEN, but it looks more to round half away from zero (the decimal 
module doesn't seem to support this exact rounding method).

The difference between ROUND_HALF_EVEN and round half away from zero is 
subtle. The two rounding methods only return a different result on the 
following case:

   divmod(t + us * 1e-6, 1.0)[1] * 1e6 == 0.5

where t and us are integers (t is a number of seconds created by mktime() and 
us is a number of microseconds in [0; 99]).

I don't think that the case can occur. I failed to find such case for various 
values of t between 0 and 2**40, and us=0 or us=1. 1e-6 (10^-6 = 0.01) 
cannot be represented exactly in base 2 (IEEE 754).

--

To move forward, we should agree on which rounding method 
datetime.datetime.fromtimestamp() should use, implement it in pytime.c (add a 
constant in pytime.h, implement it in pytime.c, and then write unit tests in 
test_time.py), and then use it in datetime.datetime.fromtimestamp().

IMHO we should only modify the rounding method used by 
datetime.datetime.fromtimestamp() and datetime.datetime.utcfromtimestamp(), 
other functions use the right rounding method.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com