[systemsettings] [Bug 431057] Refresh rate values in Display Configuration menu are always incorrect due to floating-point precision issues
https://bugs.kde.org/show_bug.cgi?id=431057 Nate Graham changed: What|Removed |Added CC|m...@ratijas.tk | -- You are receiving this mail because: You are watching all bug changes.
[systemsettings] [Bug 431057] Refresh rate values in Display Configuration menu are always incorrect due to floating-point precision issues
https://bugs.kde.org/show_bug.cgi?id=431057 Ivan Korytov changed: What|Removed |Added CC||boot@yandex.ru -- You are receiving this mail because: You are watching all bug changes.
[systemsettings] [Bug 431057] Refresh rate values in Display Configuration menu are always incorrect due to floating-point precision issues
https://bugs.kde.org/show_bug.cgi?id=431057 --- Comment #9 from Nate Graham --- +1 for storing as fixed point with two decimal places at the very minimum. -- You are receiving this mail because: You are watching all bug changes.
[systemsettings] [Bug 431057] Refresh rate values in Display Configuration menu are always incorrect due to floating-point precision issues
https://bugs.kde.org/show_bug.cgi?id=431057 --- Comment #8 from ratijas --- I'm in favor of changing config format. Let's store framerate as a string which is a fixed point number with two decimal digits. Exactly two should be enough to distinguish any valid modes, e.g. - 640x480 @ 59.94 vs 59.93; - 1920x1080 @ 59.94 vs 60.00; note: this isn't a problem for [1280x720 @ 50.00] vs [720x576 @ 50.00], because in this case resolution is different. If there are legit two modes with same resolution but with refresh rate only diverting at or after third decimal digit, then I'd assume it doesn't matter which to pick up. > Based on the attachment, it appears that the issue is that the refresh rates > are stored as floats, like: >> "refreshRate": 120.10496520996094, >> "refreshRate": 120.00162506103516, > That's not gonna work. We are losing precision so anything that tries to > compare one to another will get a result of false. As a workaround, I could implement a fuzzy lookup, such that a mode with the closest value to the one loaded from config will be picked up, iff absolute delta is less than 1Hz. -- You are receiving this mail because: You are watching all bug changes.
[systemsettings] [Bug 431057] Refresh rate values in Display Configuration menu are always incorrect due to floating-point precision issues
https://bugs.kde.org/show_bug.cgi?id=431057 Nate Graham changed: What|Removed |Added See Also||https://bugs.kde.org/show_b ||ug.cgi?id=424997 -- You are receiving this mail because: You are watching all bug changes.
[systemsettings] [Bug 431057] Refresh rate values in Display Configuration menu are always incorrect due to floating-point precision issues
https://bugs.kde.org/show_bug.cgi?id=431057 Nate Graham changed: What|Removed |Added CC||m...@ratijas.tk Summary|Refresh rate values in |Refresh rate values in |Display Configuration menu |Display Configuration menu |are always incorrect|are always incorrect due to ||floating-point precision ||issues --- Comment #7 from Nate Graham --- Based on the attachment, it appears that the issue is that the refresh rates are stored as floats, like: > "refreshRate": 120.10496520996094, > "refreshRate": 120.00162506103516, That's not gonna work. We are losing precision so anything that tries to compare one to another will get a result of false. -- You are receiving this mail because: You are watching all bug changes.
