Precision often means different things to different people. Here is my take:
- Floating point numbers have almost always rounding errors. And floating point numbers use binary fractions, a decimal fraction like 1.0 has no precise representation as a binary fraction. Type 0.1 + 0.2 into python or haskell or any other language that gives you bare floating point numbers and enjoy the result. - Fixed precision decimal numbers do not have rounding errors since they are essentially scaled integers and hence they are precise as long as calculations stay within the range. - Floating point numbers can cover a large number space (from very tiny to really big), fixed precision decimal numbers are much more restrictive. - In XML and JSON, numbers are rendered in strings that likely do not look much different if its a decimal64 or a float or ... If you really care about size, use a binary encoding like CBOR. /js On Tue, Jul 07, 2020 at 07:06:20AM -0400, Christian Hopps wrote: > I received feedback in my YANG doctor review (thanks Mahesh) regarding the > use of decimal64 for most of the values in the geo location grouping > (https://tools.ietf.org/html/draft-ietf-netmod-geo-location-04). In my > comparison sections I note that some precision (at the very extremes) may be > lost when converting from other geo location formats that use string (or > double for w3c) to decimal64. Given that mention of loss of extreme > precision, the reviewer was asking if some justification for the decimal64 > should be given in the document. > > What are the advantages to using decimal64 for floating point numbers vs > using a string with a pattern "[0-9]+(\.[0-9]+)?" (convert that to yang > pattern language). The advantage of using a string is that the precision of > the value is not restricted by the model. Does the YANG decimal64 values have > a concise binary format that can be more efficiently transported or stored in > binary form? If so is this the only advantage, and is it enough of one to > limit the precision in the model? > > It's definitely worth noting that the precision of the decimal64 values seem > vastly adequate for geo location data (e.g., for Cartesian coordinates and > height values which are measured in meters the fractional digits is 6 which > means the surface could be up to 9 billion kilometers large (or away from for > height) and the precision is to the micrometer. For ellipsoidal coordinates > there are 12 fractional digits for the degrees. > > Thanks, > Chris. > _______________________________________________ > netmod mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/netmod -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1 | 28759 Bremen | Germany Fax: +49 421 200 3103 <https://www.jacobs-university.de/> _______________________________________________ netmod mailing list [email protected] https://www.ietf.org/mailman/listinfo/netmod
