[
https://issues.apache.org/jira/browse/CASSANDRA-9224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14545142#comment-14545142
]
Stefania commented on CASSANDRA-9224:
-------------------------------------
The best I could figure out so far is to increase the precision to take into
account the digits on the left of the decimal separator when '%g' won't use
scientific notation. To work this out we can follow the rule described in the
documentation
(https://docs.python.org/2/library/string.html#format-specification-mini-language):
if {{-4 <= exponent < precision}} then %g will not not use scientific
notation. If this is true, it will still be true with a bigger precision. So
when {{-4 <= exponent < precision}} is true, we add to the precision the number
of digits to the left (exponent + 1), therefore ensuring we always have
precision digits after the decimal separator.
> Figure out a better default float precision rule for cqlsh
> ----------------------------------------------------------
>
> Key: CASSANDRA-9224
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9224
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Reporter: Tyler Hobbs
> Assignee: Stefania
> Labels: cqlsh
> Fix For: 3.x, 2.1.x
>
>
> We currently use a {{DEFAULT_FLOAT_PRECISION}} of 5 in cqlsh with formatting
> {{'%.*g' % (float_precision, val)}}. In practice, this is way too low. For
> example, 12345.5 will show up as 123456. Since the float precision is used
> for cqlsh's COPY TO, it's particularly important that we maintain as much
> precision as is practical by default.
> There are some other tricky considerations, though. If the precision is too
> high, python will do something like this:
> {noformat}
> > '%.25g' % (12345.5555555555555555,)
> '12345.55555555555474711582'
> {noformat}
> That's not terrible, but it would be nice to avoid if we can.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
