[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890801#comment-16890801 ]

Liudmila Kornilova commented on CASSANDRA-15232: ------------------------------------------------ This is how postgres calculates scale https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/numeric.c#L7800 Before this commit Cassandra used to limit precision off all operations to 34 (maximum number of significant digits) After this commit: * add, multiply and subtract operations limit precision to 10000 digits * add, multiply and subtract trim trailing zeros, so operations do not affect scale/precision of next operations * division sets minimum scale to 32. So 1000 / 3 produces 333.33333333333333333333333333333333 (32 digits after dot) * division also tries to guess the position of first significant digit in result (this idea is taken from postgres) so 0.00001 / 3 will produce 5+32=37 digits after dot, 32 of them will be significant * division sets maximum scale to 1000 (the constant is also taken from postgres) * division will not use scale that is lower that scale of any operands The PR is not 100% ready because mod operation needs to be implemented > Arithmetic operators over decimal truncate results > -------------------------------------------------- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics > Reporter: Benedict > Assignee: Liudmila Kornilova > Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org