Hi *! 127.13 / 3.4618 = 36,72366976717315
EXECUTE BLOCK RETURNS ( c0 NUMERIC(15,10), c1 NUMERIC(15,10), c2 NUMERIC(15,6), c3 NUMERIC(15,5), c4 NUMERIC(15,4)) AS DECLARE VARIABLE a NUMERIC(15,2); DECLARE VARIABLE b NUMERIC(15,4); BEGIN a = 127.13; b = 3.4618; c0 = a / CAST(b AS DOUBLE PRECISION);--36,7236697672, this is OK c1 = a / b;--36,7236690000, this shoud be 36,7236700000 c2 = a / b;--36,723669, this shoud be 36,723670 c3 = a / b;--36,72367, this is OK c4 = a / b;--36,7237, this is OK SUSPEND; END If return variable scale is bigger or equal than operation result scale, then computed value is truncated, but when return variable scale is smaller than operation result scale then computed value is rounded. I think in all case value should be rounded, this behaviour is inconsistent, and also truncation is unexpected behaviour. I opened a GitHub issue , but was closed : https://github.com/FirebirdSQL/firebird/issues/6928 Mark suggested to start a conversation here. #1 : This behaviour should be documented, I found nothing regarding this, and this result is unexpected. Calculation do not happens in double precision, and calculation stops when result scale is reached. #2 : Calculation method should be cofigurable #2/A: possible configuration modes - default: current mode - double precision: operands converted to double, calculations happens in double, then result rounded and converted back to result type - operand max: current calcuation method, but calcuation stops at max(op1.precision, op2.precision) instead of res.scale, then result rounded to res.scale - <fix number>: current calcuation method, but calcuation stops at <fix number> instead of res.scale, then result rounded res.scale #2/B: possible configuration places - some kind of SQL command (connection level), e.g.: SET CALCULATION PRECISION DEFAULT | DOUBLE PRECISION | OP_MAX | <fix number>. Then in a database on connect trigger can set this, when needed - new dialect (don't prefer, this just affects calculation not storage) - firebird.conf (don't prefer, this should be linked to the database file) - connection string (don't prefer, this should be linked to the database file) [cid:image002.png@01D79E83.EB3C4920]<http://www.libra.hu/> CÉGÜNK A LIBRA CSOPORT TAGJA Molnár Attila fejlesztő LIBRA Szoftver Zrt. 1113 Budapest, Karolina út 65. +36 (1) 255-3939 i...@libra.hu<mailto:i...@libra.hu> | www.libra.hu<http://www.libra.hu> __________ Information from ESET Mail Security, version of virus signature database 23884 (20210831) __________ The message was checked by ESET Mail Security. http://www.eset.com
Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
