Obviously the limits don't match up exactly, so it depends on whether you want to express the decimal bounds in a way that captures all the possible underlying values or in a way that allows all the decimal values to be represented as the underlying type. Or more succinctly, "111,111,111,111,111.1111" will fit into a decimal64; would you prevent it from being stored in one so that you can describe the column as "decimal(18, 4)"? In any event, all scaled integer representations of decimal values have the same question and it's more-or-less orthogonal to having a decimal64 type.
On Thu, Nov 9, 2023 at 8:17 AM Raphael Taylor-Davies <r.taylordav...@googlemail.com.invalid> wrote: > Perhaps my maths is incorrect, but a decimal64 would have a maximum > precision of 18, not 19? log(9223372036854775807) = 18.9? > > On 09/11/2023 16:01, Curt Hagenlocher wrote: > > Recently, someone opened an issue on GitHub ([C++] Decimal64/32 support? > · > > Issue #38622 · apache/arrow (github.com) > > <https://github.com/apache/arrow/issues/38622>) asking for support for > > narrower decimal types. They were advised to start a thread on the > mailing > > list, and as they haven't done so yet I will start. > > > > It's fairly common to store currency in databases in a type that's > > compatible with decimal64. Both PostgreSQL and Microsoft SQL Server have > > "money" data types; for Postgres, this is a decimal(19, 2) and for SQL > > Server it's decimal(19, 4). Microsoft Analysis Services also uses > > decimal(19, 4) as one of its core data types. If you search the internet > > for suggestions on the database type to use for money, the vast majority > > recommend a decimal type with a precision <= 19. Currency is something > > stored very frequently as data, and it makes sense to have a type that's > > optimized for this purpose. I submit that it's a far more common type > than > > float16, and that even if it's not as hip as the AI scenarios which > > popularized float16, the ultimate goal of those scenarios is, after all, > to > > make more "money". > > > > decimal64 is considerably easier to work with on modern CPUs and in > common > > programming languages than decimal128, and requires half the amount of > > storage space. And while adding new types to Arrow obviously needs to be > > done very sparingly, it's harder to imagine a new type for which support > > would be easier to implement than this one. > > > > I think decimal32 is much harder to justify. MS SQL Server has a > > "smallmoney" (decimal(10, 4)), but I suspect it's not that heavily used. > > Maybe others have more feedback on this one. > > > > > > -Curt > > >
