Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Vlad Khorsun
15.02.2022 20:32, Mark Rotteveel wrote: On 2022-02-15 16:20, Vlad Khorsun wrote:   I'd vote to remove idx_numeric2 in favour of idx_decimal and look how to improve Decimal128::makeIndexKey() performance. For example, it stores every 3 decimal digits into 10 bits using relatively complex

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Mark Rotteveel
On 2022-02-15 16:20, Vlad Khorsun wrote: I'd vote to remove idx_numeric2 in favour of idx_decimal and look how to improve Decimal128::makeIndexKey() performance. For example, it stores every 3 decimal digits into 10 bits using relatively complex (computationally) algorithm with shifts and

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Dimitry Sibiryakov
Dmitry Yemanov wrote 15.02.2022 18:14: Your schema upgrade may change keys from INT to BIGINT (when tables grow more than expected) or, more usually -- NUMERIC(N, 3) is changed to NUMERIC(N, 5) for quantities, or NUMERIC(12, N) is changed to NUMERIC(18, N), etc. And it should be applied to

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Dmitry Yemanov
15.02.2022 18:37, Kjell Rilbe wrote: With respect, since I'm not in the dev team, I fail to see it as an important feature to avoid index rebuild when changing between integer and numeric column types: 1. Changing between such types must be pretty rare, at least on production databases.

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Kjell Rilbe
Den 2022-02-15 kl. 16:10, skrev Vlad Khorsun: 15.02.2022 15:20, Dmitry Yemanov wrote: ... I can think of two ways to proceed: 1) Keep rebuilding the index every time the type is changed. Cast all column values into the new format before storing keys into the index. Forget about being

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Vlad Khorsun
15.02.2022 13:28, Dmitry Yemanov wrote: All, Historically, we store most numerics as double precision inside index keys. It makes stored keys independent of the declared scale and allows both prefix and suffix compression. However, int64 keys (BIGINT and largish NUMERICs/DECIMALs) do not fit

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Vlad Khorsun
15.02.2022 15:20, Dmitry Yemanov wrote: ... I can think of two ways to proceed: 1) Keep rebuilding the index every time the type is changed. Cast all column values into the new format before storing keys into the index. Forget about being scale-independent, pick the scale from the latest

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Dmitry Yemanov
15.02.2022 14:41, Dimitry Sibiryakov wrote: Is "stored keys independent of the declared scale" a really useful and used feature? That's a separate but also good question. Double precision keys allow independence of both precision (within its range) and scale. AFAIU, the idea was to allow

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Adriano dos Santos Fernandes
On 15/02/2022 08:28, Dmitry Yemanov wrote: > > Any other comments? > Would not it be possible to use some form of varint encoding (VLQ) there? Adriano Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Dimitry Sibiryakov
Dmitry Yemanov wrote 15.02.2022 12:28: Does anyone think we should investigate this possibility more closely? Any other comments? Is "stored keys independent of the declared scale" a really useful and used feature? If we prohibit that, the index key can be a plain integer without

[Firebird-devel] INT64 and index keys

2022-02-15 Thread Dmitry Yemanov
All, Historically, we store most numerics as double precision inside index keys. It makes stored keys independent of the declared scale and allows both prefix and suffix compression. However, int64 keys (BIGINT and largish NUMERICs/DECIMALs) do not fit the double format without risk of