Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Leyne, Sean
It depends. Primary key may be stored more-or-less in regard to physical disk location. And the measure how good is INDEX vs NATURAL scan can be available to the optimizer (index clustering factor). Without the knowledge of way that Primary Keys map to storage order an assumption

[Firebird-devel] Questions on External Sort logic

2014-05-03 Thread Leyne, Sean
All, I have been thinking about some possible performance improvements to the way that External Sorts are handled but want to be sure I understand the current logic. Say I submit a query which will return a row set which will be larger than the defined internal sort size (even if server has

Re: [Firebird-devel] Questions on External Sort logic

2014-05-03 Thread Dmitry Yemanov
03.05.2014 21:32, Leyne, Sean wrote: So, here are my questions: 1. A query’s operations (the reading of the data and sorting) execute in a single thread/process, correct? Correct. 2. Is Internal sorting performed by the ExecuteWriteTempSortBlock() step? Internal buffer is sorted as

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Dmitry Yemanov
03.05.2014 20:48, Leyne, Sean wrote: My statement was based on my memory of why the use of IS NOT NULL was disabled in earlier FB releases. If IS NOT NULL can be treated the same as any other condition, why isn't it now? Firebird (and many other DBMS) simply don't use indices for negated

Re: [Firebird-devel] Questions on External Sort logic

2014-05-03 Thread Vlad Khorsun
3. Does sorting use pointers to row sort data or are the rows moved in memory during sort? Rows are moved. This is done this way to avoid duplicated record fetches (when sort results are retrieved) causing random I/O. Sorting itself may be also highly I/O bound, but it always operates

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Dmitry Yemanov
03.05.2014 20:48, Leyne, Sean wrote: I don't see what statistics will tell the engine. Unlike index distribution statistics, which provides some details on the commonality of values, I don't see how stats on the orderedness of a primary key provide any meaningful details, when the stat is

Re: [Firebird-devel] Questions on External Sort logic

2014-05-03 Thread Dmitry Yemanov
03.05.2014 22:12, Vlad Khorsun wrote: Internal sort (quicksort, btw) does not move records, only pointers. Records are re-ordered in sort order before full run is written to disk (if necessary). Sorry, I was talking about external sorting in general, not quicksorting the internal buffer.

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Dmitry Yemanov
02.05.2014 23:25, Vlad Khorsun wrote: - also it allows to have index coverage (also requires to use such index key encoding which allows to recover original value from index key) Is the problem mostly in numeric-double conversion or does compress() mangle the key unrecoverrably in any

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Vlad Khorsun
AFAIK, transformation of string into key is up to ICU which does not provide a way for backward transformation. INTL collations are a separate beast, I'm more interested in simpler datatypes. If we could mark at least some indices as decodable, it would already be a step forward.

Re: [Firebird-devel] Questions on External Sort logic

2014-05-03 Thread Leyne, Sean
Rows are moved. This is done this way to avoid duplicated record fetches (when sort results are retrieved) causing random I/O. Sorting itself may be also highly I/O bound, but it always operates quite large blocks. Internal sort (quicksort, btw) does not move records, only pointers.