Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-28 Thread Philippe Makowski
Le 27/05/14 22:59, Adriano dos Santos Fernandes a écrit : At the same time, Github provides pretty powerful communication tools (eg commenting on commits, linking issues with commits, etc) + support for nicely formatted documentation straight from the repository (although this is also

[Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-27 Thread marius adrian popa
1.One of the feature that i wish from project is to migrate from svn to github (serveral projects are on github and it's is a lot easier to contribute : MariaDB , Postgresql ...) 2.Faster release cycle with less features between releases (Some of the less used features could be handled after 3.0

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-27 Thread Jim Starkey
On 5/27/2014 2:15 PM, Frank Schlottmann-Gödde wrote: On 05/27/2014 02:58 PM, marius adrian popa wrote: 1.One of the feature that i wish from project is to migrate from svn to github (serveral projects are on github and it's is a lot easier to contribute : MariaDB , Postgresql ...) Why do you

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-27 Thread Mark Rotteveel
On 27-5-2014 20:19, Dmitry Yemanov wrote: (several projects are on github and it's is a lot easier to contribute : MariaDB , Postgresql ...) How is it easier to contribute? You either have commmit rights or not. That is not true. Especially Github makes contributing a lot simpler for users

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-27 Thread Adriano dos Santos Fernandes
On 27-05-2014 17:36, Mark Rotteveel wrote: On 27-5-2014 20:19, Dmitry Yemanov wrote: (several projects are on github and it's is a lot easier to contribute : MariaDB , Postgresql ...) How is it easier to contribute? You either have commmit rights or not. That is not true. Especially Github

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-14 Thread Vlad Khorsun
10.05.2014 14:03, Vlad Khorsun wrote: As for A-B-A updates i see no problem as there will be 3 separate index entries with own tx numbers. Currently we also have 3 index entires for such scenario, btw. Old IB's optimization to not store existing key second time is not working in

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-14 Thread Dimitry Sibiryakov
14.05.2014 17:48, Vlad Khorsun wrote: 10.05.2014 14:03, Vlad Khorsun wrote: If I'm not mistaken, these repeating index entries works as a reference counters and Nope. A-A...A-B will produce two index entries Right. And it means that there are two _series_ of versions: one with A

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-13 Thread Dimitry Sibiryakov
10.05.2014 14:03, Vlad Khorsun wrote: As for A-B-A updates i see no problem as there will be 3 separate index entries with own tx numbers. Currently we also have 3 index entires for such scenario, btw. Old IB's optimization to not store existing key second time is not working in concurrent

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-13 Thread Jim Starkey
On 5/13/2014 10:59 AM, Dimitry Sibiryakov wrote: 10.05.2014 14:03, Vlad Khorsun wrote: As for A-B-A updates i see no problem as there will be 3 separate index entries with own tx numbers. Currently we also have 3 index entires for such scenario, btw. Old IB's optimization to not store

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-13 Thread Dimitry Sibiryakov
13.05.2014 17:10, Jim Starkey wrote: On 5/13/2014 10:59 AM, Dimitry Sibiryakov wrote: If I'm not mistaken, these repeating index entries works as a reference counters and allow to stop using staying list for indexes even in current code for the same reason as for blobs: three

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-13 Thread Jim Starkey
On 5/13/2014 11:24 AM, Dimitry Sibiryakov wrote: 13.05.2014 17:10, Jim Starkey wrote: On 5/13/2014 10:59 AM, Dimitry Sibiryakov wrote: If I'm not mistaken, these repeating index entries works as a reference counters and allow to stop using staying list for indexes even in current code

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-13 Thread Dimitry Sibiryakov
13.05.2014 17:43, Jim Starkey wrote: For example, when cleaning up after a crash, does the code scan all record versions to avoiding taking out a single version when the code was expecting a double entry? Nice catch. Yes, if a version don't have all index entries inserting finished,

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-13 Thread Kjell Rilbe
Jim Starkey skriver: On 5/13/2014 11:24 AM, Dimitry Sibiryakov wrote: 13.05.2014 17:10, Jim Starkey wrote: On 5/13/2014 10:59 AM, Dimitry Sibiryakov wrote: If I'm not mistaken, these repeating index entries works as a reference counters and allow to stop using staying list for indexes

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-12 Thread Molnár Attila
e.g. - Select value from two dynamic source, add them and write back. - a generic or a very dynamic code (EXECUTE STATEMENT) currenly have to store values in VARCHAR variables, because in other case (variable defined for every data type) the code would be VERY ugly. And because of this VARCHAR

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-12 Thread Vlad Khorsun
11.05.2014 16:46, Dimitry Sibiryakov wrote: 11.05.2014 10:44, Vlad Khorsun wrote: It is better to write application to test it. And application should be clever enough to be able to put already read blob_id into parameter instead of creating new blob_id. I tested with attached

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-11 Thread Vlad Khorsun
is your scenario even possible? wouldn't the engine report an update collision when update version3 was attempted? Since tx2 is read-committed and tx4 is already committed - yes, it is possible. Regards, Vlad

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-11 Thread Dimitry Sibiryakov
11.05.2014 0:18, Vlad Khorsun wrote: It is wrong for lost update scenario, which is allowed for read-committed transactions: att1: start tx1, insert version1 (blob1), commit tx1 att1: start tx2 (read-committed), read version1 (blob1) att2: start tx4, update version2 (blob2), commit

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-11 Thread Vlad Khorsun
11.05.2014 0:18, Vlad Khorsun wrote: It is wrong for lost update scenario, which is allowed for read-committed transactions: att1: start tx1, insert version1 (blob1), commit tx1 att1: start tx2 (read-committed), read version1 (blob1) att2: start tx4, update version2 (blob2), commit

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-11 Thread Dimitry Sibiryakov
11.05.2014 10:44, Vlad Khorsun wrote: It is better to write application to test it. And application should be clever enough to be able to put already read blob_id into parameter instead of creating new blob_id. I tested with attached application and got this: tx1: blob id = 80:1 tx2:

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Dimitry Sibiryakov
10.05.2014 9:03, Molnár Attila wrote: *VARIANT data type in PSQL* - gain : a little memory and/or CPU overhead but much cleaner code. Also rdb$get/set_context value colud be variant. Welcome to the hell of unpredictable type conversion. *Optimization II. *- temporal indexing of

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Molnár Attila
On 2014.05.10. 9:52, Dimitry Sibiryakov wrote: 10.05.2014 9:03, Molnár Attila wrote: *VARIANT data type in PSQL* - gain : a little memory and/or CPU overhead but much cleaner code. Also rdb$get/set_context value colud be variant. Welcome to the hell of unpredictable type

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Dimitry Sibiryakov
10.05.2014 10:12, Molnár Attila wrote: OK, this just means smaller treshold rate. But with this some case you can gain HUGE performance boost (especially when the resultset can't fit into memory and have to write to disk) Big result set is a sign of wrong application design. And in any case

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Molnár Attila
Derived table usage for this problem is GREAT idea! Why the ... I didn't tought that! Thank you! On 2014.05.10. 10:26, Dimitry Sibiryakov wrote: 10.05.2014 10:12, Molnár Attila wrote: OK, this just means smaller treshold rate. But with this some case you can gain HUGE performance boost

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Vlad Khorsun
On 5/2/2014 3:25 PM, Vlad Khorsun wrote: OK, the alternative to record lookups is to store the transaction id in index. This would require an index insertion for all indexes defined on a table even if the key value didn't change. It would also require a corresponding index deletion for each

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Dimitry Sibiryakov
10.05.2014 14:03, Vlad Khorsun wrote: For blob's we still need that lists, yes No. In my workcopy I got rid of them for blobs. The trick is not to garbage collect blobs that are referenced by potentially active versions in undo log. -- WBR, SD.

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Dimitry Sibiryakov
10.05.2014 14:50, Vlad Khorsun wrote: Explain, please, because i see no way to use undo log (which is maintained in memory for *currently* active tx) for garbage collection of record chain which primary version was created by *another* tx. It is not needed because blob id is a

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Vlad Khorsun
In short: blob id which going version is referring to can cleaned only if it is not referred by both previous or next versions. All other versions are irrelevant because they cannot refer the same blob id. At least in current codebase which creates a new copy of blob content on

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Dimitry Sibiryakov
10.05.2014 15:30, Vlad Khorsun wrote: Here you wrong: if blob contents was not changed by update, there will be no new blob. Yes, and in this case blob cannot be garbage collected because next record version refers to it. No need to analyze whole versions chain (collect staying list)

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Dimitry Sibiryakov
10.05.2014 15:30, Vlad Khorsun wrote: if blob contents was not changed by update, there will be no new blob. Imagine that we have version chain like this (only blob ids are listed): 1-2-2-3-3-3(first staying)-3-4-5-5 Because to the right from first staying version cannot be any blob id

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Adriano dos Santos Fernandes
On 10-05-2014 04:03, Molnár Attila wrote: *VARIANT data type in PSQL* This would be very easy to implement (variables only). What would be the use cases? Adriano -- Is your legacy SCM system holding you back? Join

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Jim Starkey
On 5/10/2014 10:43 AM, Adriano dos Santos Fernandes wrote: On 10-05-2014 04:03, Molnár Attila wrote: *VARIANT data type in PSQL* This would be very easy to implement (variables only). What would be the use cases? Any place you need a variable. I would argue against an explicit VARIANT type,

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Ann Harrison
On Sat, May 10, 2014 at 3:03 AM, Molnár Attila amol...@mve.hu wrote: *Optimization II. *- temporal indexing of materialization : e.g. when ORDER/GROUP BY has no index then currently the whole resultset is materialized, and the sorting moves the whole row each time. Instead of this it

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Vlad Khorsun
10.05.2014 15:30, Vlad Khorsun wrote: Here you wrong: if blob contents was not changed by update, there will be no new blob. Yes, and in this case blob cannot be garbage collected because next record version refers to it. No need to analyze whole versions chain (collect staying

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-06 Thread Dmitry Yemanov
05.05.2014 20:13, Jim Starkey wrote: AFAIR, int64 integers are not converted into double, they're compressed using their native format. If so, it's a bug. The purpose of the index encoding is that all numbers have the same representation, based on actual value rather than declaration. It

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-06 Thread Dimitry Sibiryakov
06.05.2014 10:28, Dmitry Yemanov wrote: Decreasing the scale is allowed and should work fine, AFAIU. Shouldn't it be contrary?.. If in table is value 1.234 and scale is changed from 3 to 2, this value will be displayed as 1.23, but cannot be found by eq. Vice versa it works: value 1.23

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-06 Thread Dmitry Yemanov
06.05.2014 12:35, Dimitry Sibiryakov wrote: Decreasing the scale is allowed and should work fine, AFAIU. Shouldn't it be contrary?.. If in table is value 1.234 and scale is changed from 3 to 2, this value will be displayed as 1.23, but cannot be found by eq. Vice versa it works: value

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Molnár Attila
Hello Sean! On 2014.05.01. 21:24, Leyne, Sean wrote: TRIGGER : NEW/OLD values accessible by column name, and column number (PSQL) - e.g.: NEW['id'], NEW[0] - gain : code reduction, dynamic code (don't have to alter the trigger ICO the table structure altered) 1- What advantage

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Molnár Attila
Hello Dmitry! On 2014.05.02. 8:34, Dmitry Yemanov wrote: 30.04.2014 13:50, Molnár Attila wrote: *SIZE OF CHAR/VARCHAR domain or variable name, SCALE OF NUMERIC domain or variable name* - SIZE OF : returns max CHAR/VARCHAR length or NUMERIC precision, SCALE OF : return scale of NUMERIC

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Dmitry Yemanov
05.05.2014 11:05, Molnár Attila wrote: I'm createing domains, and using TYPE OF. But as I write it's not enough. This is just for variable declaration but I need a pair in PSQL body. Maybe this example would help to understand. EXECUTE BLOCK AS DECLARE VARIABLE tmp TYPE OF COLUMN

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Molnár Attila
On 2014.05.05. 9:27, Dmitry Yemanov wrote: 05.05.2014 11:05, Molnár Attila wrote: I'm createing domains, and using TYPE OF. But as I write it's not enough. This is just for variable declaration but I need a pair in PSQL body. Maybe this example would help to understand. EXECUTE BLOCK AS

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Frank Schlottmann-Gödde
Am 05.05.2014 08:48, schrieb Molnár Attila: I believe that there are UDFs available for this already, no? A very old and unsuppoted udf. In my case I do not expect to read back from file to DB just one way. It's old but not unsupported :-) And works as expected.

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Jim Starkey
On 5/2/2014 3:25 PM, Vlad Khorsun wrote: OK, the alternative to record lookups is to store the transaction id in index. This would require an index insertion for all indexes defined on a table even if the key value didn't change. It would also require a corresponding index deletion for each

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Dmitry Yemanov
05.05.2014 19:06, Jim Starkey wrote: My sole reservation is that there can be information lost when a large valued 64 integer is converted into a double to generate the index key. AFAIR, int64 integers are not converted into double, they're compressed using their native format. My second

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Jim Starkey
On 5/5/2014 11:53 AM, Dmitry Yemanov wrote: 05.05.2014 19:06, Jim Starkey wrote: My sole reservation is that there can be information lost when a large valued 64 integer is converted into a double to generate the index key. AFAIR, int64 integers are not converted into double, they're

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Dimitry Sibiryakov
05.05.2014 17:06, Jim Starkey wrote: My third concern is that garbage collection will be very tricky, but as lone as the going/staying lists are maintained, doable. Actually, transaction numbers in nodes should allow to stop using going/staying lists among with overhead of collecting them.

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Leyne, Sean
It depends. Primary key may be stored more-or-less in regard to physical disk location. And the measure how good is INDEX vs NATURAL scan can be available to the optimizer (index clustering factor). Without the knowledge of way that Primary Keys map to storage order an assumption

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Dmitry Yemanov
03.05.2014 20:48, Leyne, Sean wrote: My statement was based on my memory of why the use of IS NOT NULL was disabled in earlier FB releases. If IS NOT NULL can be treated the same as any other condition, why isn't it now? Firebird (and many other DBMS) simply don't use indices for negated

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Dmitry Yemanov
03.05.2014 20:48, Leyne, Sean wrote: I don't see what statistics will tell the engine. Unlike index distribution statistics, which provides some details on the commonality of values, I don't see how stats on the orderedness of a primary key provide any meaningful details, when the stat is

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Dmitry Yemanov
02.05.2014 23:25, Vlad Khorsun wrote: - also it allows to have index coverage (also requires to use such index key encoding which allows to recover original value from index key) Is the problem mostly in numeric-double conversion or does compress() mangle the key unrecoverrably in any

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-03 Thread Vlad Khorsun
AFAIK, transformation of string into key is up to ICU which does not provide a way for backward transformation. INTL collations are a separate beast, I'm more interested in simpler datatypes. If we could mark at least some indices as decodable, it would already be a step forward.

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-02 Thread Dmitry Yemanov
30.04.2014 13:50, Molnár Attila wrote: *SIZE OF CHAR/VARCHAR domain or variable name, SCALE OF NUMERIC domain or variable name* - SIZE OF : returns max CHAR/VARCHAR length or NUMERIC precision, SCALE OF : return scale of NUMERIC - gain : by defining a variable with DOMAIN or TYPE

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-02 Thread Leyne, Sean
Using an index may not help: 1- an index is stored without regard to physical disk location, so using it will create a huge amount of random disk IO. Whereas a NATURAL scan follows the table. It depends. Primary key may be stored more-or-less in regard to physical disk location.

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-02 Thread Jim Starkey
On 5/1/2014 11:45 PM, Dmitry Yemanov wrote: 2- In an version based database like Firebird each row will need to be read to confirm the current value of the target field. It's not about version based databases, it's just about our index implementation. And there are possibilities to avoid

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-02 Thread Vlad Khorsun
OK, the alternative to record lookups is to store the transaction id in index. This would require an index insertion for all indexes defined on a table even if the key value didn't change. It would also require a corresponding index deletion for each index defined on the table when a

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-02 Thread Dmitry Yemanov
02.05.2014 22:03, Leyne, Sean wrote: It depends. Primary key may be stored more-or-less in regard to physical disk location. And the measure how good is INDEX vs NATURAL scan can be available to the optimizer (index clustering factor). Without the knowledge of way that Primary Keys map to

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-02 Thread Jim Starkey
On 5/2/2014 3:25 PM, Vlad Khorsun wrote: OK, the alternative to record lookups is to store the transaction id in index. This would require an index insertion for all indexes defined on a table even if the key value didn't change. It would also require a corresponding index deletion for each

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-01 Thread Leyne, Sean
TRIGGER : NEW/OLD values accessible by column name, and column number (PSQL)     - e.g.: NEW['id'], NEW[0]     - gain : code reduction, dynamic code (don't have to alter the trigger ICO the table structure altered) 1- What advantage does NEW['id'] have over the current syntax new.id??? 2-

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-01 Thread Mark Rotteveel
On 1-5-2014 21:24, Leyne, Sean wrote: Optimization I. - VOLATILE (default) / DETERMINISTIC flag for UDF and FUNCTION (maybe lifecycle for deterministic : statement, transaction, connection) - gain : preformance (can cache DETERMINISTIC UDF/FUNCTION results) Actually, I think that

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-01 Thread Mark Rotteveel
On 30-4-2014 11:50, Molnár Attila wrote: *TRIGGER : NEW/OLD values accessible by column name, and column number (PSQL)* - e.g.: NEW['id'], NEW[0] Just an aside: in SQL indexes are 1-based, not 0-based. Mark -- Mark Rotteveel

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-01 Thread Leyne, Sean
On 1-5-2014 21:24, Leyne, Sean wrote: Optimization I. - VOLATILE (default) / DETERMINISTIC flag for UDF and FUNCTION (maybe lifecycle for deterministic : statement, transaction, connection) - gain : preformance (can cache DETERMINISTIC UDF/FUNCTION results) Actually, I

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-01 Thread Dmitry Yemanov
01.05.2014 23:24, Leyne, Sean wrote: Optimization I. - VOLATILE (default) / DETERMINISTIC flag for UDF and FUNCTION (maybe lifecycle for deterministic : statement, transaction, connection) - gain : preformance (can cache DETERMINISTIC UDF/FUNCTION results) Actually, I think that

[Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-04-30 Thread Molnár Attila
Hello Dimitry! Here is my wish list. DOMAIN OF COLUMN table or view.column     - should work like TYPE OF COLUMN, but CHECK and NOT NULL constraints are checked when value assigned to this variable     - gain : performance (allows early check, error raised