Re: [Firebird-devel] Support for timed-zones datatypes

[Dmitry Yemanov]

21.11.2017 17:55, Adriano dos Santos Fernandes wrote:


Implementing TIME WITH TIME ZONE and TIMESTAMP WITH TIME ZONE datatypes
as specified in the SQL standard is not a big problem.

However, standard says that CURRENT_TIME / CURRENT_TIMESTAMP returns the
timed-zone datatype.

That causes two problems:
- Internally (say in PSQL), conversion to string would return different
result than now
- Externally, tools will take time to support this datatype


Externally, all the apps use SQL_TIME/SQL_TIMESTAMP and I expect TZ 
values to be implicitly casted (coerced) to non-TZ data types when 
dealing with existing apps. Surely, proper support of TZ values will 
require time/efforts, but at least nothing will be broken (even if 
CURRENT_TIME/TIMESTAMP are used).


So only conversion to string seems to be a real problem. And IMHO we 
could live with that.


Session-level option comes to mind, but this new feature will require 
one new session-level option anyway (client TZ) and I'd rather avoid 
having one more option just to please time->string conversions.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] isc_info_svc_timeout incompatibility between Interbase and Firebird

[Dmitry Yemanov]

04.10.2017 17:48, Dimitry Sibiryakov wrote:


   In Interbase isc_info_svc_timeout is used like this:


*spb++ = isc_info_svc_timeout;
ADD_SPB_NUMERIC(spb, 60); /* 1 minute timeout */


   In Firebird it is used like this:


*p++ = isc_info_svc_timeout;
ADD_SPB_LENGTH(p, 4);
ADD_SPB_NUMERIC(p, 1);


   Is this incompatibility intended?


Different SPB version perhaps?


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Default port on client side

[Dmitry Yemanov]

27.09.2017 16:05, Roman Simakov wrote:


I noticed that client uses RemoteServicePort parameter from
firebird.conf if it's not specified in connection string. Is it by
design?


Yes, IIRC since Interbase times.


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5611) Higher memory consumption for prepared statements in FB3

[Dmitry Yemanov (JIRA)]

Higher memory consumption for prepared statements in FB3


 Key: CORE-5611
 URL: http://tracker.firebirdsql.org/browse/CORE-5611
 Project: Firebird Core
  Issue Type: Bug
  Components: Engine
Affects Versions: 4.0 Alpha 1, 3.0.2, 3.0.1, 3.0.0, 4.0 Initial
Reporter: Dmitry Yemanov


Customer has reported about noticably higher memory consumption with FB 3.0 
than they see with FB 2.5. This is somewhat expected, given the metadata cache 
is per-attachment in FB3 and their production database is complex. But then 
Paul Reeves and I were able to reproduce this with TPCC which database has just 
a few tables and procedures, so the metadata cache should not be a big issue.

This is what I've got looking at mon$memory_usage (values are avg/max):

v2.5 SS
---
database: 22MB
attachment: 150KB
transaction: 1.7KB / 18KB
statement: 25KB / 90KB

v3.0 SS
---
database: 160MB
attachment: 2.4MB
transaction: 1.1KB / 13KB
statement: 88KB / 154KB

Database counter is aggregated among all attachments, it doesnt show where the 
problem is, but it does show that the problem exists. Attachment counter 
clearly shows noticably higher memory usage, but we still cannot say for sure 
whether it's effect of the metadata caching or not. Transaction counter show 
only a little increase, it shouldn't be a problem. Finally, statement counter 
does indicate a problem. We see a 2x-3x more memory used for DSQL statements in 
v3.0 as compared to v2.5.

Quick debugging shows that even trivial statements like "execute procedure X 
(A, B, C)" consume 3x more memory: 10KB vs 32KB (with the same impure size). 
Here are examples from the memory pool dumps for the same statements:

v2.5
0x72b5e530 USED: size=80 allocated at ../src/jrd/par.cpp:686
0x72b5e5a0 USED: size=96 allocated at ../src/jrd/par.cpp:686
0x72b5e620 USED: size=80 allocated at ../src/jrd/par.cpp:686
0x72b5e690 USED: size=72 allocated at ../src/jrd/par.cpp:686
0x72b5e6f8 USED: size=80 allocated at ../src/jrd/par.cpp:686
0x72b5e768 USED: size=80 allocated at ../src/jrd/par.cpp:686
0x72b5e7d8 USED: size=80 allocated at ../src/jrd/par.cpp:686
0x72b5e848 USED: size=80 allocated at ../src/jrd/par.cpp:686
0x72b5e8b8 USED: size=72 allocated at ../src/jrd/par.cpp:686
0x72b5e920 USED: size=80 allocated at ../src/jrd/par.cpp:686

v3.0
USED 0x7fffe3405d20: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe3405d58: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe3405d90: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../common/classes/array.h:464
USED 0x7fffe3405dc8: size=232 allocated at /work/v3-release/src/jrd/par.cpp:1234
USED 0x7fffe3405eb0: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe3405ee8: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe3405f20: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../common/classes/array.h:464
USED 0x7fffe3405f58: size=232 allocated at 
/work/v3-release/src/dsql/ExprNodes.cpp:7783
USED 0x7fffe3406040: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe3406078: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe34060b0: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../common/classes/array.h:464
USED 0x7fffe34060e8: size=232 allocated at 
/work/v3-release/src/dsql/ExprNodes.cpp:7795
USED 0x7fffe34061d0: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe3406208: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe3406240: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../common/classes/array.h:464
USED 0x7fffe3406278: size=232 allocated at /work/v3-release/src/jrd/par.cpp:1230
USED 0x7fffe3406360: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe3406398: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../jrd/../jrd/../dsql/Nodes.h:681
USED 0x7fffe34063d0: size=56 allocated at 
/work/v3-release/src/jrd/../jrd/../common/classes/array.h:464
USED 0x7fffe3406408: size=232 allocated at /work/v3-release/src/jrd/par.cpp:1234

Besides the fact that the execution tree node itself is bigger in v3.0 (e.g. 
232 vs 80 bytes), we also see 3 extra allocations by 56 bytes that didn't exist 
in v2.5. This makes a huge total difference. Given that the execution trees 
exist not only for DSQL statements but also for every cached PSQL object, this 
issue affects also the metadata cache thus making per-attachment memory usage 
even bigger.

I can see several reasons:
1) Every node now contains vtable wi

Re: [Firebird-devel] Lateral derived tables

[Dmitry Yemanov]

08.09.2017 16:55, Mark Rotteveel wrote:



%type  table_primary
table_primary
 : table_proc
 | derived_table
 | '(' joined_table ')'
 ;

IMHO, it wouldn't look logical to support LATERAL for "table_proc" but 
disallow it for "joined table".


But LATERAL **is** a table_primary, it doesn't **use** table_primary (at 
least not until you dive down into the select_expr), the rule would become


I understood that you suggested to allow "LATERAL table_proc" (for 
procedures). "LATERAL derived_table" is by standard, no questions here. 
But you suggested to disallow "LATERAL ( joined_table )". This is what I 
see as inconsistent. I'd rather disallow LATERAL for both non-standard 
rules, or allow it for both.



%type  table_primary
table_primary
 : table_proc
 | derived_table
 | lateral_derived_table
 | '(' joined_table ')'
 ;


This is how I see it too.

Note that this may be an indication that I need to revise my opinion on 
point 3 though.


It seems so ;-)


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Lateral derived tables

[Dmitry Yemanov]

08.09.2017 15:31, Mark Rotteveel wrote:


Isn't this all contained in the SQL specification?


Reading the spec may differ. And we don't follow the standard strictly 
sometimes.



3) LATERAL was historically implied for joined stored procedures, e.g.


I suggest 3b


4) LATERAL in nested parenthesis joins, e.g.


Not allowed by the syntax construction in the SQL specification:


In our parser:

%type  table_primary
table_primary
: table_proc
| derived_table
| '(' joined_table ')'
;

IMHO, it wouldn't look logical to support LATERAL for "table_proc" but 
disallow it for "joined table".



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] Lateral derived tables (was: Plans to support cross and outer apply)

[Dmitry Yemanov]

All,

The key point of this standard feature is to allow sub-queries to 
reference priorly defined contexts (in joins).


While thinking about this, I have a few questions to raise here. The 
standard defines LATERAL for derived tables only. This sounds logical 
but there are some corner cases to be discussed.


1) LATERAL used when it doesn't make sense, e.g.

-- single context
FROM LATERAL (SELECT ...)

-- first context
FROM LATERAL (SELECT ...) JOIN T ...

In both cases, there's nothing priorly defined to be referenced to.

a) should be prohibited syntactically
b) should raise error at prepare
c) should raise warning at prepare
d) should be silently ignored

2) LATERAL does not reference prior contexts

-- T is not referenced from inside the derived table
FROM T JOIN LATERAL (SELECT ...)

a) should raise error at prepare
b) should be ignored, but join order must be enforced (derived table is 
executed per every row of T)

-- perhaps with a warning?
c) should be ignored, join order can be changed by the optimizer
-- perhaps with a warning?

3) LATERAL was historically implied for joined stored procedures, e.g.

FROM T JOIN P(T.ID) ON 1=1

It used to produce invalid plan (and runtime errors) in older versions 
but FB3 handles it properly by executing P after reading T.


a) keep status quo, disallow LATERAL syntax for procedures
b) keep status quo, allow LATERAL for procedures as syntax noise 
(LATERAL is implied even if missing)
c) allow LATERAL for procedures, allow external references via 
parameters only if LATERAL is specified (thus breaking legacy behaviour)
d) disallow LATERAL for procedures, require to be rewritten as 
standard-friendly (thus breaking legacy behaviour):


FROM T JOIN LATERAL (SELECT ID FROM P(T.ID))

4) LATERAL in nested parenthesis joins, e.g.

FROM T1 JOIN LATERAL (T2 JOIN T3 ON T1.ID1 = T2.ID2 AND T2.ID3 = T3.ID3) 
ON TRUE


a) should be allowed
b) should not be allowed

Opinions, please.

My personal votes: 1d, 2c, 3a, 4b

BTW, I see no way to allow LATERAL in RIGHT or FULL outer joins, so it 
should raise error.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird 3, stored procs and plans

[Dmitry Yemanov]

25.08.2017 13:06, Paul Reeves wrote:


Is there a simple hack I could do to build FB3.0 with the plan enabled
for SPs?


Not so simple, as the code was completely refactored. I will try to 
provide you with a patch during the next days.



But whatever happened to the SP debugger?


AFAIK, it was never started (although was discussed a bit).


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird 3, stored procs and plans

[Dmitry Yemanov]

25.08.2017 11:38, Paul Reeves wrote:


In FB 3.0 the plan output for stored procs is always PLAN (NATURAL)
whereas with FB 2.5 the plan for each sql statement to be executed
within the SP are returned.

Not having the plan available in FB 3.0 makes it quite difficult to see
what is actually happening in the SP. However, if we take any SP source,
remove the 'create proc myproc' and insert 'execute block' and then
prepare it with input param markers the plan (or plans) is/are actually
returned.

So obviously this information is available internally. How come it is
supressed for SPs in FB 3.0? Is this a bug?


It was suppressed intentionally.

First of all, there's no such thing as SP plan -- the output is purely 
artificial, listing all the access paths inside SP. But different code 
paths may be executed depending on parameters, so that "plan" becomes 
somewhat useless (except when you want something as simple as finding 
the unexpected "NATURAL" there).


Second, you didn't get all the access info anyway if your SP calls (or 
selects from) another SPs. Only the first level was printed in the plan, 
you need to dig deeper manually.


Then, the plan becomes unreadable if the SP sub-plan is injected in the 
middle of the outer plan (e.g. join of procedure and table -- it's hard 
to get where that table belongs to). There more complex is the outer 
query the less sense we have in "embedded" SP plans there.


Finally, if applied to the "explained plan" feature, its verbosity 
becomes overcomplicated.


EXECUTE BLOCK shares some of these issues with SP, but its internal 
implementation is different so it wasn't affected by the change.


I agree it was handy in simple cases, but it had too many conceptual 
problems. I believe we need some other tool for this purpose, maybe 
INSPECT or ANALYZE command for PSQL objects. Any better idea?



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird 4.0 Alpha 1 release is available for testing

[Dmitry Yemanov]

24.08.2017 23:34, livius wrote:


However i see that context limit is "the same" for the single query
Is this really plan for Firebird4 or it is postponed?


It will be removed in the next Alpha/Beta release.


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] block size exceeds implementation restriction at JStatement::openCursor

[Dmitry Yemanov]

Post a tracker entry to have it fixed. Or create an index for joining.Dmitry18:35, 24 августа 2017 г., "Jiří Činčura" : What is the plan? I suspect some hardcoded limit for the hash join  algorithm has been reached. Do you see HASH in the plan?Here's the plan:PLAN HASH (EMTD NATURAL, JOIN (SORT (DR DR NATURAL), EWR INDEX(IDX_EWR_SENT_A_TEMP)))Could I somehow confirm it's this? And more importantly how can Iprevent falling into this trap? :)-- Mgr. Jiří Činčurahttps://www.tabsoverspaces.com/--Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org! http://sdm.link/slashdotFirebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel-- Отправлено из мобильного приложения Яндекс.Почты--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdotFirebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] block size exceeds implementation restriction at JStatement::openCursor

[Dmitry Yemanov]

24.08.2017 17:50, Jiří Činčura wrote:


I have a query that on one particular database throws:

2017-08-24T16:29:11.5140 (2704:01620040) ERROR AT
JStatement::openCursor
 I:\DOWNLOADS\BI2.FDB (ATT_16, SYSDBA:NONE, NONE,
 TCPv6:::1/51084)
 
C:\Users\Jiri\Documents\devel\bin\Firebird-3.0.2.32703-0_x64\isql.exe:7116
335544381 : Implementation limit exceeded
335544388 : block size exceeds implementation restriction

On other databases (same structure, same plan produced) on the same
server (different data of course) it works fine. Backup & restore didn't
help. I can provide the database, but given it's a customer data it
needs to be handled as private data.

The minimum query I can replicate it looks like this:
with DAYS
as (select distinct(DR.F_DA_DATE)
 from T_DAILY_RAW DR
)
select count(*)
from T_EMAILS_WEBDB_RAW EWR
inner join DAYS DR on DR.F_DA_DATE = EWR.F_EWR_SENT
inner join T_EMAIL_TYPE_DIM EMTD on EMTD.F_EMT_VALUE = EWR.F_EWR_TYPE;


What is the plan? I suspect some hardcoded limit for the hash join 
algorithm has been reached. Do you see HASH in the plan?



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] Firebird 4.0 Alpha 1 release is available for testing

[Dmitry Yemanov]

Firebird Project announces the first Alpha release of Firebird 4.0, the
next major version of the Firebird relational database, which is now
available for testing.

This Alpha release arrives with an early preview of the features and
improvements currently under development by the Firebird development
team, as well as with countless bugfixes. Our users are appreciated
giving it a try and providing feedback to this mailing list. Apparent
bugs can be reported directly to the bugtracker.

Alpha releases are not encouraged for production usage or any other
goals that require a stable system. They are, however, recommended for
those users who want to help in identifying issues and bottlenecks thus
allowing to progress faster through the Beta/RC stages towards the final
release.

Please read the Release Notes carefully before installing and testing
this Alpha release.


Download page:
http://www.firebirdsql.org/en/firebird-4-0-0-alpha1/

Release Notes:
http://web.firebirdsql.org/downloads/prerelease/v40alpha1/Firebird-4.0.0_Alpha1-ReleaseNotes.pdf


--
Dmitry Yemanov

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] 3.0.2 consistent crash

[Dmitry Yemanov]

19.08.2017 16:06, Jiří Činčura wrote:


Running it (3.0.3.32798) for about an hour now. So far so good.

Sounds good ;)


So far it survived Friday night (usually a busy day in entertainment
business), waiting for Saturday night. Fingers crossed. Quite scary to
run on random snapshot build on production. :)

BTW is there a ETA for 3.0.3?


September-October, I suppose.


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Alpha tag and mandatory features

[Dmitry Yemanov]

16.08.2017 16:51, Dimitry Sibiryakov wrote:


I see that Alpha tag is already set but I cannot find in master any 
mention of replication (neither in docs nor in sources), which was 
supposed to be a mandatory feature and thus available for at least 
preview in Alpha version.


Some mandatory features will be introduced in the next testing release.


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Release build vs snapshot build

[Dmitry Yemanov]

15.08.2017 14:30, Gabor Boros wrote:


If an important bug fixed between releases, no other choice than switch 
to a snapshot. But I am not too brave to use a snapshot in production 
because built with different compiler than the release build. Or not? 
For example msvc*.dll files are different in the Win* release and the 
snapshot kits.


AFAIR, this happens only with v2.5 and only for Windows, no other 
releases were/are affected. And snapshots are built using a slightly 
newer (read: patched, service pack) compiler, so theoretically should be 
more robust. You may need to take care about the new MSVC runtime 
dependency though.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Changing numerics scale

[Dmitry Yemanov]

10.08.2017 21:57, Carlos H. Cantu wrote:


DY> e.g. 1234567890.1234 is a valid (15, 4) but cannot be converted to (15, 6).

Are you sure? I can store such value both in (15, 4) as well in (15, 6).


Formally, it cannot be stored inside (15, 6). But historically, FB 
ignores the declared precision and always uses the maximum for the 
underlying storage. This is buggy (from the SQL spec POV) and I expect 
to have it fixed/improved one day.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Changing numerics scale

[Dmitry Yemanov]

10.08.2017 20:29, Carlos H. Cantu wrote:


Can someone explain why I can't change a field from numeric (15,4) to
numeric (15,6) [error is "New scale specified for column TESTE must be
at most 4."], but can change it to numeric (17,6) ?


Because (15, 6) provides less integral precision (9 digits instead of 
11) and theoretically some existing values may not fit, making those 
records unreadable.


e.g. 1234567890.1234 is a valid (15, 4) but cannot be converted to (15, 6).


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] HASH function (CORE-4436)

[Dmitry Yemanov]

19.07.2017 17:44, Leyne, Sean wrote:


Why do we need to extend the current function?

Why not create separate, built-in, functions for each hash type with names* 
that align with the common algorithm name?

MD2()
...
MD5()
SHA0()
SHA1()
SHA_224()
...
SHA512_256()
...
SHA3_224()
...
SHA3_512()


Too many new keywords to be reserved.


(would save needing to look at documentation to determine the supported 
algorithms)?


HASH(X USING MD5) is self-documentary as well.


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] HASH function (CORE-4436)

[Dmitry Yemanov]

19.07.2017 01:32, Adriano dos Santos Fernandes wrote:




   Algorithm name could define result length.


Yes, for constants.


We need to decide whether the algorithm name can be passed dynamically 
(and thus be presented as "value" in the grammar) or must be predefined 
(via a string literal or maybe token). The latter gives us more 
flexibility regarding the result type. Tricks with runtime type coercion 
are possible but require careful programming at the user side (proper 
CASTs must present), I'd rather avoid that.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] HASH function (CORE-4436)

[Dmitry Yemanov]

18.07.2017 22:00, Leyne, Sean wrote:


Would this approach have any performance advantages over using UDFs?


People hate writing UDFs for common tasks. And IMHO getting a robust 
hash belongs to this category.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] HASH function (CORE-4436)

[Dmitry Yemanov]

18.07.2017 21:55, Adriano dos Santos Fernandes wrote:


We have HASH function that returns a 64 bit integer. The algorithm is
bad and the result too small for a hash.


Yes, and we have CORE-4436 with related complains.


I propose to extend the same function with a second parameter with the
algorithm name.


Agreed.


When two parameters are passed, it will return a VARCHAR(64) CHARACTER
SET OCTETS. That's sufficient for a SHA-256, for example.


Would it make sense to reserve more bytes, 256 or 1024 octets for 
example? Just to avoid extending the result every five years...



We implement (preferably getting code with compatible license or public
domain - like was done for SHA-1 used in authentication - instead of use
a bloated library) some algorithms (MD5, some useful SHA-*). No need for
"hash plugins" for now.


Agreed.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Removal of whitespace in changelog and readme

[Dmitry Yemanov]

17.07.2017 10:48, Mark Rotteveel wrote:


I just came across this commit by Adriano:

https://github.com/FirebirdSQL/firebird/commit/c2584cf84be78f4aa157f003445dd17c8bc52c3f 



It removes a number of double-whitespaces in the CHANGELOG.md and 
README.md. Double whitespace at the end of a line has meaning in 
markdown: it introduces an explicit linebreak, otherwise the linebreak 
is just non-linebreak whitespace when rendered.


In my opinion this commit breaks the formatting and should be reverted.


Agreed. I will let Adriano to revert this himself.


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Detecting connection is using wire encryption

[Dmitry Yemanov]

16.07.2017 13:31, Mark Rotteveel wrote:


That said, I would still like to know if there is a specific info item I 
can use as well.


There's no such an item, AFAIK.


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] PORT_symmetric

[Dmitry Yemanov]

14.07.2017 04:36, Adriano dos Santos Fernandes wrote:


When client and server "architecture" are identical, remote code set
PORT_symmetric flag and that causes some optimizations.


That happened in the past. Asymmetric mode is unconditionally used since 
v1.5, in order to optimize bandwidth (VARCHAR packing). It could make 
sense to use PORT_symmetric for XNET though, worth testing.



But, architectures are:

#if defined(__sun)
#   ifdef sparc
const P_ARCH ARCHITECTURE   = arch_sun4;
#elif (defined i386 || defined AMD64)
const P_ARCH ARCHITECTURE   = arch_sunx86;
#   else
const P_ARCH ARCHITECTURE   = arch_sun;
#   endif
#elif defined(HPUX)
const P_ARCH ARCHITECTURE   = arch_hpux;
#elif (defined AIX || defined AIX_PPC)
const P_ARCH ARCHITECTURE   = arch_rt;
#elif defined(LINUX)
const P_ARCH ARCHITECTURE   = arch_linux;
#elif defined(FREEBSD)
const P_ARCH ARCHITECTURE   = arch_freebsd;
#elif defined(NETBSD)
const P_ARCH ARCHITECTURE   = arch_netbsd;
#elif defined(DARWIN) && defined(__ppc__)
const P_ARCH ARCHITECTURE   = arch_darwin_ppc;
#elif defined(WIN_NT) && defined(AMD64)
const P_ARCH ARCHITECTURE   = arch_winnt_64;
#elif defined(I386)
const P_ARCH ARCHITECTURE   = arch_intel_32;
#elif defined(DARWIN64)
const P_ARCH ARCHITECTURE   = arch_darwin_x64;
#elif defined(DARWINPPC64)
const P_ARCH ARCHITECTURE   = arch_darwin_ppc64;
#elif defined(ARM)
const P_ARCH ARCHITECTURE   = arch_arm;
#endif

But for the given purpose, this is misleading. Each platform/OS is
treated in a different way. Why Linux is treated in the same manner in
all platforms (big and little endian)?


This question should be addressed to the Borland engineers, I suppose. 
Maybe Ann may recall and explain something. I agree the logic is confusing.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Result set from where afield = afield

[Dmitry Yemanov]

10.07.2017 15:11, Paul Reeves wrote:


I've also seen, in stored procedures, this sort of construct...

   where (( e.EMP_NO = :AEMP_NO  ) OR ( :AEMP_NO IS NULL))

which, if the input parameter AEMP_NO is NULL will also behave
as if a full result set was requested. ie, the stored procedure will
return a single record if AEMP_NO has a value and the entire table if
it is null. The advantage of using that sort of construct is that only
one SQL statement is required, rather than a more complex 'if then
else' but is it safe from future implementation changes ?


Well-known trick and FB3 supports it at the runtime/optimizer level 
(using either index scan or full scan depending on parameter value).



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Optimisation of where afield = coalesce(value1, value2 )

[Dmitry Yemanov]

10.07.2017 14:20, Paul Reeves wrote:


I can understand that this plan might appear to be invalid from the
perspective of the optimiser. But surely the whole point of adding the
PLAN clause is because I think I know better than the optimiser what I
want.


Unless the engine physically cannot execute your plan. This was exactly 
the case.



So to return to the original query:

   where e.EMP_NO = COALESCE( ?, e.emp_no )

surely I ought to be able to add

   PLAN (E INDEX (RDB$PRIMARY7))

Then if the parameter resolves to a value at run-time the index will be
used. Obviously if the parameter is null performance would be much
worse than a natural scan because the engine must walk the index pages
and then walk the data pages. But the customer is always right,
surely :-)


You seem to believe that the engine can execute the INDEX plan without 
lower/upper bounds (which require e.emp_no to be known in advance). In 
theory, it could. In practice, it's pointless and thus prohibited by the 
optimizer.



Why not create a second stream in that case?

that seems to be what happens when I create this query:

SELECT  *
FROM employee
WHERE
  EMP_NO = 2
OR
  EMP_NO is null

this plan is chosen...

   PLAN (EMPLOYEE INDEX (RDB$PRIMARY7, RDB$PRIMARY7))


This is still a single stream and bitmap based on two index scans.

Solution with two streams (chosen conditionally at runtime) is 
implemented in FB3 but only for some specific syntax. Perhaps your 
COALESCE trick could join this special category, but it doesn't yet.



Because there are two streams the fetches and the indexed reads more or
less double because of the OR clause.

(Again, a subject for another day is why does the optimiser even bother
with the second stream when EMP_NO can never be null ? )


The optimizer don't take NOT NULL constraints into account. They don't 
have existence locks and thus can affect query results if dropped in the 
meantime.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Optimisation of where afield = coalesce(value1, value2 )

[Dmitry Yemanov]

10.07.2017 12:17, Paul Reeves пишет:

On Fri, 7 Jul 2017 18:07:55 +0300 Dmitry Yemanov wrote


07.07.2017 17:51, Paul Reeves wrote:


I understand that evaluating COALESCE(?, e.emp_no ) at prepare time
may require a circular logic and is thus impractical


It cannot be done at runtime either (without deducting that e.emp_no is 
the same in both paths of the condition).



but surely
the more important piece of information is in the

   where e.emp_no =

A unique index exists on this column so surely the optimiser should
choose it instead of a natural scan ?


No, it cannot. Imagine "where T.A = T.B", it cannot use an index for 
either A or B. Only full table scan is possible.



The other question I asked was...

 why can't I force the plan...
   
   PLAN (E INDEX (RDB$PRIMARY7))


Because it's invalid in this case.


What are the rules for deciding when a PLAN statement will be rejected?

From one logical point of view if have this sort of query


   select * from mytable m where afield = whatever;

and I add

   plan (m index(myindex ))

the engine should accept that. Shouldn't it?


It depends on "whatever". If it's literal or independent expression or 
priorly evaluated field, index scan can be used. If it includes a field 
from the same stream, it cannot.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Optimisation of where afield = coalesce(value1, value2 )

[Dmitry Yemanov]

07.07.2017 18:26, Adriano dos Santos Fernandes wrote:


BTW, isn't ConditionalStream used for something in this field?


Yep, but the optimizer so far handles just one specific case (A = ? or ? 
is null). It could be extended though.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Optimisation of where afield = coalesce(value1, value2 )

[Dmitry Yemanov]

07.07.2017 18:12, Dimitry Sibiryakov wrote:


In this particular case it is enough to know parameter value to 
choose plan. Parameters are known before reading table.


True, but our optimizer is developed for generic cases, not such 
specific ones. It could be improved, but I'd say we have more important 
issues to solve there.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Optimisation of where afield = coalesce(value1, value2 )

[Dmitry Yemanov]

07.07.2017 17:51, Paul Reeves wrote:


But that doesn't answer all my questions...

Given

   where e.EMP_NO = COALESCE(?, e.emp_no )

and that there is an index on EMP_NO, why doesn't the optimiser default
to the index. After all, it is logically more likely that a value will
be passed in the where condition, rather than a NULL.


To evaluate COALESCE, e.emp_no must be known. How it can be known before 
we start reading the table (via index scan)? Chicken and egg problem.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] 6 Month ago, SQL:2016 was released , a more detailed write-up

[Dmitry Yemanov]

15.06.2017 20:28, marius adrian popa wrote:


6 Month ago, SQL:2016 was released,  a more detailed write-up

http://modern-sql.com/blog/2017-06/whats-new-in-sql-2016


Glad to know that DECFLOAT is now standard ;-)


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Effect of DST on current_timestamp values

[Dmitry Yemanov]

13.06.2017 11:12, Leyne, Sean wrote:

A post to the Firebird support list pointed out that current_timestamp 
values do not correctly reflect the effect of DST time changes while the 
server is running.
In order for current_timestamp to reflect the correct local time values, 
the server needs to be restarted.
Though it never occurred to me to check time condition, I would have to 
say that the current logic seems wrong.  Since current_timestamp is 
supposed to return the local time, all values should be valid in that 
context, and it should not require a server restart in order for the 
values to be correct.
I was going to log a case about this, but thought that I should post 
here first.


Old known issue. The C runtime library caches the current time zone at 
application startup, so any background changes at the system level 
remain invisible to applications.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5567) Direct system table modifications are not completely prohibited

[Dmitry Yemanov (JIRA)]

Direct system table modifications are not completely prohibited
---

 Key: CORE-5567
 URL: http://tracker.firebirdsql.org/browse/CORE-5567
 Project: Firebird Core
  Issue Type: Bug
  Components: Engine
Affects Versions: 3.0.2, 3.0.1, 3.0.0, 4.0 Initial
Reporter: Dmitry Yemanov


SQL> create database '/work/data/systab.fdb';
SQL> create domain my_type numeric(18, 2);
SQL> commit;

SQL> show domain MY_TYPE;
MY_TYPE NUMERIC(18, 2) Nullable

SQL> update rdb$fields set rdb$field_scale = -3 where rdb$field_name = 
'MY_TYPE';
Statement failed, SQLSTATE = 42000
UPDATE operation is not allowed for system table RDB$FIELDS
-- THIS IS EXPECTED
SQL> rollback;

SQL> show domain MY_TYPE;
MY_TYPE NUMERIC(18, 2) Nullable

SQL> set term ^;
SQL> execute block as
CON> begin
CON> update rdb$fields set rdb$field_scale = -3 where rdb$field_name = 
'MY_TYPE';
CON> end^
Statement failed, SQLSTATE = 42000
UPDATE operation is not allowed for system table RDB$FIELDS
-At block line: 3, col: 1
-- ALSO EXPECTED
SQL> set term ;^
SQL> rollback;

SQL> show domain MY_TYPE;
MY_TYPE NUMERIC(18, 2) Nullable

SQL> set term ^;
SQL> execute block as
CON> declare procedure hack as
CON> begin
CON> update rdb$fields set rdb$field_scale = -3 where rdb$field_name = 
'MY_TYPE';
CON> end
CON> begin
CON> execute procedure hack;
CON> end^
SQL> set term ;^
SQL> commit;

SQL> show domain MY_TYPE;
MY_TYPE NUMERIC(18, 3) Nullable

-- WTF???


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5566) Server crashes while restoring backup if the shadow file already exists

[Dmitry Yemanov (JIRA)]

Server crashes while restoring backup if the shadow file already exists
---

 Key: CORE-5566
 URL: http://tracker.firebirdsql.org/browse/CORE-5566
 Project: Firebird Core
  Issue Type: Bug
  Components: Engine
Reporter: Dmitry Yemanov


gbak -c -rep C:\TEMP\TEST.FBK C:\TEMP\TEST.FDB

Database is configured to have a shadow at C:\TEMP\CITYCARD.SHD. If there's no 
such a file, restore succeeds. If the file exists, expected error is reported:

gbak: ERROR:I/O error during "CreateFile (create)" operation for file 
"C:\TEMP\CITYCARD.SHD"
gbak: ERROR:Error while trying to create file
gbak: ERROR:Файл существует.
gbak:Exiting before completion due to errors
 
but then the engine crashes with the following stack:

  msvcr100d.dll!memcmp()  Line 127 Asm
  engine12.dll!Firebird::MetaName::compare(const Firebird::MetaName & m={...})  
Line 90 + 0x38 bytes C++
  engine12.dll!Firebird::MetaName::operator==(const Firebird::MetaName & 
m={...})  Line 96 + 0x32 bytes C++
  engine12.dll!PAR_find_proc_field(const Jrd::jrd_prc * 
procedure=0x033f2260, const Firebird::MetaName & name={...})  Line 771 
+ 0x16 bytes C++
  engine12.dll!Jrd::FieldNode::parse(Jrd::thread_db * tdbb=0x0012a4f0, 
Firebird::MemoryPool & pool={...}, Jrd::CompilerScratch * 
csb=0x0341ad50, const unsigned char blrOp='')  Line 4855 + 0x15 bytes 
C++
  engine12.dll!PAR_parse_node(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::CompilerScratch * csb=0x0341ad50)  Line 1611 + 0x4b bytes C++
  engine12.dll!PAR_parse_value(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::CompilerScratch * csb=0x0341ad50)  Line 1544 + 0xf bytes C++
  engine12.dll!parseMap(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::CompilerScratch * csb=0x0341ad50, unsigned long stream=0x0001) 
 Line 3326 + 0x15 bytes C++
  engine12.dll!Jrd::UnionSourceNode::parse(Jrd::thread_db * 
tdbb=0x0012a4f0, Jrd::CompilerScratch * csb=0x0341ad50, const 
short blrOp=0x004c)  Line 1641 + 0x1a bytes C++
  engine12.dll!PAR_parseRecordSource(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::CompilerScratch * csb=0x0341ad50)  Line 1281 + 0x15 bytes C++
  engine12.dll!PAR_rse(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::CompilerScratch * csb=0x0341ad50, short rse_op=0x0043)  Line 1306 
+ 0x15 bytes C++
  engine12.dll!PAR_rse(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::CompilerScratch * csb=0x0341ad50)  Line 1426 + 0x15 bytes C++
  engine12.dll!PAR_parse_node(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::CompilerScratch * csb=0x0341ad50)  Line 1587 + 0x15 bytes C++
  engine12.dll!PAR_blr(Jrd::thread_db * tdbb=0x0012a4f0, Jrd::jrd_rel * 
relation=0x0341a750, const unsigned char * blr=0x00129298, 
unsigned long blr_length=0x01d8, Jrd::CompilerScratch * 
view_csb=0x, Jrd::CompilerScratch * * 
csb_ptr=0x001295f8, Jrd::JrdStatement * * 
statementPtr=0x, const bool trigger=false, unsigned short 
flags=0x)  Line 235 + 0x1a bytes C++
  engine12.dll!MET_parse_blob(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::jrd_rel * relation=0x0341a750, Jrd::bid * 
blob_id=0x00129898, Jrd::CompilerScratch * * 
csb_ptr=0x001295f8, Jrd::JrdStatement * * 
statementPtr=0x, const bool trigger=false, bool 
validationExpr=false)  Line 7750 + 0x58 bytes C++
  engine12.dll!MET_get_dependencies(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::jrd_rel * relation=0x0341a750, const unsigned char * 
blob=0x, const unsigned long blob_length=0x, 
Jrd::CompilerScratch * view_csb=0x, Jrd::bid * 
blob_id=0x00129898, Jrd::JrdStatement * * 
statementPtr=0x, Jrd::CompilerScratch * * 
csb_ptr=0x00129a18, const Firebird::MetaName & object_name={...}, int 
type=0x0001, unsigned short flags=0x, Jrd::jrd_tra * 
transaction=0x0308c710, const Firebird::MetaName & 
domain_validation={...})  Line 5826 + 0x8d bytes C++
  engine12.dll!MET_scan_relation(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::jrd_rel * relation=0x0341a750)  Line 8796 + 0x92 bytes C++
  engine12.dll!scan_relation(Jrd::thread_db * tdbb=0x0012a4f0, short 
phase=0x0003, Jrd::DeferredWork * work=0x03414b98, Jrd::jrd_tra * 
__formal=0x0308c710)  Line 10983 C++
  engine12.dll!DFW_perform_work(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::jrd_tra * transaction=0x0308c710)  Line 4746 + 0x28 bytes C++
  engine12.dll!TRA_commit(Jrd::thread_db * tdbb=0x0012a4f0, 
Jrd::jrd_tra * transaction=0x0308c710, const bool retaining_flag=false) 
 Line 380 C++
  engine12.dll!commit(Jrd::thread_db * tdbb=0x0012a4f0, Jrd::jrd_tra * 
transaction=0x0

Re: [Firebird-devel] Windows snapshots are outdated

[Dmitry Yemanov]

06.06.2017 09:32, Gabor Boros wrote:


The latest 3.0/4.0 kits have 2017-05-24 date.


VM has some troubles, we're working on that.


Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] SUPERSERVER_V2

[Dmitry Yemanov]

25.05.2017 18:53, Adriano dos Santos Fernandes wrote:


What's SUPERSERVER_V2 in the code?


Old attempt by Borland to implement some features that can be utilized 
by a properly threaded SuperServer.



Will it be used some day?
Will it be removed some day?


We preserve it as a reference. Some work by Vlad is partially based on 
that code.



Dmitry

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Under what condition can RDB$RELATION_TYPE be NULL?

[Dmitry Yemanov]

16.05.2017 22:27, Mark Rotteveel wrote:
>
> But it does hurt if you have a condition: "where rdb$relation_type = 0"
> (or in this case: "where rdb$relation_type in (0, 3)"; shouldn't the
> restore fix this up and make NULL explicit 0?

Historically, many system fields inherit this behaviour and should be 
handled using coalesce(field, 0), so this case is not unique. That said, 
we can surely fix it up during restore. Please create a ticket.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Under what condition can RDB$RELATION_TYPE be NULL?

[Dmitry Yemanov]

17.05.2017 01:02, Leyne, Sean wrote:
>
> gbak is not a special process, it is restricted the same as user connections, 
> so with v3+ it would not be able to execute any DML operations on system 
> tables.

gbak is a special process and it does work with system tables directly.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Under what condition can RDB$RELATION_TYPE be NULL?

[Dmitry Yemanov]

16.05.2017 22:14, Mark Rotteveel wrote:

> A bug was just reported for Jaybird 3
> (http://tracker.firebirdsql.org/browse/JDBC-494); I made changes to use
> RDB$RELATION_TYPE to discriminate between the different relation types,
> but apparently it can be null under some conditions.
>
> What are those conditions, or would this be a bug in Firebird if it is
> null?

Perhaps a restore from some ancient version could leave this field being 
NULL. The engine treats NULL as 0 (aka PERSISTENT), so it shouldn't hurt.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Start transaction from base transaction

[Dmitry Yemanov]

05.05.2017 20:01, Vlad Khorsun wrote:
>
> %typesnap_shot
> snap_shot
>   : SNAPSHOT
>   | SNAPSHOT TABLE
>   | SNAPSHOT TABLE STABILITY
> + | SNAPSHOT SHARING FROM 

SNAPSHOT BASED ON 
?


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Changing IRoutineMetadata in Plugin::makeProcedure

[Dmitry Yemanov]

18.04.2017 17:40, Jiří Činčura wrote:
>
> Also going from SQL_TEXT to SQL_VARYING should be possible, right?

Generally it is, unless SQL_TEXT is of length 32766 or 32767.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] MON$Attachment fields vs. RDB$Get_Context("SYSTEM", ....)

[Dmitry Yemanov]

04.04.2017 16:30, Leyne, Sean wrote:
>
> BTW, why would they have names which are different from the names already 
> established in the MON$ table?
>
>>> - MON$REMOTE_PID
>>> - MON$REMOTE_PROCESS

Because the naming mismatch exists since the very beginning 
(CLIENT_ADDRESS / MON$REMOTE_ADDRESS) and new values follow this 
"tradition".


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] MON$Attachment fields vs. RDB$Get_Context("SYSTEM", ....)

[Dmitry Yemanov]

31.03.2017 01:11, Leyne, Sean wrote:
>
> One of my developers was looking for a simple way to determine what
> Process was the current DB operation related to.
>
> I knew that MON$Attachments has a MON$Remote_Process value, so I
> expected the same details would be available thru
> RDB$Get_Context("SYSTEM",) call.  But I was wrong.

RDB$GET_CONTEXT('SYSTEM', 'CLIENT_PID')
RDB$GET_CONTEXT('SYSTEM', 'CLIENT_PROCESS')

> There are several MON$Attachments columns which are not available, thru
> the "SYSTEM" namespace:
> - MON$SERVER_PID
> - MON$STATE
> - MON$REMOTE_PID
> - MON$CHARACTER_SET_ID
> - MON$TIMESTAMP
> - MON$GARBAGE_COLLECTION
> - MON$REMOTE_PROCESS
> - MON$STAT_ID
>
> Is there a reason why the values could not be added to "SYSTEM" namespace?

We add them on demand, based on user requests. I don't think all of them 
are really needed.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] [FB-Tracker] Created: (CORE-5507) Wrong value of the new field at the old records, created before that new field was added.

[Dmitry Yemanov]

24.03.2017 09:33, Vlad Khorsun wrote:
>
>> Firebird is known to upgrade the record format while reading. "Upgrade"
>> here means using the latest (aka current) format. The current format is
>> the one that can be seen in RDB$RELATION_FIELDS. So one may expect that
>> the default value to be used is also the latest one, that's stored in
>> RDB$ tables. IIRC, this is how FB 2.5 works.
>
> Exactly. When i read the table and know that some field should have default
> value, i expect to see this value at system catalog.

It depends on the definition. In fact, adding a NOT NULL DEFAULT X 
column means two things: (1) DEFAULT X will be used implicitly for 
inserts or explicitly for the DEFAULT keyword and (2) X will be used to 
replace the missing values. For (1), everything works as expected. For 
(2), your expectation relies on the fact that X is substituted while 
reading. But this is just an implementation detail. The contract is 
"replacing missing values", whatever it could mean. Someone else does 
not know such internals and expects this "replacement" happening by some 
voodoo magic during ALTER ;-)

Also, lets consider this:

SQL> create table t (col1 int);
SQL> insert into t values (1);
SQL> commit;
SQL> alter table t add col2 int default 123 not null;
SQL> select * from t;

 COL1 COL2
 
1  123

SQL> alter table t alter col2 drop default;
SQL> select * from t;

 COL1 COL2
 
1  123

There's no default at the end, but COL2 is still returning 123. This 
matches the "update" expectation, not the "convert missing to the 
current default" one.

Yesterday I considered this issue being a bug, but now I'm not that 
sure. It may look obvious for some users but counter-intuitive for the 
others.

Anyone outside the development team cares to comment? :-)


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] [FB-Tracker] Created: (CORE-5507) Wrong value of the new field at the old records, created before that new field was added.

[Dmitry Yemanov]

24.03.2017 02:29, Mark Rotteveel wrote:

> To me the behavior described under "actual" intuitively sounds like the
> correct behavior. Why do you expect that the column value would change
> to 'ABC'?

This is really a tricky case. The "replace non-existing value with the 
default one" hack is a native Firebird feature that's not covered by the 
standard, it allows adding fields without updating the whole table. The 
question is what default value must be used, if there are/were many.

Firebird is known to upgrade the record format while reading. "Upgrade" 
here means using the latest (aka current) format. The current format is 
the one that can be seen in RDB$RELATION_FIELDS. So one may expect that 
the default value to be used is also the latest one, that's stored in 
RDB$ tables. IIRC, this is how FB 2.5 works.

 From another side, storing the default value inside the format is a 
smart hack that allows to avoid updating the table. It was designed to 
act like an implicit update but without the overhead. And considering 
the update semantics, the new behaviour may look correct.

The big question is what expectations are better suitable to our users.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Where is Android port?

[Dmitry Yemanov]

23.03.2017 05:19, Carlos H. Cantu wrote:

> I see two mentions in the 3.02 ReleaseNotes about Android port
> [(CORE-3885) and (CORE-5332)], but I can't find any Android link in
> the download page.
>
> This needs to be fixed.

To be published, I'd want it to be rebuilt from the 3.0.2 codebase. The 
latest Android binaries are v3.0.0, not something to be put on the 
v3.0.2 download page.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] NBAK simplification RFC

[Dmitry Yemanov]

20.03.2017 20:06, Dmitry Yemanov wrote:
>
> Agreed. But I'm really worried about taking locks in ASTs, this sounds
> as a dangerous practice to me. I smell new deadlocks.

Just to clarify: I meant taking LM locks here. It could be safer for 
low-level locks.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] NBAK simplification RFC

[Dmitry Yemanov]

20.03.2017 19:43, Vlad Khorsun wrote:
>
>> Firebird doesn't use signals for AST delivery for a long time already
>> and it might be possible to issue new locks from inside AST handler.
>>
>> Therefore it might make sense to go back to this logic. This would make
>> nbackup state locks very transient and do not require flushing page
>> cache during transitions.
>
>It would be very good to avoid flushing.

Agreed. But I'm really worried about taking locks in ASTs, this sounds 
as a dangerous practice to me. I smell new deadlocks.

>> It also might be possible to use shared memory for synchronization
>> instead of lock manager calls to speed things up.
>
>Could you explain more ?

I suppose Nickolay means replacing GlobalRWLock with native OS-level RW 
primitives used similar to shared memory mutexes. For Windows, they're 
just shared as named objects, for POSIX they're stored in shmem.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] windows test build - least painful way?

[Dmitry Yemanov]

28.02.2017 17:18, Michal Kubecek wrote:
>
> The problem is the series also touches windows specific code (including
> WNET and XNET)

BTW, you may wipe out WNET at all. We seemed to have an agreement on that.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] [FB-Tracker] Created: (CORE-5493) Add context variable about transaction start timestamp

[Dmitry Yemanov]

01.03.2017 19:50, Vlad Khorsun wrote:
>
>>> If this variable is added as a context variable into "SYSTEM" namespace, 
>>> simple
>>> "START_TIMESTAMP" should be enough
>>
>> It could be confused with attachment-start timestamp.
>
> I thought it is clear that ticket is about new system contex variable (like
> CURRENT_TRANSACTION), not about new variable for the RDB$GET_CONTEX().

Well, I missed that, sorry. START_TIMESTAMP is a bad name for both 
implementations though.

> Also, I prefer to avoid to return as a string something that is not a string.

Agreed here. From another side, I don't like polluting the global 
namespace with non-standard but reserved keywords.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] [FB-Tracker] Created: (CORE-5493) Add context variable about transaction start timestamp

[Dmitry Yemanov]

01.03.2017 19:33, Dimitry Sibiryakov wrote:

> If this variable is added as a context variable into "SYSTEM" namespace, 
> simple
> "START_TIMESTAMP" should be enough

It could be confused with attachment-start timestamp.


Dmitry



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] windows test build - least painful way?

[Dmitry Yemanov]

28.02.2017 17:18, Michal Kubecek wrote:
>
> as my Hackweek project, I played with struct rem_port and tried to turn
> it into class hierarchy with virtual functions. The plan is to use this
> to implement listening on multiple sockets (CORE-5219) and possibly also
> AF_UNIX socket support but I didn't get that far yet. Current state of
> the cleanup is at
>
>   https://github.com/mkubecek/firebird/commits/mk/4.0/remote-cleanup
>
> The problem is the series also touches windows specific code (including
> WNET and XNET) and I'm pretty sure there are some typos and omissions
> which I would like to fix before submitting a pull request.

When you're ready to PR, just ask here to build from your fork's branch. 
We'll report if any issues are present.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird friendly error messages

[Dmitry Yemanov]

01.03.2017 00:18, Dmitry Kuzmenko wrote:
>
> As to BLR, I don't think that eliminating BLR is a good idea. You need
> to have same or better speed of interpreting SQL

Execution time does not depend on whether it's BLR or SQL. Parsing time 
(read: prepare time) depends.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] RFC: Timeouts

[Dmitry Yemanov]

25.02.2017 17:21, Adriano dos Santos Fernandes wrote:
>
> But, client side already can set it own timer and cancel the statement.

It was considered. However, it would mean that our implementation is 
completely useless for Java and .NET clients, they would have to 
implement timeouts from scratch.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] RFC: Timeouts

[Dmitry Yemanov]

25.02.2017 12:37, Mark Rotteveel wrote:

>> Do you/anyone know if these engines return full results sets or follow the 
>> "page set" approach?
>
> As far as I know Oracle[1], PostgreSQL[2], SQL Server[3] support it. I
> believe MySQL does as well. Don't know about other database systems, but
> I assume most of them will support this (the SQL CLI standard also
> defines it).

AFAIK, in these databases page sets are primarily a transport/API 
feature. The important question, however, is whether the engine does 
some work during fetch() besides copying the row(s) for the next batch. 
I suspect InterBase/Firebird is the only engine that may process 
execute() as a no-op and perform all the data retrieval / joining / etc 
during fetching.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] RFC: Timeouts

[Dmitry Yemanov]

All,

Let me jump into discussion and share my own concerns.

Depending on the plan, statement may take 99% of its "working" time 
inside execute() or inside fetch() or that time could be distributed 
among the API calls. Neither client nor DBA has any control on that. So 
I consider seriously wrong removing fetching time from the accounting.

 From the client side, a timeout can be seen from two different angles. 
It could be either statement execution time (including fetches, see 
above) - this is what we have implemented now. Or it could be the API 
call timeout, to avoid "blocking" for more than the specified time. This 
means that any higher-level accounting should be done by the application 
or connectivity library. I guess the latter approach gonna cost much 
more due to often timer resets. It also adds more work to the 
client-side developers.

Now about what is "working time". If the client specifies 10 seconds, I 
really doubt it expects to see the statement timing out after 30 
seconds. From the client POV, "working time" includes all the waits, 
round-trips and so on. It's not about server resources, they are outside 
client's business. It's rather about an application reaction time. So I 
consider Vlad's position perfectly valid. And I'm surprised that Mark 
and Jiri disagree.

For server-side statement timeouts, situation is a bit different. It's 
really about long-running queries and server resources. DBA can hardly 
guess about application logic (time between fetches) or about network 
latencies. So if we speak only about engine doing some work, performing 
disk I/O or waiting for something, then Sean's point has some value. Of 
course, it has nothing to do with CPU time spent, it's more about time 
spent inside the engine for whatever reason. But it surely defines 
"long-running" statements from one side, as something being served by 
the engine.

 From another side, we have long-running "sleeping" (even if 
occasionally) statements that block metadata objects, occupy memory and 
freeze transaction counters. These are different resources but it does 
not mean they should be ignored. Vlad's solution accounts that, Sean's 
does not.

So we have different requirements for the same feature and they conflict 
with each other. The positive side of the implemented solution is that 
it suits both client-side and server-side usage. Sean's suggestion does 
not fit client-side usage, IMO. Also, if the timer is suspended/resumed 
too often (per every fetch call), it gonna be costly.

That said, I'd vote against reworking the current design. Perhaps, we 
could additionally implement what Sean suggests, but *only* at the 
server side. But I have no idea how to mix them nicely. Having two 
independent timeouts looks ugly, complicates the engine code and is 
likely to confuse users.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] RFC: Timeouts

[Dmitry Yemanov]

25.02.2017 03:55, Leyne, Sean wrote:
>
> it is the value that represent a direct CPU cost of a SQL statement.

You actually seem wanting CPU quotas. But they're not timeouts. A 
long-running statement may produce almost zero CPU load.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5481) Available indices are not used in some cases if ORDER BY expression is a filtered one

[Dmitry Yemanov (JIRA)]

Available indices are not used in some cases if ORDER BY expression is a 
filtered one
-

 Key: CORE-5481
 URL: http://tracker.firebirdsql.org/browse/CORE-5481
 Project: Firebird Core
  Issue Type: Bug
  Components: Engine
Affects Versions: 3.0.1, 3.0.0, 4.0 Initial
Reporter: Dmitry Yemanov


CREATE TABLE ORG_ACCOUNTS
(
ORGACCOUNTID BIGINT NOT NULL PRIMARY KEY
);

CREATE TABLE BALANCES
(
BALANCEID BIGINT NOT NULL,
ORGACCOUNTID BIGINT NOT NULL,
BALANCEDATE DATE  NOT NULL
);

ALTER TABLE BALANCES ADD CONSTRAINT PK_BALANCES PRIMARY KEY (BALANCEID);
ALTER TABLE BALANCES ADD CONSTRAINT FK_BALANCES_ORGACCOUNTS FOREIGN KEY 
(ORGACCOUNTID) REFERENCES ORG_ACCOUNTS (ORGACCOUNTID);
ALTER TABLE BALANCES ADD CONSTRAINT BALANCES_BALANCEDATE_ORGACCOUNT UNIQUE 
(ORGACCOUNTID, BALANCEDATE);

CREATE DESCENDING INDEX BALANCES_BALANCEDATE_DESC ON BALANCES (BALANCEDATE);

select first 1 *
from Balances B
where OrgAccountID=18 and
   BalanceDate<='01.01.2017'
order by BalanceDate desc;
 
v2.5:
PLAN (B ORDER BALANCES_BALANCEDATE_DESC INDEX (BALANCES_BALANCEDATE_ORGACCOUNT))

v3.0:
PLAN (B ORDER BALANCES_BALANCEDATE_DESC)

Correct (best) plan should be:
PLAN (B ORDER BALANCES_BALANCEDATE_DESC INDEX (FK_BALANCES_ORGACCOUNTS))

Now the funny thing:
ALTER TABLE BALANCES DROP CONSTRAINT BALANCES_BALANCEDATE_ORGACCOUNT;

v3.0:
PLAN (B ORDER BALANCES_BALANCEDATE_DESC INDEX (FK_BALANCES_ORGACCOUNTS))

I.e. existing compound index BALANCES_BALANCEDATE_ORGACCOUNT is not only 
ignored itself (see also CORE-5070), but it also hides possibilities to use 
another index FK_BALANCES_ORGACCOUNTS.


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Deadlock parsing BLR

[Dmitry Yemanov]

24.01.2017 12:22, Jiří Činčura wrote:
> Anyone?

It's hard to say anything without a test case.

Dmitry



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] [FB-Tracker] Created: (CORE-5460) Insert NULL into identity column with auth generated value

[Dmitry Yemanov]

19.01.2017 00:51, Ann Harrison wrote:
>
> In what universe does that make sense?  The field is NOT NULL.  You're 
> storing NULL in it.  That's an error.

I'd say it depends. What about a BEFORE trigger converting input NULL to 
something valid before storing?


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

18.01.2017 12:38, Alex Peshkoff wrote:
>
> Currently with dfw we do have a lot of DDL errors raised at commit time
> i.e. it's not a regression.

True, but only because the actual work is performed during commit. If we 
claim that DDL changes are applied immediately, but error is thrown at 
commit, this looks weird. Especially if we find a way to allow mixed DDL 
and DML - imagine ALTER TABLE and subsequent UPDATE both executing OK 
but failing at commit because of the metadata conflict.

 > But don't forget that under
 > normal circumstances such conflicts will be very rare.

I would seriously question the need to allow concurrent DDL against the 
same objects. This is simply not the way people work with the relational 
databases. I'd rather lock the metadata being changed at the DDL time 
and until commit.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

17.01.2017 14:28, Alex Peshkoff wrote:
>
>>> Returning to 'allow concurrent transactions to change the same objects'.
>>> What if both transactions create same objects or any other phase 1
>>> conflict?
>>> First committed wins?
>> Yes.
>
> That's OK for me.

I don't think I like it. With DML, conflicts with active record versions 
are detected immediately and you either get an error or allowed to wait. 
Using a completely different approach for DDL looks counter-intuitive.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] linux build failure - security.sql (bisected)

[Dmitry Yemanov]

17.01.2017 16:13, Adriano dos Santos Fernandes wrote:

>> It should be possible to add one or more alternative rules in the
>> grammar where character set names are used in a way that preserves the
>> backwards compatibility.
>
> I believe this is the thing we need to do.
>
> Just something as "identifier | BINARY" rule.

If BINARY is an existing user-defined domain, how should "DECLARE VAR 
BINARY" be treated in this case? As domain or as built-in data type?


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

17.01.2017 13:32, Dimitry Sibiryakov wrote:
>
> I always feel the same when I see special treating of system tables and system
> transaction in VIO routines.

Perhaps it's worth starting from deprecating DFW and only then come back 
to the metadata versioning?


Dmitry



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

17.01.2017 13:18, Dimitry Sibiryakov wrote:
>
>> How does your idea address this? Between CREATE TABLE and INSERT page
>> space (PP, IRP) must be allocated and initialized. And this cannot be
>> reverted via the undo log in the case of rollback.
>
> It can. PP is bound to RDB$RELATIONS and IRP to RDB$INDICES, right? Garbage 
> collector
> (VIO_purge()) can recognize these tables as it is done now in VIO_erase() and 
> delete pages
> before the record.

It's more about RDB$PAGES rather than about RDB$RELATIONS/RDB$INDICES. 
Sounds too hackish to my taste, but probably doable.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

Yet another case:

Transaction creates a new format, stores some records using this new 
format, then the server dies. After restart, changes in 
RDB$FORMATS/RDB$RELATIONS are occasionally backed out. Then it could be 
impossible to garbage collect the new record versions due to their 
format being unknown.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

16.01.2017 22:48, Dimitry Sibiryakov wrote:
>
>> But what about low-level logic like
>> VIO_get_current() which must see uncommitted changes?
>
> If format is created at the end of DDL execution (instead of DFW), it 
> can/must be
> written into database before any DML can write any record with new format.

You miss my point. It's written into RDB$FORMATS/RDB$RELATIONS but not 
committed because transaction is still active. So concurrent 
transactions cannot see these changes, accordingly to your versioning 
approach. But VIO_get_current() must see them. Do you suggest to use 
"dirty read" in such cases?


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

16.01.2017 23:08, Adriano dos Santos Fernandes wrote:
>
> DML changes in the same transaction uses real metadata, i.e., DDL being
> changed does not affect DML or metadata cache.

I.e. one cannot add a field and populate it with data within a single 
transaction. This sounds too much restrictive.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

16.01.2017 23:39, Dimitry Sibiryakov wrote:

> Then your schema won't solve problem with mixing DDL and DML. Users still 
> won't be able
> to create table and fill it with data in one transaction.

How does your idea address this? Between CREATE TABLE and INSERT page 
space (PP, IRP) must be allocated and initialized. And this cannot be 
reverted via the undo log in the case of rollback.


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Virtual metadata

[Dmitry Yemanov]

16.01.2017 22:04, Dimitry Sibiryakov wrote:
>
> Why "without gaps"? In my vision the first step is to modify garbage 
> collector/sweeper
> to clean unused formats out.

OK, it wasn't obvious from your message.

>> How concurrent readers should access/skip physically stored records if
>> their format is unknown/invisible for them?
>
> If they can read records, they can read formats.

This is the easy case, no problems. But what if they cannot? I don't 
mean regular readers -- they see active txn id and shouldn't even try to 
decode these versions. But what about low-level logic like 
VIO_get_current() which must see uncommitted changes?


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Code style - right margin

[Dmitry Yemanov]

16.01.2017 19:39, Alex Peshkoff wrote:
>
> With today 22-24" 16:9 screens

20" 4:3 is still enough for me ;-)


Dmitry


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Code style - right margin

[Dmitry Yemanov]

15.01.2017 17:50, Adriano dos Santos Fernandes wrote:

> What is our maximum code right margin?
>
> AFAIK it used to be 100 and I remember some discussion to increase it.
>
> Some people is thinking it's infinite.
>
> I think currently 100 is too low and 120 would be better.

It used to be 80 and Claudio had suggested to raise it up to 100 (since 
FB3, IIRC).

Perhaps I could live with 120, but it's pretty close to the limit of my 
personal comfort. Let's see what others think.


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird 4 identifier limits

[Dmitry Yemanov]

12.01.2017 18:00, Mark Rotteveel wrote:
>
> As far as I understand it is specifically for backwards compatibility
> (eg tools that expect/depend on the max 31 characters **and** max 31
> bytes) limit

Then IMO it should be a boolean, not a numeric limit.


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird 4 identifier limits

[Dmitry Yemanov]

12.01.2017 17:33, Leyne, Sean wrote:
>
> Why is a configuration setting for this required?
>
> This seems like a fix/complication for a problem that doesn't really exist.

That was my opinion too.


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird 4 identifier limits

[Dmitry Yemanov]

06.01.2017 17:53, Mark Rotteveel wrote:

> I just noticed that in Firebird 4 the identifier length limits are
> configurable.
>
> Is there a maximum length, or can I theoretically use a length of say
> 8191 characters for an identifier?

Maximum is 63 characters. IIRC, configuration allows to lower this limit.

 > Is there a quick and easy way to determine the maximum identifier
 > length of a database (eg a service call)?

No.


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Database initially shutdown after restore in FB4

[Dmitry Yemanov]

06.01.2017 16:33, Mark Rotteveel wrote:

> When restoring a database through the service API in FB4 (Windows 10 64
> bit, Firebird-4.0.0.487-0_x64), it looks like the database is initially
> shutdown (or something else goes wrong during the attach).
>
> Is this intentional, or should I report a bug?

Already known, this is a recent regression.


Dmitry



--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5435) Badly selective index could be used for extra filtering even if selective index is used for sorting

[Dmitry Yemanov (JIRA)]

Badly selective index could be used for extra filtering even if selective index 
is used for sorting
---

 Key: CORE-5435
 URL: http://tracker.firebirdsql.org/browse/CORE-5435
 Project: Firebird Core
  Issue Type: Bug
  Components: Engine
Affects Versions: 3.0.1, 3.0.0, 4.0 Initial
Reporter: Dmitry Yemanov


It seems that Firebird 3 is sometimes choosing the index with less selectivity, 
which can have a serious 
effect on performance. We have seen this behaviour in several queries. 

In this example (we have attached in databases.zip the databases for Firebird 
2.5 and Firebird 3): 
  -Table1 has around 30 records. 
  -Field1 has 15000 different values with 20 records each one. 
  -Field2 has 2 different values (0 with 20 records, and 1 with 10 
records). 
  -Both databases were tested immediately after a backup/restore cycle and in 
the same computer. 
  -The times are measured in the second execution of each query (though the 
first execution follow the same pattern). 

Query: 
  select * 
  from TABLE1 where FIELD1 is null and FIELD2=0 
  order by FIELD1, ID 

Firebird 2.1/2.5: 
  PLAN (TABLE1 ORDER TABLE1_F1_ID INDEX (TABLE1_F1_ID)) 
  0.002 seconds 

Firebird3: 
  PLAN (TABLE1 ORDER TABLE1_F1_ID INDEX (TABLE1_F2)) 
  Select Expression 
  -> Filter 
  -> Table "TABLE1" Access By ID 
  -> Index "TABLE1_F1_ID" Range Scan (partial match: 1/2) 
  -> Bitmap 
  -> Index "TABLE1_F2" Range Scan (full match) 
  0.240 seconds 

- 

CREATE TABLE "TABLE1" 
( 
  "ID"  INTEGER NOT NULL, 
  "FIELD1"  INTEGER, 
  "FIELD2"  INTEGER, 
CONSTRAINT "TABLE1_PK" PRIMARY KEY ("ID") 
); 

CREATE INDEX "TABLE1_F1_ID" ON "TABLE1"("FIELD1", "ID"); 
CREATE INDEX "TABLE1_F2" ON "TABLE1"("FIELD2"); 


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird 4 features

[Dmitry Yemanov]

04.01.2017 13:13, Dimitry Sibiryakov wrote:
>
> "This has the added benefit that tools can use this information to determine 
> the data
> type consistently based on type and subtype".

AFAIU, ticket suggests to match API's subtype also to field's subtype, 
in addition to matching it to the field's charset/collation. But IMO it 
means that we cannot have other subtypes for strings in the future. I.e. 
subtype 4 cannot be anything else than UTF8 and so on. This looks too 
much limiting.


Dmitry


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Firebird 4 features

[Dmitry Yemanov]

30.12.2016 18:05, Dimitry Sibiryakov wrote:

> Do anybody object if I try to dig a little into CORE-5064?

No, feel free.


Dmitry



--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Wire protocol changes in Firebird 4?

[Dmitry Yemanov]

30.12.2016 14:19, Dimitry Sibiryakov wrote:
>
> 1) Core team have no time for it.

True.

> 2) Core team have no time to review big patches from outsiders.

Bullshit.

> 3) Backward compatibility has to be kept.

True. What a surprise!


Dmitry



--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Deadlock parsing BLR

[Dmitry Yemanov]

22.12.2016 10:55, Jiří Činčura wrote:
>
> On 2.5.6 I've got error %subj%.

What is the exact error message?


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Windows snapshot builds broken?

[Dmitry Yemanov]

18.12.2016 14:28, Mark Rotteveel wrote:

> I just looked at the Windows snapshots builds, but they are a few days
> off against the Linux builds,
>
> FB 4: Win: 2016-12-16, Linux: 2016-12-18
> FB 3: Win: 2016-12-12, Linux: 2016-12-18
> FB25: Win: 2016-12-13, Linux: 2016-12-18
>
> Is there something wrong with the Windows build?

No, it's just a matter of how they're built. Linux builds are done daily 
unconditionally, Windows ones - only when something was changed in the code.


Dmitry


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5418) Inconsistent output when retrieving the server log via Services API

[Dmitry Yemanov (JIRA)]

Inconsistent output when retrieving the server log via Services API
---

 Key: CORE-5418
 URL: http://tracker.firebirdsql.org/browse/CORE-5418
 Project: Firebird Core
  Issue Type: Bug
  Components: Engine, SVCMGR
Affects Versions: 2.5.6, 2.5.5, 2.5.4, 2.5.3 Update 1, 2.1.7, 2.5.3, 2.5.2 
Update 1, 2.5.2, 2.5.1, 2.5.0
 Environment: Consistently reproduced on Windows only
Reporter: Dmitry Yemanov
Priority: Minor


When Services API is used to fetch the firebird.log contents 
(isc_svc_action_get_fb_log command), sometimes unexpected output may appear:

1) If firebird.log exists and it's empty, garbage may be returned
2) If firebird.log is not empty, its last line may be returned twice


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] FB2.5.6 - '' = ' ' is evaluated as true

[Dmitry Yemanov]

07.12.2016 16:24, Slavomir Skopalik wrote:
>
> is it correct that empty string '' in comparison with one space string '
> ' is evaluated as true?

Yes. Accordingly to the SQL standard, trailing spaces are ignored in 
[most] comparisons.


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] CORE-5099 build with gcc6 broken

[Dmitry Yemanov]

03.12.2016 18:04, marius adrian popa wrote:

> This can be closed
>
> CORE-5099 http://tracker.firebirdsql.org/browse/CORE-5099
>
> master defaults to -std=c++11
> https://github.com/FirebirdSQL/firebird/search?utf8=%E2%9C%93=-std%3Dc%2B%2B11
> and gcc6 would be forced to compile in c++11  mode not in c++14

What we have is rather a workaround, not a solution. I'd prefer to leave 
the ticket open, just rename it to "build with std=c++14 broken".


Dmitry



--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Support for schemas/namespaces

[Dmitry Yemanov]

13.11.2016 19:14, Fabiano Bonin wrote:
>
> Today I took a look at Firebird Planning Board, and didn't see a single
> mention for schemas/namespaces in the plan.

Planning board is about FB4 only. Schemas are likely to be implemented 
after that.


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] [FirebirdSQL/firebird] 5aede1: Decimal floating point numbers - first draft

[Dmitry Yemanov]

13.11.2016 19:18, Alex Peshkoff wrote:
>
> SQL> SELECT * FROM TESTDECFLOAT;
>
>  FEE_DECFLOAT   FEE_REAL   PERCENTAGE
>  == 
>  0.70 0.6999 0.05

I do see the difference with REAL but I'm asking for a difference with 
NUMERIC backed by int32/int64.

I don't mind DECFLOAT being a "better float" than FLOAT / DOUBLE 
PRECISION but so far I don't see how it's better than NUMERIC/DECIMAL 
(which also stores exactly 0.70 in dialect 3).

>> Moreover, what are we going to do when people ask as for precisions
>> beyond the 34 decimal digits? Introduce blr_dec256/blr_dec512/etc or
>> switch to blr_varydec backed by decNumber (and probably stored as packed
>> BCD)? Are there any reasons why the current implementation doesn't
>> follow this way other than hardware accelerated computations for 64/128
>> bits?
>
> Use of unlimited length fields is certainly great but I suppose we
> should switch to it in future versions (including unlimited length
> strings). Better precision calculations are needed right now, in v.4.

My question wasn't about v4 but rather about extensibility in general, 
how do you plan to support longer precisions? By increasing bits or by 
switching to some variable-length implementation? If the latter, why is 
dec64/dec128 better *now* than varydec? Just hardware backed 
computations or anything else?


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] [FirebirdSQL/firebird] 5aede1: Decimal floating point numbers - first draft

[Dmitry Yemanov]

11.11.2016 18:26, Dimitry Sibiryakov wrote:

>> - Added new datatypes: DECFLOAT(16) and DECFLOAT(34), using 64/128 bits
>> for numbers representation.
>
> What is the point of these new types? Cannot you just expand list of back-end 
> storage
> for standard DECIMAL?

This is my concern too. What is the user-visible difference between 
NUMERIC(15) based on blr_int64 and DECFLOAT(15) based on blr_dec64? Are 
both arithmetics compatible and if not then why? I bet we have a demand 
for longer standard NUMERICs rather than for non-standard DECFLOAT.

Moreover, what are we going to do when people ask as for precisions 
beyond the 34 decimal digits? Introduce blr_dec256/blr_dec512/etc or 
switch to blr_varydec backed by decNumber (and probably stored as packed 
BCD)? Are there any reasons why the current implementation doesn't 
follow this way other than hardware accelerated computations for 64/128 
bits?


Dmitry


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5393) Bad optimization of some operations with views containing subqueries

[Dmitry Yemanov (JIRA)]

Bad optimization of some operations with views containing subqueries


 Key: CORE-5393
 URL: http://tracker.firebirdsql.org/browse/CORE-5393
 Project: Firebird Core
  Issue Type: Bug
  Components: Engine
Affects Versions: 3.0.1, 3.0.0, 4.0 Initial
Reporter: Dmitry Yemanov


Bug is caused by internally created derived expressions being based on all view 
streams, including streams burried inside subqueries, etc. This is causing 
various optimization issues due to nested subqueries being non-computable at 
the moment.

Test case may look a bit complicated as it exploits the fact that simple field 
references become wrapped by derived expressions if located inside an explicit 
cursor. But I suspect this issue may be visible in other cases too.

create table test (id int primary key, col int);
commit;

insert into test (id, col) values (1, 1);
insert into test (id, col) values (2, 2);
commit;

-- view must contain a subquery
create or alter view v_test (id1, id2, col1, col2, dummy)
as
select t1.id, t2.id, t1.col, t2.col, (select 1 from rdb$database)
from test t1 join test t2 on t1.col = t2.id;

-- trigger makes the view updatable
create trigger t_v_test
for v_test before update
as
begin
end;

set plan;

set term ^;

execute block
as
begin
  for select id1 from v_test as cursor c do
  begin
update v_test set col1 = 1
where id1 = c.id1;
--where current of c;

update v_test set col1 = 1
where id1 = c.id1;
  end
end^

set term ;^

PLAN (V_TEST RDB$DATABASE NATURAL)
PLAN JOIN (V_TEST T1 INDEX (RDB$PRIMARY1), V_TEST T2 INDEX (RDB$PRIMARY1))
PLAN (V_TEST RDB$DATABASE NATURAL)
PLAN JOIN (V_TEST T1 INDEX (RDB$PRIMARY1), V_TEST T2 INDEX (RDB$PRIMARY1))
PLAN JOIN (C V_TEST T1 NATURAL, C V_TEST T2 INDEX (RDB$PRIMARY1))

The only naturally-read join is cursor C, this is expected. Both updates 
utilize the primary key index for table T1. So far so good.

set term ^;

execute block
as
begin
  for select id1 from v_test as cursor c do
  begin
update v_test set col1 = 1
--where id1 = c.id1;
where current of c;

update v_test set col1 = 1
where id1 = c.id1;
  end
end^

set term ;^

PLAN (C V_TEST RDB$DATABASE NATURAL)
PLAN (V_TEST RDB$DATABASE NATURAL)
PLAN JOIN (V_TEST T1 NATURAL, V_TEST T2 INDEX (RDB$PRIMARY1))
PLAN JOIN (C V_TEST T1 NATURAL, C V_TEST T2 INDEX (RDB$PRIMARY1))

The first update is not reported in the plan because it's based on the same 
cursor as the select itself. However, the second update is unable to utilize 
the primary key index for table T1 anymore.

In the production database, this issue is causing 100x degradation in execution 
time.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Reopened: (CORE-5382) Incorrect processing (truncation) of SQL statement with length 10MB+1

[Dmitry Yemanov (JIRA)]

 [ 
http://tracker.firebirdsql.org/browse/CORE-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Yemanov reopened CORE-5382:
--


Sorry, Pavel was too fast to resolve this ticket. The current behaviour is 
intended, but honestly, I don't remember why I decided to truncate instead of 
raising the error.

> Incorrect processing (truncation) of SQL statement with length 10MB+1
> -
>
> Key: CORE-5382
> URL: http://tracker.firebirdsql.org/browse/CORE-5382
> Project: Firebird Core
>  Issue Type: Bug
>  Components: Engine
>Affects Versions: 3.0.2
> Environment: Firebird 3.0.2.32619, IBProvider, direct INET-connection 
> to FB3 (without fbclient.dll).
>Reporter: Kovalenko Dmitry
>
> SQL Statement (total length = 10*1024*1024+1):
>  "select/*--  ---*/ID from DUAL "
> ---
> Prepare error:
> Dynamic SQL Error
> SQL error code = -204
> Table unknown
> DUA
> ---
> 1. I think, the error message must be a little another.
> 2. Special for this case, I created the new table "DUA" and server prepared 
> original query without any problem.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Allowed C++11 feature

[Dmitry Yemanov]

23.09.2016 02:27, Adriano dos Santos Fernandes wrote:

> I would like to add the following well-supported C++11 feature to our
> allowed list:
>
> Strongly-typed enum -
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2347.pdf
>
> Comments?

No objections.


Dmitry



--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Necessity of prepare in FB3

[Dmitry Yemanov]

22.09.2016 17:03, Dimitry Sibiryakov wrote:
>
> Why it can be ineffective? It is a client-side thing, no additional 
> round-trips is
> required.

Consider the embedded engine.


Dmitry



--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Necessity of prepare in FB3

[Dmitry Yemanov]

22.09.2016 17:09, Alex Peshkoff wrote:
>
> Just one note - let's finish with batch API interface first. Looks like
> required functionality may be present in that interfaces in much more
> logical way.

I'd say these features are orthogonal. I don't see why batch API calls 
should be used just to get a singleton result set as a cursor.

> Or we are going to add it to 3.0?

Why not? It does not change the API, just the underlying behaviour. 
Surely not for v3.0.1 but maybe for the next point release.


Dmitry



--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Necessity of prepare in FB3

[Dmitry Yemanov]

20.09.2016 17:02, Alex Peshkoff wrote:
>
> I just wantedto say that modifying behaviour of openCursor()
> and letting IResultSetfetch from non-cursor object is hardly
> correct solution.

While this approach is surely not elegant (and suboptimal from the 
performance POV), I do see it as being user-friendly. So if there's a 
demand for such usage, I think we could allow that.


Dmitry


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Necessity of prepare in FB3

[Dmitry Yemanov]

22.09.2016 16:48, Dimitry Sibiryakov wrote:
>
>> So if there's a
>> demand for such usage, I think we could allow that.
>
> It will make unnecessary the rest of API calls for statement execution.

If someone wants absolutely unified but ineffective API - maybe yes. 
Other API calls are for those who care about performance.


Dmitry


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] CNCT_user and CNCT_host

[Dmitry Yemanov]

16.09.2016 19:00, Jiří Činčura wrote:
>
> is the CNCT_user and CNCT_host visible for normal developer somewhere? I
> suppose CNCT_user is overriden by isc_dpb_user_name, right? Similar with
> CNCT_host and isc_dpb_host_name.

CNCT_host = isc_dpb_host_name = MON$REMOTE_HOST
CNCT_user = isc_dpb_os_user = MON$REMOTE_OS_USER


Dmitry


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] C++11 features

[Dmitry Yemanov]

01.09.2016 15:46, Adriano dos Santos Fernandes wrote:
>
> I see that before VC++ 2015 Update 3, it accepts everything supported
> without compiler options, right?

Looks so.

> So, I think the plan should be:
>
> - Update Linux prefix files to include -std=c++11
> - Agree on a set of allowed features
> - Write a readme specifying these features
>
> I can then submit it as PR with some usage of these features, converted
> from old code, so others can see it working in their environments.

Agreed.

> For the set of allowed features, I would want we to be dynamic and
> always evaluate then as necessary, but I would start with:
>
> - auto keyword
> - range-based for loop
> - lambdas

+ nullptr and static_assert, maybe also initializer lists and non-static 
member initializers.


Dmitry


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] C++11 features

[Dmitry Yemanov]

01.09.2016 14:04, Adriano dos Santos Fernandes wrote:
>
> MSVC10 supports nothing, and even MSVC12 is also a bit limited in regard
> to MSVC14 and recent g++ and clang++:
>
> https://msdn.microsoft.com/en-us/library/hh567368.aspx

We've agreed on MSVC13 for FBv4, so we cannot use features from later 
versions. As for gcc, are there officially supported Linux distros other 
than CentOS 6 that use gcc 4.7 or older? RHEL6 is EOL now, fortunately.


Dmitry


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] C++11 features

[Dmitry Yemanov]

31.08.2016 11:06, Thomas Steinmaurer wrote:
>
> Do we know that the new language features are stable enough?

So far we speak only about C++11 which is five years old already. And we 
won't know without trying anyway. I remember us discovering bugs in 
compilers during the C++ migration in early 2000s. It was somewhat 
painful but we succeeded and it was worth it.


Dmitry


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel