Re: [Firebird-devel] Raising the BLR level
11.04.2012 17:41, Adriano dos Santos Fernandes wrote: In the past, I thought about this problem, and a possibility I saw was to create a BLR prefix, saying the next BLR-rse verb would use two bytes for stream numbers instead of one. I do not know if this is a good solution, it's just a possibility. A similar solution could be to use the same prefix globally, so that it would cover the whole BLR stream, e.g.: blr_version5, blr_long_streams, ... but it also looks a bit hackery. As a interrelated question, we try to generate IDs instead of names for non-stored statements. This is probably a bit (or completely) broken in trunk. Can't we always generate the verbs with uses the names? Supposedly we can and IIRC Jim was addressing this in Vulcan. But before doing that, we should create name-object maps for all metadata caches to lookup efficiently. Dmitry -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
-Original Message- From: Alex Peshkoff [mailto:peshk...@mail.ru] Sent: Martes, 10 de Abril de 2012 4:06 Claudio, I had the following in mind. To transfer to required place the fact that BLR4 is used, we have the following sequence of identifiers involved: bool version4; void virtual isVersion4(bool); FLAG_VERSION4 // flag in JrdStatement csb_blr_version4 // flag in csb blr_version4 If your small class helps to avoid such solutions, it will be nice. I got rid of most of those identifiers (except the function isVersion) and kept the BLR number itself when needed (maybe this may be minimized, too). C. -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
11.04.2012 6:14, Claudio Valderrama C. wrote: Example of an API call? isc_compile_request, isc_transact_request, isc_dsql_*_m family. I assume that if someone wants to talk BLR v6, he should indicate that. Sure, any BLR stream starts with its version, be it on disk or at the API level. I just thought you gave up with the BLR v6 idea and had something different in mind. Dmitry -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
-Original Message- From: Alex Peshkoff [mailto:peshk...@mail.ru] Sent: Viernes, 06 de Abril de 2012 6:58 I agree here - not good. Currently we have to keep V4 flag in various forms just to convert datetime_to_text() in different manner. Only if we decide to have separate BLR6 parser... I just need a small class to read and write StreamType (I defined it) from a v4 or v5 BLR string. For v4, I would override those functions to read/write just one byte. Considering that now, few and very limited places are affected, having a separate BLR6 parser is unjustified. Using this helper class is what I would like to try to implement. What I have is my new type StreamType is all places I could find. I haven't touched the BLR generation and I haven't raised the BLR level (the engine performs same as before, but keeping the stream values in bigger vars, as Dmitry wanted). I'm ready to commit as I can't see a big bug caused by the change. The discussion on how to handle different BLR levels can continue (I have nothing to defend, because I don't have work done yet). C. -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 04/06/12 00:46, Ann Harrison wrote: On Mon, Mar 5, 2012 at 6:25 PM, Claudio Valderrama C. cva...@usa.net wrote: (Thorny issue, I hope Ann Harrison will comment.) And now, finally, a month later she does. Included after my signature is a bit of MySQL code which may explain my desire to keep things as simple as possible. Hello, currently the engine supports BLR4 (legacy) and BLR 5. All FB versions generate BLR 5. Sorry, but that's wrong. For Dialect1 BLR4 is generated. if (dialect SQL_DIALECT_V5) blr.add(blr_version5); else blr.add(blr_version4); My understanding and memory is that within a blr_rse, each stream has a context which is expressed as a single byte. My first reaction was to suggest adding a blr_rse2 using sixteen bit contexts. Obviously, even if that didn't trigger a BLR6 it would not be backward compatible, but there are other extensions to blr that haven't caused a version change and are also not backward compatible. Forward compatibility (i.e. the ability of a new engine to handle an old database) is critically important. If someone restores an old database, it may well have significant amounts of old blr in it. People really dislike a database that won't let them at their data - even if the data is old. Claudio has proposed increasing the blr version and changing those routines that manage blr to recognize that BLR4 and BLR5 have eight bit contexts but BLR6 has sixteen bit contexts. That's at least as clean as recognizing that blr_rse has eight bit contexts while blr_rse2 has sixteen bit contexts. What bothered me was the follow-on idea of fixing lots of housekeeping issues Things we can do in the new BLR version: - enlarge some values that are currently held in a single bit Which means testing for blr version on every reference to them. - allow for reuse of holes in the BLR namespace without risk of misinterpreting a deprecated verb Which means testing for blr version on every reference to them. I agree here - not good. Currently we have to keep V4 flag in various forms just to convert datetime_to_text() in different manner. Only if we decide to have separate BLR6 parser... -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On Mon, Mar 5, 2012 at 6:25 PM, Claudio Valderrama C. cva...@usa.net
wrote:
(Thorny issue, I hope Ann Harrison will comment.)
And now, finally, a month later she does. Included after my signature is a
bit of MySQL code which may explain my desire to keep things as simple as
possible.
Hello, currently the engine supports BLR4 (legacy) and BLR 5. All FB
versions generate BLR 5. But we are hitting some limits and I think we
should increase it again (this would be for the first time for FB).
My understanding and memory is that within a blr_rse, each stream has a
context which is expressed as a single byte. My first reaction was to
suggest adding a blr_rse2 using sixteen bit contexts. Obviously, even if
that didn't trigger a BLR6 it would not be backward compatible, but there
are other extensions to blr that haven't caused a version change and are
also not backward compatible.
Forward compatibility (i.e. the ability of a new engine to handle an old
database) is critically important. If someone restores an old database, it
may well have significant amounts of old blr in it. People really dislike
a database that won't let them at their data - even if the data is old.
Claudio has proposed increasing the blr version and changing those routines
that manage blr to recognize that BLR4 and BLR5 have eight bit contexts but
BLR6 has sixteen bit contexts. That's at least as clean as recognizing that
blr_rse has eight bit contexts while blr_rse2 has sixteen bit contexts.
What bothered me was the follow-on idea of fixing lots of housekeeping
issues
Things we can do in the new BLR version:
- enlarge some values that are currently held in a single bit
Which means testing for blr version on every reference to them.
- allow for reuse of holes in the BLR namespace without risk of
misinterpreting a deprecated verb
Which means testing for blr version on every reference to them.
- allow for BLR streams bigger than 64K thus supporting procedure BLR that
will be stored in multiple blob segments if necessary (AFAIK, gbak is
prepared to handle that).
Fixing the problem that gbak always gets or puts blr in a single read or
write is something I had on my list to do since the first
gds-Galaxy release. I hope it's finally been done.
Also, somewhat related to this, I propose that for 64-bit FB, the limit
MAX_REQUESTS_SIZE should be raised, too or estimated on the fly or put in
the config file.
That would also be a good thing.
Best wishes,
Ann
Here's the code ...
int
mysql_execute_command(THD *thd)
{
int res= FALSE;
bool need_start_waiting= FALSE; // have protection against global read lock
int up_result= 0;
LEX *lex= thd-lex;
/* first SELECT_LEX (have special meaning for many of non-SELECTcommands)
*/
SELECT_LEX *select_lex= lex-select_lex;
/* first table of first SELECT_LEX */
TABLE_LIST *first_table= (TABLE_LIST*) select_lex-table_list.first;
/* list of all tables in query */
TABLE_LIST *all_tables;
/* most outer SELECT_LEX_UNIT of query */
SELECT_LEX_UNIT *unit= lex-unit;
#ifdef HAVE_REPLICATION
/* have table map for update for multi-update statement (BUG#37051) */
bool have_table_map_for_update= FALSE;
#endif
/* Saved variable value */
DBUG_ENTER(mysql_execute_**command);
#ifdef WITH_PARTITION_STORAGE_ENGINE
thd-work_part_info= 0;
#endif
/*
In many cases first table of main SELECT_LEX have special meaning =
check that it is first table in global list and relink it first in
queries_tables list if it is necessary (we need such relinking only
for queries with subqueries in select list, in this case tables of
subqueries will go to global list first)
all_tables will differ from first_table only if most upper SELECT_LEX
do not contain tables.
Because of above in place where should be at least one table in most
outer SELECT_LEX we have following check:
DBUG_ASSERT(first_table == all_tables);
DBUG_ASSERT(first_table == all_tables first_table != 0);
*/
lex-first_lists_tables_same()**;
/* should be assigned after making first tables same */
all_tables= lex-query_tables;
/* set context for commands which do not use setup_tables */
select_lex-
context.resolve_in_table_list_**only((TABLE_LIST*)select_lex-
table_list.first);
/*
Reset warning count for each query that uses tables
A better approach would be to reset this for any commands
that is not a SHOW command or a select that only access local
variables, but for now this is probably good enough.
*/
if ((sql_command_flags[lex-sql_**command] CF_DIAGNOSTIC_STMT) != 0)
thd-warning_info-set_read_**only(TRUE);
else
{
thd-warning_info-set_read_**only(FALSE);
if (all_tables)
thd-warning_info-opt_clear_**warning_info(thd-query_id);
}
#ifdef HAVE_REPLICATION
if (unlikely(thd-slave_thread))
{
if (lex-sql_command == SQLCOM_DROP_TRIGGER)
{
/*
When dropping a trigger, we need to load its table name
Re: [Firebird-devel] Raising the BLR level
-Original Message- From: Ann Harrison [mailto:aharri...@nuodb.com] Sent: Martes, 06 de Marzo de 2012 19:10 Having started this discussion by agreeing with Claudio, now let me suggest that I was probably wrong. I'll think about it a bit more, but finding a way of extending blr compatibly seems like a much better idea. That lets old databases continue to work and avoids the whole discussion of what other features to drop. More tomorrow. Ann, I hope you will come with an interesting solution. However, let's recap: - If the BLR version number doesn't serve to upgrade BLR, then it's completely useless. - The original proposal was to upgrade BLR, nothing more. Other person wanted to drop BLR v4 and then dialect 1 and the discussion became fuzzy and out of bounds. The original proposal NEVER intended to remove the old BLR and the old dialect. - The only problem that a new BLR version doesn't solve is when you want to go back to a previous server version, but if you try to use some FB3's new DSQL feature in FB2.0, it's going to fail, right? So, what's different? Some new BLR verbs don't exist in the original FB1.0, despite the BLR version not being increased. When BLR v5 was invented, older servers speaking BLR v4 couldn't understand it... but the world didn't explode. C. -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 03/06/12 23:56, Dimitry Sibiryakov wrote: 06.03.2012 20:54, Adriano dos Santos Fernandes wrote: It will be 0.333... (up to the maximum NUMBER scale precision). The thing is that for operations like addition, subtraction, NUMBER will work like NUMERIC (i.e., it's results are more precise than DOUBLE). In this case much easier is to change intermediate result of division to floating-point type. No need to introduce new type for this. There is one more problem with this solution - migrating old databases to new datatypes is nightmare. -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
07.03.2012 10:03, Alex Peshkoff wrote: At least for today dialect 1 means blr 4, dialect 3 means blr 5. Suppose this can be reworked, but far not trivially. Weren't there plans to exclude BLR from chain SQL-BLR-EXE at all?.. -- SY, SD. -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 03/07/12 13:11, Claudio Valderrama C. wrote: -Original Message- From: Ann Harrison [mailto:aharri...@nuodb.com] Sent: Martes, 06 de Marzo de 2012 19:10 Having started this discussion by agreeing with Claudio, now let me suggest that I was probably wrong. I'll think about it a bit more, but finding a way of extending blr compatibly seems like a much better idea. That lets old databases continue to work and avoids the whole discussion of what other features to drop. More tomorrow. Ann, I hope you will come with an interesting solution. However, let's recap: - If the BLR version number doesn't serve to upgrade BLR, then it's completely useless. - The original proposal was to upgrade BLR, nothing more. Other person wanted to drop BLR v4 and then dialect 1 and the discussion became fuzzy and out of bounds. The original proposal NEVER intended to remove the old BLR and the old dialect. - The only problem that a new BLR version doesn't solve is when you want to go back to a previous server version, but if you try to use some FB3's new DSQL feature in FB2.0, it's going to fail, right? So, what's different? Some new BLR verbs don't exist in the original FB1.0, despite the BLR version not being increased. When BLR v5 was invented, older servers speaking BLR v4 couldn't understand it... but the world didn't explode. Claudio, today BLR version defines dialect used. That's why discussion was raised. Some related questions are raised: - Are we going to break this relationship with BLR version6? If not, are we going to introduce new dialect with ti? - If we break that relationship - what dialect will be used for requests in BLR6? Or probably we should use absolutely new method for deciding about request's dialect? I agree that to enhance limits changing BLR version looks like the most native approach. But unfortunately that version is also used to specify request's dialect. Therefore question about dialects was raised here. -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 03/07/12 13:05, Dimitry Sibiryakov wrote: 07.03.2012 10:03, Alex Peshkoff wrote: At least for today dialect 1 means blr 4, dialect 3 means blr 5. Suppose this can be reworked, but far not trivially. Weren't there plans to exclude BLR from chain SQL-BLR-EXE at all?.. There were such plans, but if we decide ti implement all plans in FB3 afraid we will not have it for too long time... -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
07.03.2012 10:14, Alex Peshkoff wrote: Some related questions are raised: If we talk only about BLR generated and executed inside of server, I would say generate BLR6 always for any SQL dialect, but with BLR generated or interpreted in client library there is a matter of compatibility with old servers. -- SY, SD. -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
06.03.2012 10:46, Alex Peshkoff wrote: And taking into an account that we do not support ODS 12 in current engine - do we need to keep support of old BLR versions? I see 2 ways for old BLR to arrive in ODS12 database. First of all - when old database is restored. May be we should learn gbak to recompile procedures/triggers/etc when doing that restore isntead restoring them? Next - in messages from old clients. This is probably the only place where we should keep old BLR versions support. The BLR API is still in place and can be used even by new clients. And many our utilities (gbak, gpre) speak BLR too. So currently there's no chance to get rid of old BLR versions. Dmitry -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 03/06/12 12:42, Dmitry Yemanov wrote: 06.03.2012 10:46, Alex Peshkoff wrote: And taking into an account that we do not support ODS 12 in current engine - do we need to keep support of old BLR versions? I see 2 ways for old BLR to arrive in ODS12 database. First of all - when old database is restored. May be we should learn gbak to recompile procedures/triggers/etc when doing that restore isntead restoring them? Next - in messages from old clients. This is probably the only place where we should keep old BLR versions support. The BLR API is still in place and can be used even by new clients. And many our utilities (gbak, gpre) speak BLR too. And what version BLR do they create? So currently there's no chance to get rid of old BLR versions. I do not speak about removing BLR API, but for how long are we going to support old versions of BLR? Is it about supporting dialect 1? Does it mean that in dialect 1 we keep old limits like number of streams? -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On Tue, 06 Mar 2012 13:03:04 +0400, Alex Peshkoff peshk...@mail.ru wrote: I do not speak about removing BLR API, but for how long are we going to support old versions of BLR? Is it about supporting dialect 1? Does it mean that in dialect 1 we keep old limits like number of streams? Isn't it about time to drop support for dialect 1? Mark -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
-Original Message- From: Kjell Rilbe [mailto:kjell.ri...@datadia.se] Sent: Martes, 06 de Marzo de 2012 6:55 Isn't it about time to drop support for dialect 1? Yes. But I'm sure there will be objections. I'd say the world moves forward - if you don't want it to, that's your problem - live with it! Meaning: if you really want to stay at dialect 1, then fine with me, but then live with the implications, e.g. staying with an old version of FB. And what do you suggest to do with backups of dialect 1 databases? Refuse to restore them? Try to fix stored procedures, triggers and views? C. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On Tue, 6 Mar 2012 07:14:44 -0300, Claudio Valderrama C. cva...@usa.net wrote: -Original Message- From: Kjell Rilbe [mailto:kjell.ri...@datadia.se] Sent: Martes, 06 de Marzo de 2012 6:55 Isn't it about time to drop support for dialect 1? Yes. But I'm sure there will be objections. I'd say the world moves forward - if you don't want it to, that's your problem - live with it! Meaning: if you really want to stay at dialect 1, then fine with me, but then live with the implications, e.g. staying with an old version of FB. And what do you suggest to do with backups of dialect 1 databases? Refuse to restore them? Try to fix stored procedures, triggers and views? Quite simple: refuse to restore dialect 1 database with a clear error message and advise people to migrate their database to dialect 3 on Firebird 2.5 or earlier. IMHO This should have happened earlier (eg with Firebird 2). Dropping this kind of deprecated baggage would be a good thing; especially as there is an alternative for people still using dialect 1 (afaik 12+ years after it was deprecated in favor of dialect 3). Mark -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
Den 2012-03-06 11:07 skrev Mark Rotteveel såhär: On Tue, 6 Mar 2012 07:14:44 -0300, Claudio Valderrama C. cva...@usa.net wrote: -Original Message- From: Kjell Rilbe [mailto:kjell.ri...@datadia.se] Sent: Martes, 06 de Marzo de 2012 6:55 Isn't it about time to drop support for dialect 1? Yes. But I'm sure there will be objections. I'd say the world moves forward - if you don't want it to, that's your problem - live with it! Meaning: if you really want to stay at dialect 1, then fine with me, but then live with the implications, e.g. staying with an old version of FB. And what do you suggest to do with backups of dialect 1 databases? Refuse to restore them? Try to fix stored procedures, triggers and views? Quite simple: refuse to restore dialect 1 database with a clear error message and advise people to migrate their database to dialect 3 on Firebird 2.5 or earlier. IMHO This should have happened earlier (eg with Firebird 2). Dropping this kind of deprecated baggage would be a good thing; especially as there is an alternative for people still using dialect 1 (afaik 12+ years after it was deprecated in favor of dialect 3). Exactly! In a world where computers and software evolve at blazing speed it seems rather much to request that a software such as Firebird be 12+ years backwards compatible. I might accept to keep the last dialect 1 supporting version to be kept alive a little while longer, for *very* critical bug fixes. For the rest: move *forward*! Kjell -- -- Kjell Rilbe DataDIA AB E-post: kj...@datadia.se Telefon: 08-761 06 55 Mobil: 0733-44 24 64 -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 06/03/2012 07:14, Claudio Valderrama C. wrote: -Original Message- From: Kjell Rilbe [mailto:kjell.ri...@datadia.se] Sent: Martes, 06 de Marzo de 2012 6:55 Isn't it about time to drop support for dialect 1? Yes. But I'm sure there will be objections. I'd say the world moves forward - if you don't want it to, that's your problem - live with it! Meaning: if you really want to stay at dialect 1, then fine with me, but then live with the implications, e.g. staying with an old version of FB. And what do you suggest to do with backups of dialect 1 databases? Refuse to restore them? Try to fix stored procedures, triggers and views? There is not only the problem of migrate older databases to new engine, but there is also the contrary problem. if people found problems in production that can't be workaround-ed easily, they must downgrade as fast as possible. If you create different BLR for compatible procedures, that downgrade will not be possible anymore with gbak. We need, I think, a safe approach of metadata extraction (I'm not sure ISQL is safe), and data extraction/reload. Adriano -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 06/03/2012 06:55, Kjell Rilbe wrote: Meaning: if you really want to stay at dialect 1, then fine with me, but then live with the implications, e.g. staying with an old version of FB. Completely agreed. Adriano -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 03/06/12 14:47, Adriano dos Santos Fernandes wrote: On 06/03/2012 07:14, Claudio Valderrama C. wrote: -Original Message- From: Kjell Rilbe [mailto:kjell.ri...@datadia.se] Sent: Martes, 06 de Marzo de 2012 6:55 Isn't it about time to drop support for dialect 1? Yes. But I'm sure there will be objections. I'd say the world moves forward - if you don't want it to, that's your problem - live with it! Meaning: if you really want to stay at dialect 1, then fine with me, but then live with the implications, e.g. staying with an old version of FB. And what do you suggest to do with backups of dialect 1 databases? Refuse to restore them? Try to fix stored procedures, triggers and views? There is not only the problem of migrate older databases to new engine, but there is also the contrary problem. if people found problems in production that can't be workaround-ed easily, they must downgrade as fast as possible. This problem will be present even if we keep both BLR 4 and 5, but add 6. Any change in database will add new BLR version code, and downgrade will fail. If you create different BLR for compatible procedures, that downgrade will not be possible anymore with gbak. We need, I think, a safe approach of metadata extraction (I'm not sure ISQL is safe) Unfortunately unsafe. It's very easy to get a script which fails. , and data extraction/reload. Probably new gbak mode (no-blr) is easier to do? We can add it to 2.5 for example. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 03/06/12 14:47, Adriano dos Santos Fernandes wrote: On 06/03/2012 06:55, Kjell Rilbe wrote: Meaning: if you really want to stay at dialect 1, then fine with me, but then live with the implications, e.g. staying with an old version of FB. Completely agreed. Also want to agree, but I remember there were some real reasons (unfortunately do not remember them) making people stay with dialect1. May be somebody remembers that reasons? -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
06.03.2012 15:05, Alex Peshkoff wrote: Also want to agree, but I remember there were some real reasons (unfortunately do not remember them) making people stay with dialect1. May be somebody remembers that reasons? Lack of precisions bigger than 18 in NUMERIC/DECIMAL and, even more important, although very related, the overflow issues for intermediate calculation results. Dmitry -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
-Original Message- From: Alex Peshkoff [mailto:peshk...@mail.ru] Sent: Martes, 06 de Marzo de 2012 8:06 Also want to agree, but I remember there were some real reasons (unfortunately do not remember them) making people stay with dialect1. May be somebody remembers that reasons? I think that the different treatment of date format (when converting to text) and datetime storage (double v/s int64) wasn't the primary motivation. The main reason was that dialect 3 imposes exact numerics rules for multiplication and division and we don't have enough bits to account for all cases (we would need internally int128 or somewhat alike). Also, division becomes C-like for integer numbers. C. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
06.03.2012 10:55, Kjell Rilbe wrote: I'm sure there will be objections. I'd say the world moves forward - if you don't want it to, that's your problem - live with it! Meaning: if you really want to stay at dialect 1, then fine with me, but then live with the implications, e.g. staying with an old version of FB. Starting from 3.0 there are plug-in engines. Anyone will be able to use newer versions of Firebird with old engine if necessary. Because of that I think that 3.0 may be the last version with Dialect 1 support. -- SY, SD. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
06.03.2012 10:30, Alex Peshkoff wrote: On 03/06/12 13:06, Dimitry Sibiryakov wrote: 06.03.2012 10:03, Alex Peshkoff wrote: Does it mean that in dialect 1 we keep old limits like number of streams? It means that you keep old limits for applications which use old BLR dialects. But it's the same: Ok, what I wanted to say: drop old BLR generation, but keep compilation into EXE nodes. -- SY, SD. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
-Original Message- From: Claudio Valderrama C. [mailto:cva...@usa.net] Sent: Tuesday, March 06, 2012 6:29 AM -Original Message- From: Alex Peshkoff [mailto:peshk...@mail.ru] Sent: Martes, 06 de Marzo de 2012 8:06 Also want to agree, but I remember there were some real reasons (unfortunately do not remember them) making people stay with dialect1. May be somebody remembers that reasons? I think that the different treatment of date format (when converting to text) and datetime storage (double v/s int64) wasn't the primary motivation. The main reason was that dialect 3 imposes exact numerics rules for multiplication and division and we don't have enough bits to account for all cases (we would need internally int128 or somewhat alike). Also, division becomes C-like for integer numbers. This is why BroadView hasn't made the move from Dialect 1. The numeric rules imposed by Dialect 3 would have required a huge amount of re-coding and testing to ensure that our calculations/reports generated the expected results. BTW, unless I have missed something, there is no posting which has said that Dialect 1 needs to be eliminated in order for the BLR version to be increased. So, changing the BLR levels has no dependencies on the question of on-going support for Dialect 1. The subject of dialects came up due to a question from Alex about whether the support for the old BLR versions was necessary for Dialect 1. As for Dialect 1, it is my opinion that the Dialect 3 intermediate numeric rules are half-baked (at best) and that until such time as they have been have been made intelligent that support for Dialect 1 needs to be maintained (do I hear a Dialect 4? == Dialect 1 math rules, plus 'new' Boolean, Date, Time, Timestamp and INT64 datatypes). Sean -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 06/03/2012 16:01, Leyne, Sean wrote: This is why BroadView hasn't made the move from Dialect 1. The numeric rules imposed by Dialect 3 would have required a huge amount of re-coding and testing to ensure that our calculations/reports generated the expected results. BTW, unless I have missed something, there is no posting which has said that Dialect 1 needs to be eliminated in order for the BLR version to be increased. So, changing the BLR levels has no dependencies on the question of on-going support for Dialect 1. The subject of dialects came up due to a question from Alex about whether the support for the old BLR versions was necessary for Dialect 1. As for Dialect 1, it is my opinion that the Dialect 3 intermediate numeric rules are half-baked (at best) and that until such time as they have been have been made intelligent that support for Dialect 1 needs to be maintained (do I hear a Dialect 4? == Dialect 1 math rules, plus 'new' Boolean, Date, Time, Timestamp and INT64 datatypes). I think the solution (not hacks with dialects) to this problem is not difficult. Firebird don't need to change the way it does numeric/decimal arithmetics and introduce another dialect leaving current and previous one for decades. Firebird is right on the way it does. So, add a new datatype NUMBER(x, y) who does BCD arithmetics like it's done in Oracle. No secret then. Everybody would be pleased without these dialect hacks which make a single thing to act in different ways. Adriano -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
06.03.2012 20:31, Adriano dos Santos Fernandes wrote: So, add a new datatype NUMBER(x, y) who does BCD arithmetics like it's done in Oracle. What this data type will change? What will be result of 1/3 for this data type? If 0, it won't be different from DECIMAL. If 0.(3), how much 3 will it contain? -- SY, SD. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 06/03/2012 16:37, Dimitry Sibiryakov wrote: 06.03.2012 20:31, Adriano dos Santos Fernandes wrote: So, add a new datatype NUMBER(x, y) who does BCD arithmetics like it's done in Oracle. What this data type will change? What will be result of 1/3 for this data type? If 0, it won't be different from DECIMAL. If 0.(3), how much 3 will it contain? It will be 0.333... (up to the maximum NUMBER scale precision). The thing is that for operations like addition, subtraction, NUMBER will work like NUMERIC (i.e., it's results are more precise than DOUBLE). Adriano -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
06.03.2012 20:54, Adriano dos Santos Fernandes wrote: It will be 0.333... (up to the maximum NUMBER scale precision). The thing is that for operations like addition, subtraction, NUMBER will work like NUMERIC (i.e., it's results are more precise than DOUBLE). In this case much easier is to change intermediate result of division to floating-point type. No need to introduce new type for this. -- SY, SD. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
I think the solution (not hacks with dialects) to this problem is not difficult. Firebird don't need to change the way it does numeric/decimal arithmetics and introduce another dialect leaving current and previous one for decades. Firebird is right on the way it does. So, add a new datatype NUMBER(x, y) who does BCD arithmetics like it's done in Oracle. Isn't introducing a non-standard datatype a hack in its own right? What about the other Dialect 3 data types which do not exist in Dialect 1? Sean -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
06.03.2012 20:54, Adriano dos Santos Fernandes wrote: It will be 0.333... (up to the maximum NUMBER scale precision). The thing is that for operations like addition, subtraction, NUMBER will work like NUMERIC (i.e., it's results are more precise than DOUBLE). In this case much easier is to change intermediate result of division to floating-point type. No need to introduce new type for this. Division is not the only difference with math between Dialect 1 and 3 Dialect 1 select 2/3 from rdb$database = 0.667 select 2 * 1.001 from rdb$database = 2.000 Dialect 3 select 2/3 from rdb$database = 0 select 2 * 1.001 from rdb$database = 2. 002 Sean -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
06.03.2012 21:27, Leyne, Sean wrote: select 2 * 1.001 from rdb$database = 2.000 This is just a display artefact. Precision of double is enough to keep 11 significant digits. -- SY, SD. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
Having started this discussion by agreeing with Claudio, now let me suggest that I was probably wrong. I'll think about it a bit more, but finding a way of extending blr compatibly seems like a much better idea. That lets old databases continue to work and avoids the whole discussion of what other features to drop. More tomorrow. Ann -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2dFirebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
Claudio,
Hello, currently the engine supports BLR4 (legacy) and BLR 5. All FB
versions generate BLR 5. But we are hitting some limits and I think we
should increase it again (this would be for the first time for FB). Dmitry
asked me to get rid of the 255 streams limit but what I did is only the
starting point.
Problematic places that I marked in the code:
ExprNodes.cpp:
// CVC: bottleneck
const StreamType streamCount = csb-csb_blr_reader.getByte();
for (StreamType i = 0; i streamCount; ++i)
{
const USHORT n = csb-csb_blr_reader.getByte();
node-internalStreamList.add(csb-csb_rpt[n].csb_stream);
}
Number of streams is limited to 255, despite me lifting the restrictions in
other places.
Again, ExprNodes.cpp, this looks like the complementary part:
// bottleneck
fb_assert(stack.object()-ctx_context = MAX_UCHAR);
dsqlScratch-appendUChar(stack.object()-ctx_context);
RecordSourceNodes.cpp
// bottleneck
int count = (unsigned int) csb-csb_blr_reader.getByte();
// Pick up the sub-RseNode's and maps.
while (--count = 0)
There may be other places I'm not aware of. The important idea is that BLR
is expected to hold those values in single bytes and this is not enough
anymore. I see raising the BLR version as the only solution.
I think you're right.
Ann
--
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Raising the BLR level
On 03/06/12 03:42, Ann Harrison wrote:
Claudio,
Hello, currently the engine supports BLR4 (legacy) and BLR 5. All FB
versions generate BLR 5. But we are hitting some limits and I think we
should increase it again (this would be for the first time for FB). Dmitry
asked me to get rid of the 255 streams limit but what I did is only the
starting point.
Problematic places that I marked in the code:
ExprNodes.cpp:
// CVC: bottleneck
const StreamType streamCount = csb-csb_blr_reader.getByte();
for (StreamType i = 0; i streamCount; ++i)
{
const USHORT n = csb-csb_blr_reader.getByte();
node-internalStreamList.add(csb-csb_rpt[n].csb_stream);
}
Number of streams is limited to 255, despite me lifting the restrictions in
other places.
Again, ExprNodes.cpp, this looks like the complementary part:
// bottleneck
fb_assert(stack.object()-ctx_context = MAX_UCHAR);
dsqlScratch-appendUChar(stack.object()-ctx_context);
RecordSourceNodes.cpp
// bottleneck
int count = (unsigned int) csb-csb_blr_reader.getByte();
// Pick up the sub-RseNode's and maps.
while (--count = 0)
There may be other places I'm not aware of. The important idea is that BLR
is expected to hold those values in single bytes and this is not enough
anymore. I see raising the BLR version as the only solution.
I think you're right.
And taking into an account that we do not support ODS 12 in current
engine - do we need to keep support of old BLR versions? I see 2 ways
for old BLR to arrive in ODS12 database. First of all - when old
database is restored. May be we should learn gbak to recompile
procedures/triggers/etc when doing that restore isntead restoring them?
Next - in messages from old clients. This is probably the only place
where we should keep old BLR versions support.
--
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
