Re: [Firebird-devel] IAttachment::execute() returned value
On 9/17/22 18:54, Dimitry Sibiryakov wrote: Hello All. What are rules for plugin's returned value of IAttachment::execute() in the case when input transaction is not NULL? That depends upon statement executed. Should it return NULL? In a case of commit/rollback - yes. Should it return the original transaction? For all non-transacton-control statements - yes. Should it return current plugin's original transaction (i.e. YTransaction->next if input transaction is from Y-valve)? No matter of YValve - when SQL statement does not modify transaction (not start and not commit) same value to be returned. Should reference counter of the returned value to be increased if the transaction is the same as input one? Not. When new transaction is created it's ref/counter is set to 1. (This exactly matches logic of assignRefNoIncr() function.) Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] test
TEST Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Sanity checks for methods' parameters in Y-valve
On 9/15/22 18:56, Dimitry Sibiryakov wrote: Hello All. If someone by mistake calls IStatement::execute() providing metadata but no data buffer, it will pass the values as is to a provider and overall result will be segfault. Shouldn't some sanity checks to be performed there? Not sure. Check for nullptr is really simple but what if user sends garbage instead data buffer address? Luckily that segfault will never disturb remote server - only client or embedded which can suffer same way from any segfault in user process. I do not want to say segfault is good thing and will not provide as a sample trivial things like strcpy() - but almost any function in C library will segfault when passed wrong address of some data. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/15/22 17:18, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 15.09.2022 16:14: On the other hand tomcrypt is successfully used in our sample plugin - which is problematic with static linking. For me that appears more interesting and useful. Not to say about code duplication. Dead code elimination with static libraries work at module level so there is no significant duplication. Crypt modules are tiny. We do not include crypt algorithms not needed for firebird when building tomcrypt (library's customization is used). Therefore there is not too much effect from code elimination. May be in client-only case... But have you seen a lot of clients running linux? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/15/22 17:07, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 15.09.2022 12:04: With tommath there are no such issues... Why don't force build with static libs? In what way it's better than own dynamic libraries? Surely no problem with API/ABI. Therefore we link some problematic libraries statically. No mess and vulnerability with dynamic library search path. I see no mess with it but it depends on POV No multiplying of points above with every third-party plugin. May be - never used to deal with third-party plugins myself. On the other hand tomcrypt is successfully used in our sample plugin - which is problematic with static linking. For me that appears more interesting and useful. Not to say about code duplication. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/15/22 16:38, Mark Rotteveel wrote: On 15-09-2022 15:19, Alex Peshkoff via Firebird-devel wrote: On 9/15/22 14:36, Adriano dos Santos Fernandes wrote: On 15/09/2022 08:18, Alex Peshkoff via Firebird-devel wrote: This is solved by install script - it creates symlink to /inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory (provided it's missing). If we decide to _always_ provide own dynamic libraries (logically it's like using static one but does not require duplicating code) this trick becomes not needed any more. Let me remind you that this thread started with me noticing an inconsistency in how we build things when explicitly building with --with-builtin-tommath compared to the - in my opinion - more sane behaviour of --with-builtin-tomcrypt (putting things in gen/buildroot/usr/local/firebird/lib/.tm vs in gen/buildroot/usr/local/firebird/lib). I was not asking about always using our own libraries, but at least doing things in a sane way when a build is performed explicitly requesting to use our own version of libraries. We need another switch for requested behavior. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/15/22 14:36, Adriano dos Santos Fernandes wrote: On 15/09/2022 08:18, Alex Peshkoff via Firebird-devel wrote: On 9/15/22 14:00, Adriano dos Santos Fernandes wrote: Em qui., 15 de set. de 2022 07:06, Alex Peshkoff via Firebird-devel escreveu: > > Why don't force build with static libs? > In what way it's better than own dynamic libraries? Not put files in system lib outside of a package manager. Using own dynamic libraries does not require putting something into system lib . I'm talking about this: This is solved by install script - it creates symlink to /inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory (provided it's missing). If we decide to _always_ provide own dynamic libraries (logically it's like using static one but does not require duplicating code) this trick becomes not needed any more. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/15/22 14:00, Adriano dos Santos Fernandes wrote: Em qui., 15 de set. de 2022 07:06, Alex Peshkoff via Firebird-devel escreveu: > > Why don't force build with static libs? > In what way it's better than own dynamic libraries? Not put files in system lib outside of a package manager. Using own dynamic libraries does not require putting something into system lib . Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/15/22 12:30, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 15.09.2022 11:08: Also, it's different in the sense that official package builds and package tomcrypt, but not tommath. We did discuss sometimes ago why for universal binary package we have to use built-in tomcrypt. Data structures, passed In it's API, depend upon compilation flags. It's pretty easy to get an error in API call or even segfault due to it. Myself lost a lot of time when started to work with it and did not have correct defines when included tomcrypt.h. (At least it was so 5 years ago - may be fixed in fresh versions?) With tommath there are no such issues... Why don't force build with static libs? In what way it's better than own dynamic libraries? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/15/22 04:52, Adriano dos Santos Fernandes wrote: On 05/09/2022 08:13, Mark Rotteveel wrote: On 05-09-2022 13:00, Alex Peshkoff via Firebird-devel wrote: When adding it I cared about an ability to fix secirtiy issues in 3d-party libraries not rebuilding and reinstalling FB packages. With used schema one can simply install fixed system package for tommath in a case of some issues in it. None were found - but that was when tommath was at pre-release stage. I find these inconsistencies quite confusing, but lets leave it at that. Also, it's different in the sense that official package builds and package tomcrypt, but not tommath. We did discuss sometimes ago why for universal binary package we have to use built-in tomcrypt. Data structures, passed In it's API, depend upon compilation flags. It's pretty easy to get an error in API call or even segfault due to it. Myself lost a lot of time when started to work with it and did not have correct defines when included tomcrypt.h. (At least it was so 5 years ago - may be fixed in fresh versions?) With tommath there are no such issues... Should this be changed and be consistent in FB 5, i.e., build and package both? May be it's really simplest solution... Let's first check stability of current tomcrypt API - may be it's OK now and we may do not package both libraries. BTW, we always recommended to those who need embedded-only access full install (classic) and server stop. Embedded (just built, not installed) works very ok for me. What should be the problem? Lock directory access only? Yes, group firebird should be created. In a case when single-user access is needed - everything works fine w/o it. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] getInfo() result format
On 9/8/22 12:49, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 07.09.2022 18:40: Do not remember - does ClumpletReader already supports getInfo() result format? Sorry, can't remember. Supports, down to FB3. I.e. looks like we need an src review. Add a ticket please. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] getInfo() result format
On 9/7/22 19:33, Dimitry Sibiryakov wrote: At fourth line you can get read-past-buffer and may be crash. And this code is copy-pasted everywhere in Firebird source. Do not remember - does ClumpletReader already supports getInfo() result format? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] getInfo() result format
On 9/7/22 19:18, Dimitry Sibiryakov wrote:
Hello All.
isc_info_truncated is a terminal item which is NOT followed by two
bytes length, right?
Not sure what you mean by 'terminal item', but it may be followed by
isc_info_end if that fits into buffer.
What about two bytes length - it's single-byte item not followed by
anything.
see how it's added:
bool Service::ck_space_for_numeric(UCHAR*& info, const UCHAR* const end)
{
if ((info + 1 + sizeof(ULONG)) > end)
{
if (info < end)
*info++ = isc_info_truncated;
if (info < end)
*info++ = isc_info_end;
return false;
}
return true;
}
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/5/22 13:20, Mark Rotteveel wrote: On 05-09-2022 12:06, Alex Peshkoff via Firebird-devel wrote: On 9/5/22 12:59, Mark Rotteveel wrote: On 05-09-2022 11:50, Alex Peshkoff via Firebird-devel wrote: On 9/5/22 12:31, Mark Rotteveel wrote: That is inconsistent compared to what is done for --with-builtin-tomcrypt. I explicitly requested --with-builtin-tommath, so I expect it to end up directly in the lib directory like tomcrypt. This switch works another way. It seems to me it does the same thing, but for a different library, so why does it do so in an entirely different way? I do not know what library do you talk about. There are many of them - and rules for them vary slightly. I explicitly mention it multiple times (see also above): tomcrypt. The behaviour for --with-builtin-tomcrypt (putting it directly in lib) seems more sane to me compared to what is done for --with-builtin-tommath (putting it in lib/.tm). Why is it handled differently? When adding it I cared about an ability to fix secirtiy issues in 3d-party libraries not rebuilding and reinstalling FB packages. With used schema one can simply install fixed system package for tommath in a case of some issues in it. None were found - but that was when tommath was at pre-release stage. BTW, we always recommended to those who need embedded-only access full install (classic) and server stop. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/5/22 12:59, Mark Rotteveel wrote: On 05-09-2022 11:50, Alex Peshkoff via Firebird-devel wrote: On 9/5/22 12:31, Mark Rotteveel wrote: On 05-09-2022 11:23, Alex Peshkoff via Firebird-devel wrote: This is solved by install script - it creates symlink to /inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory (provided it's missing). That's done this way in order to make it possible to use system tommath when present. That is inconsistent compared to what is done for --with-builtin-tomcrypt. I explicitly requested --with-builtin-tommath, so I expect it to end up directly in the lib directory like tomcrypt. This switch works another way. It seems to me it does the same thing, but for a different library, so why does it do so in an entirely different way? I do not know what library do you talk about. There are many of them - and rules for them vary slightly. If I understand you correctly, I will need to massage what I have in buildroot even further to get a usable build. I won't be using the install script, because I don't want to install it, I want to package it for an embedded deployment. Yes, you need to change something not suitable for embedded, I even expect you to add configure switch for it - we never deployed FB for linux w/o install script. We'll see, if I can find the motivation and energy to figure this out. If you just shell scripts and send me changes I can do the rest myself... Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/5/22 12:31, Mark Rotteveel wrote: On 05-09-2022 11:23, Alex Peshkoff via Firebird-devel wrote: This is solved by install script - it creates symlink to /inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory (provided it's missing). That's done this way in order to make it possible to use system tommath when present. That is inconsistent compared to what is done for --with-builtin-tomcrypt. I explicitly requested --with-builtin-tommath, so I expect it to end up directly in the lib directory like tomcrypt. This switch works another way. If I understand you correctly, I will need to massage what I have in buildroot even further to get a usable build. I won't be using the install script, because I don't want to install it, I want to package it for an embedded deployment. Yes, you need to change something not suitable for embedded, I even expect you to add configure switch for it - we never deployed FB for linux w/o install script. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 9/3/22 16:12, Mark Rotteveel wrote: I'm investigating ways of packaging Firebird Embedded for Java, so it can be used as drop-in Java library (which is then unpacked to a temporary directory automatically). Given Firebird complains about missing libtommath when I use the normal build, and I don't want to impose on users to install additional libraries, I tried a custom build (from current state of v4.0-release branch) with: ./autogen.sh --with-builtin-tommath --with-builtin-tomcrypt --enable-binreloc make make dist I have a problem with libtommath in this build: gen/buildroot/usr/local/firebird/lib contains libtomcrypt.so.1.0.1 (and symlinks libtomcrypt.so and libtomcrypt.so.1), but no libtommath files, these are in a .tm subdirectory of lib, so gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so.0 gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so.0.0.41 gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so I would have expected these files directly in gen/buildroot/usr/local/firebird/lib/, not in gen/buildroot/usr/local/firebird/lib/.tm/. Trying to load embedded this way complains about missing libtommath. This is solved by install script - it creates symlink to /inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory (provided it's missing). That's done this way in order to make it possible to use system tommath when present. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Service errors returned by API
On 8/25/22 21:52, Dimitry Sibiryakov wrote: Perhaps warnings should be returned both ways: the binary form in status and duplicate text in output. In this case applications will get them as usual and advanced ones may be interested in parsing status. I will look at what can be done here. Putting warnings (if any) into status vector is definitely not a problem, duplicating them into text output too, what may be not trivial is messages currently printed by burp-like services to stdX. Certainly we may require analyzing warnings from clients who want to receive verbose messages in machine-readable form. Is it what you want? Rather no. As written above - messages belong to (one of) output streams (I believe services API should be able to deliver multiple streams in parallel: messages and binary stream for backup as an example). I was thinking about it. Suppose adding N buffers for stream data instead current one should not be too hard to do. But what about API for that, format of data sent to user, etc. - that should be well though before starting with this. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Service errors returned by API (was: [FirebirdSQL/firebird] [FR] Database restore must make every effort on activating deferred indexes (Issue #7269))
On 8/24/22 20:59, Dimitry Sibiryakov wrote: Alexander Peshkov wrote 24.08.2022 19:06: On 8/24/22 14:31, Dimitry Sibiryakov wrote: > Without it services seem to be pretty useless. I think this to be suggested in devel and if nobody sees problems - feel free to add a ticket, I will implement it. Ok. But I don't think that a simple hack returning only thread exit code is a right way to go. If I was thinking that way that feature could be present since 2. 5 :-) Notice - in status vector it will be a notice that service thread failed with exit code NNN, original message with a reason for is unknown at the final point. It is not quite try. In the old dark days when the service thread just run an external utility and mindlessly redirect whole output to the client it was so. But nowadays the thread is doing whole work itself Just small notice - an ability to return errors in SV does not directly depend upon does service run as a process or as a thread. It's possible to pass SV cross-process but nobody cared about. With service threads pre-2.5 it was a mix - some errors were returned as SV, some - as text lines. so it can return a proper errors, warnings and verbose messages through isc_svc_query() as soon as they happen. But here are some problems: - An error when querying service is treated by 99% of callers (starting with fbsvcmgr) as fatal one. Other errors will not be displayed. - What about warnings - yes, they probably may be used at the first glance. But once again big problems with old clients. What should return isc_service_query - success or failure? On failure one prints status and detachs from service, on success - ignores SV. - Verbose messages are already passed through isc_svc_query(). Certainly we may require analyzing warnings from clients who want to receive verbose messages in machine-readable form. Is it what you want? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 8/24/22 19:33, Jiří Činčura wrote: How crazy idea it would be to have even Linux builds with binaries? May be I've lost context - but we release linux builds with binaries since the first days of the project. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 8/24/22 17:14, Mark Rotteveel wrote: On 24-08-2022 16:06, Alex Peshkoff via Firebird-devel wrote: On 8/24/22 17:04, Mark Rotteveel wrote: On 24-08-2022 15:36, Dimitry Sibiryakov wrote: Mark Rotteveel wrote 24.08.2022 15:32: If the term is confusing or ambiguous, it already is so in its current form. Yes, it is. That's why I would suggest to change that. The Windows pdb packages are complete builds though. Same for linux - not stripped binaries provided. The Linux builds are not complete builds compared to Windows pdb builds (e.g. config files, documentation, security database, etc are missing) Yep, they contain only binaries with debug-info. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 8/24/22 17:10, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 24.08.2022 16:06: Same for linux - not stripped binaries provided. Nope. Firebird-debuginfo-4.0.2.2816-0.amd64.tar.gz contains only extracted debug symbols, not binaries. Sorry, but they are created by 'cp' command ;) One can run them directly. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 8/24/22 17:04, Mark Rotteveel wrote: On 24-08-2022 15:36, Dimitry Sibiryakov wrote: Mark Rotteveel wrote 24.08.2022 15:32: If the term is confusing or ambiguous, it already is so in its current form. Yes, it is. That's why I would suggest to change that. The Windows pdb packages are complete builds though. Same for linux - not stripped binaries provided. So maybe those should be "debug", while the Linux builds should be "debug-symbols" or something like that. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 8/24/22 15:03, Adriano dos Santos Fernandes wrote: On 24/08/2022 08:59, Mark Rotteveel wrote: On 24-08-2022 13:45, Adriano dos Santos Fernandes wrote: This is our Firebird 4.0.2 released files: [..] It's difficult to understand, they do not mention OS and hardware architectures are inconsistent. I propose this name convention starting with Firebird 5: [..] Firebird-5.0.0.2816-0-linux-x64.tar.gz Firebird-5.0.0.2816-0-linux-x64-debuginfo.tar.gz Firebird-5.0.0.2816-0-linux-arm.tar.gz (armv7, other?) Firebird-5.0.0.2816-0-linux-arm-debuginfo.tar.gz Firebird-5.0.0.2816-0-linux-arm64.tar.gz Firebird-5.0.0.2816-0-linux-arm64-debuginfo.tar.gz [..] In general I'm OK with this, but currently our download page claims the ARM32 and ARM64 build are for Android, and not generically Linux. So, are they actually generically Linux, or specifically for Android? You might be correct. Alex can certainly explain better. Specifically for Android. I suppose this to be reflected in the name cause it's quite possible that we will provide arm64 builds for generic linux. I'm also not sure about arm64 vs aarch64 differences. That's "serious" difference - exactly like between amd64, x86_64 and x64 :) Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Purpose of op_batch_sync
On 8/19/22 19:02, Mark Rotteveel wrote: On 19-08-2022 17:30, Alex Peshkoff via Firebird-devel wrote: On 8/19/22 18:24, Mark Rotteveel wrote: It looks like op_batch_sync is just used to force the client to send the deferred messages to the server, and the server to send the deferred responses. Is that correct? exactly So, just for confirmation, using op_ping (which I found earlier as a trick to get the server to send deferred messages) is a similar solution? May be - never tried :) Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Purpose of op_batch_sync
On 8/19/22 18:24, Mark Rotteveel wrote: It looks like op_batch_sync is just used to force the client to send the deferred messages to the server, and the server to send the deferred responses. Is that correct? exactly Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Do repeated op_batch_create leak the batch?
On 8/13/22 14:59, Mark Rotteveel wrote: I'm implementing batch execution in Jaybird. Looking at the code of rem_port::batch_create(P_BATCH_CREATE* batch, PACKET* sendL) in server.cpp, it looks like sending multiple op_batch_create requests for the same statement handle could leak a previously created batch, as it doesn't call statement->rsr_batch->release() before assigning a new batch to statement->rsr_batch. Is my assessment correct? Seems to be so. In the engine there is a check avoiding opening batch when batch or cursor is already opened, sane should be added to remote server. Add a ticket please. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] BLOB_APPEND and NULL
On 8/11/22 19:04, Vlad Khorsun wrote: So far I don't see it as mistake. +1 Function is anyway damned non-standard. And as soon as we have documented behavior - no problems at all. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Ability to use non-local protocol to create DB which alias is declared as self-security
On 8/10/22 10:36, Pavel Cisar wrote: Hi, Dne 09. 08. 22 v 17:02 Alex Peshkoff via Firebird-devel napsal(a): *Pavel & Pavel!* I understand you have some problems with testing system due to unablity to create self-security database remotely. But I do not understand how could as solution be suggested not to check credetioals at all, i.e. let everyone create databases. That's like let everyone atach to server and enter SYSDBA password when security database failed to proper initialize at install phaze for some reason. We used to fight for FB security for many years (almost twenty) and it's very strange for me to here such suggestions. We added special privilege for create database in order to avoid creation of databases by regular users. And after it suggestion to let do it everyone - at least for some databases... I did not suggested to do not authenticate user! My question was why user has to be authenticated AGAINST YET TO BE CREATED database. It simply doesn't make sense at all for me. I would expect that for create database, the user would be authenticated in the same way as for attachment to service manager. You wrongly understand how does security in service manager work. Yes, you *may* do not mention database you plan to work with as long as that database is served by default security database. But try yourself to attach to service manager not specifying what database you plan to work with and after it backup some self-security database. The reason is that user does not need to have any prior valid attachment to call create database, so credentials passed for create database should be used to verify that such database could be created for that user, but not the database specification itself! It should be used for connection to created database, but not for creation. It's IMHO illogical that setting for not yet created database are used to do that (which fails when created db is self-security) instead default security database. Once again - credentials are used not to check an ability to attach or create database (or use services manager). When server verifies them it has no idea about particular operation, just an ability to attach to server is checked. The creation of self-security db via local (i.e. bypass of authentication check) is IMHO a hackish workaround that beats the purpose of security, because self-security databases could NOT be ever created in insecure way. You mix local (xnet) protocol and embedded connection. An ability to establish embedded connecton means that one is able to start processes on the host (like hexeditor for example) and has R/W access to database file (or a directory where it's going to be created). Also one can just copy earlier created database under the name mentioned in databases.conf - and that is perfectly good way to go. I.e. when embedded connection is used user does not create database on firebird server, he just creates a file on the host, running firebird server. FB tools (like isql) may be used for it or may be not. Therefore it's impossible to talk about beatibg FB security - it just does not work here because it can not work! What about 'hackish workaround' I do not agree - embedded access to databases is our documented feature and recommended in some cases and actively used by some users for a long time. I can think about letting user, authenticated in default security database with appr.privileges, create self-security databases remotely. But definitely not what was suggested. Yes, that was suggested, just probably wrongly worded. OK, in that case looks like we have some common understanding. Just please forget that ... I though that when referencing that right to create database should be checked in the same way as attachment to service manager (which does not use some database reference passed by user to decide which security db to use) instead database that does not exists yet is quite clear. attachment to services manager let's one bypass per-database security checks :-) But that change does not appear to be trivial - it should be done not to break security architecture. Just for example - what to do when database is created with overwrite option? Where to check credentials on server attach? Remember - at that step we have no idea why does user wants to attach to server, moreover - is overwrite option used. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Ability to use non-local protocol to create DB which alias is declared as self-security
On 8/9/22 17:14, Pavel Cisar wrote: Hi, Dne 09. 08. 22 v 6:33 Alex Peshkoff via Firebird-devel napsal(a): Why this restriction exists ? Because login / password to be validated in the database that does not exist yet. Therefore password check fails and you can't attach to server. Without connection to server it's hard to create something on it ;) I really don't understand WHY it's necessary to authenticate user against some database for *create database*. User to be authenticated to avoid creation of databases by unknown connections. Your question contains very right words - "against some database". When authentication is done using dedicated security database nobody says - do not perform authentication for 'create database'. But when self database is used - somewhy that appears extra. One sample - imagine a case when admin created new alias and user for whom alias was created tries should create database itself. Meantime malicious user connects to server w/o authentication and creates that database itself. Next he adds his login with known only to him password to newly created database (why not - he is DBO) and user for whom alias was created can't connect to that database. It's definitely not necessary to provide any kind of database reference to attach to service manager. Because by default we use defaut security.db. I understand that create database also works as connect (i.e. returns attachment to database), To be precise - to execute any of CONNECT or CREATE one should first of all establih connection with server, and valid credentials are needed at this step. If credentials are passed but should be validated against missing database - server can't check if they valid or not. so it will use the same credentials for such connection to created database in normal way, but to authenticate the right to invoke the database creation itself, no database should be needed (like it's not needed for attachment to service manager). Try to attach to service manager and do something with any self-security database. You will see that it's also impossible. *Pavel & Pavel!* I understand you have some problems with testing system due to unablity to create self-security database remotely. But I do not understand how could as solution be suggested not to check credetioals at all, i.e. let everyone create databases. That's like let everyone atach to server and enter SYSDBA password when security database failed to proper initialize at install phaze for some reason. We used to fight for FB security for many years (almost twenty) and it's very strange for me to here such suggestions. We added special privilege for create database in order to avoid creation of databases by regular users. And after it suggestion to let do it everyone - at least for some databases... I can think about letting user, authenticated in default security database with appr.privileges, create self-security databases remotely. But definitely not what was suggested. Alex. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Ability to use non-local protocol to create DB which alias is declared as self-security
On 8/9/22 12:41, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 09.08.2022 6:33: PPS. If one has access to database.conf to create new alias he definitely has embedded access to server, i.e. problem appears to be rather artificial. Consider a shared database hosting where aliases are created by admin for every user. Agreed, possible scenario (on practice currently everyone is using VM's, but anyway ...). And a case when such databases use self-security also fits well into such schema. But in that case to create database (and let people perform some recovery when for example were revoked admin rights from all users in database) such hosting needs special authentication plugin able to validate per-database "super"-users and grant them on login full access to apropriate database(s). Writing such plugin (based on SRP but using another databases access rules) is possible. But is it really actual? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Ability to use non-local protocol to create DB which alias is declared as self-security
On 8/8/22 15:41, Pavel Zotov wrote:
Currently one need to use only local connection in order to create
self-security database, i.e.:
1) we create in the databases.conf alias of DB that we want to create,
e.g.:
tmp_self= $(dir_sampleDb)/tmp_self_example.fdb
{
SecurityDatabase = tmp
}
2) we run "create database" statement and specify there alias
'tmp_self', *without* protocol prefix ('localhost:', 'inet://',
'xnet://'):
D:\FB\fb308>isql -q
SQL> create database 'tmp_self'; set list on; select
mon$database_name, mon$owner, mon$sec_database from mon$database; drop
database;
. . .
MON$DATABASE_NAME D:\FB\FB308\EXAMPLES\EMPBUILD\TMP_SELF_EXAMPLE.FDB
MON$OWNER ZOTOV
MON$SEC_DATABASE Self
So far so good.
But if we try to create DB using command like this:
=
create database 'localhost:tmp_self' user sysdba password 'masterkey';
drop database;
=
-- then error raises:
SQL> Statement failed, SQLSTATE = 08006
Error occurred during login, please check server firebird.log for
details
Content of firebird.log in that case will be following:
Authentication error
I/O error during "CreateFile (open)" operation for file "tmp_self"
Error while trying to open file
[ can not find specified file ] // localized message here
Why this restriction exists ?
Because login / password to be validated in the database that does not
exist yet. Therefore password check fails and you can't attach to
server. Without connection to server it's hard to create something on it ;)
Database file did not exist also when we used local protocol.
From DS:
Have you forgot that it means embedded mode and user name/password
are not checked at all?
And yes - when there is no need to check login info there is no need to
attach to non-existent database.
So, what's the problem just to ignore this (i.e. missed .fdb file) it
and try to create this DB ?
The problem is need to follow security rule - each conection to server
to be validated by plugins.
PS. 2 more cents.
One can create database with self-security alias remotely if some
plugin, configured on server, does not need to attach to it. But for
tody we do not have one - even win_sspi needs DB (and security DB -
which is same database in your case) connection to load mappings.
PPS. If one has access to database.conf to create new alias he
definitely has embedded access to server, i.e. problem appears to be
rather artificial.
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Is batch release necessary in the wire protocol?
On 7/26/22 18:30, Leyne, Sean wrote: -Original Message- From: Alex Peshkoff via Firebird-devel On 7/26/22 14:39, Mark Rotteveel wrote: On 16-07-2022 10:54, Mark Rotteveel wrote: Is sending op_batch_rls necessary in the wire protocol, or is closing the statement itself sufficient? Similar question, is a batch automatically released when unpreparing the statement, or preparing a new statement text? Yes, it's released. Alex, You didn't really answer the "when" part of the question. Is the statement released: (a) when the statement is unprepared, OR (b) when new statement text is prepared, OR (c) both (a) and (b) Sean, yes to both. From server POV both 'unprepared' and 'new statement text is prepared' means that old statement is dropped (next new one may be created when needed). And certainly server cleans up during statement drop, including batch data structures. Alex. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Is batch release necessary in the wire protocol?
On 7/26/22 14:39, Mark Rotteveel wrote: On 16-07-2022 10:54, Mark Rotteveel wrote: Is sending op_batch_rls necessary in the wire protocol, or is closing the statement itself sufficient? Similar question, is a batch automatically released when unpreparing the statement, or preparing a new statement text? Yes, it's released. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Is batch release necessary in the wire protocol?
On 7/16/22 11:54, Mark Rotteveel wrote: Is sending op_batch_rls necessary in the wire protocol, or is closing the statement itself sufficient? Closing the statement should be sufficient Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Operating system call pthread_mutex_lock failed. Error code 22 #Fb 4.0.1 #Centos 7
On 6/3/22 14:58, Lucas Schatz wrote: Hello Just do clarify, 600-900 seconds or ms? I think makes sense to be ms, anyway I'll put 30 seconds just to be sure, because almost every second the DB gets attached As soon as I have news I'll post here (hope not), at most probably next month I'll give a feedback Thanks I've talked about 600-900 seconds (i.e. 10-15 min) alter database set linger to 600; But if it's attached almost every second 30 vs 600 makes no real difference :) Next, with that attach/detach rate linger should have some erformance effect. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Operating system call pthread_mutex_lock failed. Error code 22 #Fb 4.0.1 #Centos 7
On 5/27/22 23:06, Lucas Schatz wrote: Installed, now wait for the error, probably only next week With that dump I've more or less understood where bug happens. But fixing that may be not quick. I'm almost sure that use of big enough linger (600 or 900 or even bigger) should make this bug happen very rare. Try that please - in the best case segfaults will be gone. A. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
On 6/2/22 14:45, Adriano dos Santos Fernandes wrote: On 02/06/2022 08:08, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 02.06.2022 13:01: It is "ALTER TABLE" which must be executed on replica. Otherwise you'll have problem with ALTER TABLE ADD FIELD. Is it impossible to analyze clauses of statement? Theoretically - yes. On practice it would require full Firebird SQL parser which is hard. Simpler would be to downgrade the "missing tablespace" error to a warning if the database is in replica mode or SQL is performed by Applier. Commands are replicated in text and executed in replica. The replica execution could ignore things when are running a replicated command. I see no difficulty there. Yes. Moreover - if ALTER TABLE has more clauses (like ADD FIELD) in addition to tablespaces related, only that tablespaces related clauses to be skipped. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
On 6/2/22 13:03, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 02.06.2022 9:31: ALTER TABLE SET TABLESPACE is tablespaces-related DDL operator - i.e. it should not be executed on replica. Why conflict? It is "ALTER TABLE" which must be executed on replica. Otherwise you'll have problem with ALTER TABLE ADD FIELD. Is it impossible to analyze clauses of statement? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Tablespaces proposal
On 6/2/22 10:14, Ilya Eremin wrote: Hello! The last tablespaces proposal says: > Tablespaces are excluded from replication because it's physical layout > of a database and a replica may have another one. which means that tablespaces-related DDL operators won't be executed on a replica. But some conflicts may happen, for example, when a table is being moved to another tablespace via ALTER TABLE SET TABLESPACE. ALTER TABLE SET TABLESPACE is tablespaces-related DDL operator - i.e. it should not be executed on replica. Why conflict? If the tablespace doesn't exist in a replica, the statement will fail. The tablespace should be created in a replica before the statement is executed. It is possible to do it in another way: replicate tablespaces-related DDL to have the same set of tablespaces in a replica. The main problem here is the path to a tablespace file. It still can be absolute (will work if the path is also valid for a replica) or relative. It might be helpful to use directory aliases which can be set in some configuration file. And this is a downside - need to edit the configuration file on a replica(s). I do not think that such need is too big trouble. When one wants to combine replication and tablespaces that does require some efforts, including configuration. Which way is better for default behaviour? Would be good to hear opinions on this. Ideally use of tablespaces in replication should be configurable. One wants to quickly setup replica and forget about configuration issues, other wants to have exact copy including tablespaces. Someone else may use on replica another tablespaces set in order to optimize performance in different way from master database. Why force everyone to go same way? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] event_t::event_count
On 5/30/22 14:37, Adriano dos Santos Fernandes wrote:
Hi!
Isn't this thread unsafe?
struct event_t
{
SLONG event_count;
int pid;
pthread_mutex_t event_mutex[1];
pthread_cond_t event_cond[1];
};
int SharedMemoryBase::eventPost(event_t* event)
{
...
++event->event_count;
...
}
Should not event_count use a std::atomic?
Or is SharedMemoryBase::eventPost expected to not be thread safe to be
called concurrently?
It's always called with shared memory mutex taken, i.e. no concurrency
problems.
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Operating system call pthread_mutex_lock failed. Error code 22 #Fb 4.0.1 #Centos 7
On 5/27/22 04:38, Lucas Schatz wrote: Hello, I moved the database to another server with Fb4.0.0 for some days, and "fortunately" today crashed too So, if you can send me the tar.gz I'll install with no problem http://web.firebirdsql.org/download/rabbits/alex/MutexError22/ Any additional risk of losing data? Additional - no. Moreover - for a database being attached in the moment of a crash absolutely no risks: it's file is just opened at that time. Certainly other databases accessed by server at this time may be affected by a crash - but possibility of data loss is very low. For sure it will not be higher compared with std build. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Operating system call pthread_mutex_lock failed. Error code 22 #Fb 4.0.1 #Centos 7
On 5/25/22 17:43, Lucas Schatz wrote: Sorry, just to clarify: server Fb4.0.1, remote client Fb 3.0+PHP7.3 Anyway I'll send you the links as soon as possible. I've made a close look at that dump. Unfortunately I could not find something very useful in it. I.e. it's more or less clear at what step happens 'error 22' and why segfault later details remin in dark - FB tried too much to recover after an error. If I prepare special build which will make core dump immediatrly after mutex error can you run it? I.e. server will fail a bit earlier (1-2 sec real time) but it anyway dies currently :-( Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Operating system call pthread_mutex_lock failed. Error code 22 #Fb 4.0.1 #Centos 7
On 5/25/22 17:35, Lucas Schatz wrote: I got 4 coredumps, can I send PM to you the link? Please do - but compress them separately. Also, I think we spotted the problem's source, looks like it's a PHP related issue Looking at the stack-trace, it looks like something related to a web server with PHP7.3 FPM and Fb V3.0.4.33054. Attached stack trace is 4.0.1. And I doubt PHP can directly affect SS, running in separate process. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Operating system call pthread_mutex_lock failed. Error code 22 #Fb 4.0.1 #Centos 7
On 5/25/22 17:23, Lucas Schatz wrote: 17gb +- Can you compress it (xz is preferred for obv.reasons) and put to some downloadable place? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Operating system call pthread_mutex_lock failed. Error code 22 #Fb 4.0.1 #Centos 7
On 5/23/22 16:07, Lucas Schatz wrote: The stack-trace is attached, as soon I get more stack-traces I'll send here too How big is core dump? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] gbak output not flushing?
On 5/17/22 10:41, Fabiano Bonin wrote: Firebird version 3.0.9 64bits on Debian When I use gbak with "nohup" in Debian, it seems gbak's output to nohup.out is written in chunks, nohup gbak -b -v -g database database.fbk & It shows, for instance: gbak:52 records written gbak:53 re And after many seconds or minutes, it writes another chunk to the log. This behavior appeared in Firebird version 3, it worked as expected in Firebird version 2.5. I never had a problem with this behavior, since it was just a matter of waiting for the end of the process, but this week gbak raised an error in a stored procedure when restoring a database that was not saved to the log file, probably because the restore process was interrupted and the log was not flushed. When the output is to the console, it works as expected. fixed https://github.com/FirebirdSQL/firebird/issues/7184 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Refresh roles
On 5/11/22 19:40, Adriano dos Santos Fernandes wrote: On 11/05/2022 11:55, Roman Simakov wrote: Can you make up or describe a potential case of the situation? Undoubtedly the feature would be useful but we've lived without it for all the time. In the profiler, I will create a role, the tables and views in another connection. I'll grant privileges to the role and default grant the role to public (currently not working due to #7178, so I'm testing with another user instead of public). But the user connection is already made and is going to use the profiler. Now I need to see what is its current_role, then "set role plg$profiler" (or any other role different than current), then "set role old_current_role" to refresh the current roles. If a role is revoked from a user, it will affect the next attachment only, won't it? I think this can be considered as a not well defined behavior and changeable, but anyway, if there is simple command to refresh the active roles instead of automatically, it's not going to be a problem. If select from some table is revoked from firebird user it will not affect running requests. If OS (at least *nix family) user opened file but later access to that file was revoked from him existing file descriptor remains valid. I.e. lets not try to make all and any access rights changes propagate automatically. On the other hand I see no problems if we have dedicated command peforming desired action. Suggest syntax: SET ROLE without parameters. (but not insist on it) Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_spb_version Call Without Global User?
On 5/4/22 10:29, Mark Rotteveel wrote: On 03-05-2022 18:56, Dimitry Sibiryakov wrote: Scott Morgan via Firebird-devel wrote 03.05.2022 18:29: What can you do to get the server's info if you only have per-DB user accounts? Use isc_dpb_expected_db with your database name to inform Firebird which setting from databases.conf should be used during service attach (including security database). I assume you mean isc_spb_expected_db, right? definitely Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] WITH CALLER PRIVILEGES propagation
On 22.04.2022 18:07, Jiří Činčura wrote: Yes. Fails. Do you want me to create script for testing? Can you please check your script on some old version? Like 2.5. May be I miss something. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] WITH CALLER PRIVILEGES propagation
On 22.04.2022 08:20, Jiří Činčura wrote: Without it, it obviously fail. Because the "limited" user does not have permissions. That's what I'm trying go around thru the permissions of calling object. Fails w/o execute statement ? Just using select from procedure instead? Very strange - this always worked with caller privileges. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] WITH CALLER PRIVILEGES propagation
On 21.04.2022 13:38, Jiří Činčura wrote: I guess the "caller privileges" is propagated only into `pkg_test_limited.test` when calling, but not further into `t_test`. Can I somewhat make it work/propagate? Or did I misunderstood the feature? "with caller privileges" was designed to make privileges, granted to calling object, be used when processing dynamic SQL statement. SO first of all I suggest to perform all this chain of calls w/o execute statement and see does it make any difference. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Fw: [firebird-support] ARM64 Support
On 19.04.2022 20:06, Hugo Larson via Firebird-devel wrote: Hi, I want to build FB4 for Linux (Ubuntu) arm64 and wonder if there is documentation with instructions? Nothing special. I've tried that build a year or so ago, it builds w/o any special measures. ./configure make Certainly required dev (important for ubuntu) packages should be installed. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Status of 3.0.10
On 4/13/22 21:57, Paul Reeves wrote: For now, all I can suggest is that we remove the option to set legacy auth support in the Fb3 installer. The best solution on my mind! Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Status of 3.0.10
On 4/13/22 17:11, Pavel Cisar wrote: Hi, because 3.0.9 has significant regression (#7137 - we've got multiple reports from large affected and unhappy users), I'd like ask what are the prospect to release v3.0.10 soon. BTW, it would also address contracted collation performance issues that we hoped would be addressed by 3.0.9, but as 3.0.9 release was rushed ahead of schedule, this was deferred to 3.0.10. We have one test failing on linux - bugs/core_1885.fbt. Create collation should fail with some option, but it succeeds. uppose due to upgrade of ICU on linux test server. In other aspects I suppose we are ready for release. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Handling exception from external plugins
On 4/12/22 11:53, Roman Simakov wrote: Alex, Is it documented somewhere? Exactly in this form - not. I.e. a fact that exceptions are caught when throwing from OO API functions is documented (in Using_OO_API) and special position of FbException is documented too, but need of first Status parameter to make all that work is not emphasized. To be precise - _any_ exception in _any_ OO API function will be caught, but it will not be delivered in original way to caller. The humor is that FB_PLUGIN_ENTRY_POINT (except missing Status parameter) is not OO API call - it's just plain C function which is needed to 'boot' OO API in plugin, and certainly it should not throw over module boundaries. пн, 11 апр. 2022 г. в 18:45, Alex Peshkoff via Firebird-devel : On 4/11/22 17:56, Dimitry Sibiryakov wrote: Vasiliy Yashkov wrote 11.04.2022 16:52: What is the right way to throw an exception? Plugins (as any other DLL) is not supposed to throw exceptions ever. Yes, throwing directly (like in your code) is impossible. But there is a way to return an exception information from plugin using Status interface, which is passed as first parameter in that case. Pay attention - FB_PLUGIN_ENTRY_POINT has no such parameter and therefore is not expected to return exception information. (there are more reasons why exceptions not to be used in that call but let me not dive too deep here) But if some plugin entry has first Status parameter you can throw FbException in it, it will be caught by thin API layer which actually invoked your plugin entry and delivered to the code which called plugin. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Handling exception from external plugins
On 4/11/22 17:56, Dimitry Sibiryakov wrote: Vasiliy Yashkov wrote 11.04.2022 16:52: What is the right way to throw an exception? Plugins (as any other DLL) is not supposed to throw exceptions ever. Yes, throwing directly (like in your code) is impossible. But there is a way to return an exception information from plugin using Status interface, which is passed as first parameter in that case. Pay attention - FB_PLUGIN_ENTRY_POINT has no such parameter and therefore is not expected to return exception information. (there are more reasons why exceptions not to be used in that call but let me not dive too deep here) But if some plugin entry has first Status parameter you can throw FbException in it, it will be caught by thin API layer which actually invoked your plugin entry and delivered to the code which called plugin. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Security vulnerability in zlib library
On 4/1/22 01:30, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 31.03.2022 16:08: The crash happen when a stream of definite data is tried to be compressed. IMHO, it is hard (if possible at all) to purposefully construct such stream *from* server to crash or exploit it. How long should it be? Can it be put into blob? Yes, but according to the bug description it also requires usage of Z_FIXED option which Firebird doesn't. Have a look at this - bug is already reproduced with default strategy: https://seclists.org/oss-sec/2022/q1/201 Luckily other parameters (like memlevel) are not default and such values of them are not used by firebird but you see: the range of conditions where bug can be reproduced spreads. I.e. it's definitely better to upgrade. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Security vulnerability in zlib library
On 3/31/22 16:39, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 31.03.2022 15:21: Note that the crash happen on compression so it doesn't affect Firebird security. Did not catch why - we use zlib compression on the wire (since fb3) and in gbak (since fb4). Both cases are not default but anyway not good. The crash happen when a stream of definite data is tried to be compressed. IMHO, it is hard (if possible at all) to purposefully construct such stream *from* server to crash or exploit it. How long should it be? Can it be put into blob? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Security vulnerability in zlib library
On 3/31/22 16:13, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 31.03.2022 15:05: On 3/31/22 11:11, Mark Rotteveel wrote: A security vulnerability was found in zlib: https://nakedsecurity.sophos.com/2022/03/29/zlib-data-compressor-fixes-17-year-old-security-bug-patch-errr-now/ Will we include an updated version in the next release? On linux that's not our problem - we always use system libz.so. On windows I think yes, we should upgrade version. Note that the crash happen on compression so it doesn't affect Firebird security. Did not catch why - we use zlib compression on the wire (since fb3) and in gbak (since fb4). Both cases are not default but anyway not good. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Security vulnerability in zlib library
On 3/31/22 11:11, Mark Rotteveel wrote: A security vulnerability was found in zlib: https://nakedsecurity.sophos.com/2022/03/29/zlib-data-compressor-fixes-17-year-old-security-bug-patch-errr-now/ Will we include an updated version in the next release? On linux that's not our problem - we always use system libz.so. On windows I think yes, we should upgrade version. Can people just drop in a replacement? Yes. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] IBatch::TAG_BUFFER_BYTES_SIZE and p_batch_data
On 3/17/22 15:30, Jiří Činčura wrote: Hi *, Is the IBatch::TAG_BUFFER_BYTES_SIZE directly related to how big p_batch_data (in total) can be? Yes. BTW - setting this tag to 0 is treated as use maximum size, currently 256Mb. Or is it something different. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Local tables
On 3/15/22 20:41, Dmitry Yemanov wrote: 15.03.2022 20:31, Alex Peshkoff via Firebird-devel wrote: For me, "created" LTT is similar to GTT (i.e. stored in the schema) but with data isolated per request (per PSQL routine). I'd consider about CREATE'd LTT as attachment-private object. I see no need to store its definition at the persistent schema. If multiple procedures process the same layout of temporary data, IMHO it's handier to create such LTT definition once (as persistent) rather than declare the same LTT in the every procedure (or attachment). Same for almost all objects declared in procedure that are more complex than INT. For example cursor definition may be rather complex and also usable in multiple procedures. True. But I don't remember CREATE CURSOR in the SQL spec ;-) Anyway, IMO "declared" LTTs are much more useful than "created" LTTs, so the latter ones may be deferred. Or we may live without them at all ;-) Taking into an account mentioned in this thread restriction: a) PSQL routines can't see CREATE'd LTT's I fully agree with you here. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Local tables
On 3/15/22 20:27, Dmitry Yemanov wrote: 15.03.2022 20:17, Vlad Khorsun wrote: For me, "created" LTT is similar to GTT (i.e. stored in the schema) but with data isolated per request (per PSQL routine). I'd consider about CREATE'd LTT as attachment-private object. I see no need to store its definition at the persistent schema. If multiple procedures process the same layout of temporary data, IMHO it's handier to create such LTT definition once (as persistent) rather than declare the same LTT in the every procedure (or attachment). Same for almost all objects declared in procedure that are more complex than INT. For example cursor definition may be rather complex and also usable in multiple procedures. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Local tables
On 3/15/22 17:14, Adriano dos Santos Fernandes wrote: Hi! As part of "Support multiple rows for DML RETURNING (#6815)" feature, BLR verbs for "local table" were created. Local tables (LT) as defined there works outside transaction control. For #6815 this does not matter, but I want to add LT feature that will use these verbs. LTs are defined inside PSQL routines and its data is separated per routine invocation. For example a recursive routine will have different data in its LTs per each invocation, like local variables. Very interesting - but other name for them should be used. It's too easy to loose difference between LT & LTT that are absolutely different things. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Compiled statement cache
On 2/28/22 20:30, Adriano dos Santos Fernandes wrote: On 28/02/2022 13:18, Alex Peshkoff via Firebird-devel wrote: I suppose it's with mentoned 2Mb cache. Yes. But it's the same even with 100K as the test uses very few and small statements. Taking into an account that bigmost was 21k there are really few statements :) Did you try with smaller cache size? One more question. Suppose some statement remains in a cache for very long time cause it's reused again and again. That's fine - but with DB grow optimal plan can change. Is it solved somehow? Suppose in first implementation not, Correct. And it's a problem also now when application holds prepared statement manually, but that probably is not going to be changed. Yes, it's out of scope. but are there plans to solve it? I think there are two possible ways: - Timeout of cached statement (counting from its first appearance in cache, not last usage) Yes, re-preparing all statements once per relatively big timeout should not cause visible performance problem. - When engine detects a condition which could change a plan, it may ask cache for invalidation. At the first glance that's changing index active/inactive and in the future bulk insert. That's first things that come to my mind. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Statement::verifyAccess of internal requests
On 3/1/22 21:33, Adriano dos Santos Fernandes wrote:
Hi!
Do we have any good reason to make this method do for internal requests
everything it does for user's statement?
Or could we start it with this?
void Statement::verifyAccess(thread_db* tdbb)
{
if (flags & FLAG_INTERNAL)
return;
...
}
I do not see usage of internal requests that contradict your suggestion
in current codebase.
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Snapshot buuilds
On 3/1/22 19:19, Karol Bieniaszewski wrote: Hi Snapshots are not avilable http://web.firebirdsql.org/download/snapshot_builds/win/3.0 and whole page http://web.firebirdsql.org/ I've notified Sean privately about it. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_dpb_lc_ctype and isc_dpb_set_db_charset and isc_dpb_utf8_filename
On 2/28/22 21:39, Jiří Činčura wrote: Thanks. I suppose not only TPB, but also buffers for batch, events, etc., right? Yes. Basically only DPB and SPB are "UTF8-ready". If one wants everyting to be utf8 why not have appropriate attachment? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Knowing whether fbclient supports isc_dpb_utf8_filename
On 3/1/22 13:45, Jiří Činčura wrote: Hi *, Can somebody think of a way to detect whether fbclient supports isc_dpb_utf8_filename? For example when calling isc_create_database in .NET provider with "Embedded" I need to know whether I can use UTF8 in database path. On Windows I can check version of the DLL and when version >= 2.5 I can assume I'm safe (What happens when I connect using 2.5 to older database?). But on Linux, etc. the binary has no such version info in header. Bummer. from ibase.h: // /* Client information functions */ // void ISC_EXPORT isc_get_client_version ( ISC_SCHAR *); int ISC_EXPORT isc_get_client_major_version (); int ISC_EXPORT isc_get_client_minor_version (); Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Compiled statement cache
On 2/28/22 16:54, Adriano dos Santos Fernandes wrote: On 26/02/2022 22:05, Adriano dos Santos Fernandes wrote: I will report back some numbers (memory, prepare times). Here is a performance test I did: https://pastebin.com/3UnB5BNU It's relative simple (not much indices) and common metadata. It's tested as whole (create database, create metadata, populate, run queries and get results, drop database). The last select has a statement size of 21.5KB. For reference, "select 1 from rdb$database" has size 6KB. Test timings: - Without statement cache: 19800ms - With statement cache: 16700ms More than 10% is great! I suppose it's with mentoned 2Mb cache. One more question. Suppose some statement remains in a cache for very long time cause it's reused again and again. That's fine - but with DB grow optimal plan can change. Is it solved somehow? Suppose in first implementation not, but are there plans to solve it? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_dpb_lc_ctype and isc_dpb_set_db_charset and isc_dpb_utf8_filename
On 2/28/22 13:38, Jiří Činčura wrote: Does the isc_dpb_utf8_filename imply also UTF8 strings in TPB (i.e. for table names used with isc_tpb_lock_write, etc.)? They are expected in attachment charset. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_dpb_lc_ctype and isc_dpb_set_db_charset and isc_dpb_utf8_filename
On 2/28/22 13:40, Jiří Činčura wrote: Also, does the isc_dpb_utf8_filename imply UTF8 for database name in i.e. op_attach (not DPB, the name that's before DPB)? Yes. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_arg_string in status vector
On 2/28/22 13:10, Jiří Činčura wrote: Hi *, If I get pointer to a string in isc_arg_string in status vector the encoding is UTF8 if isc_dpb_utf8_filename was specified and ANSI/system charset when isc_dpb_utf8_filename was not specified? Data in status vector should be according to attachment charset and not affected by isc_dpb_utf8_filename. If that's not so then it should be a bug. ps.In some cases error may happen when trasliterating UTF8 line to attachment charset. In that case strig willl be returned is UTF. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Compiled statement cache
On 2/26/22 17:14, Adriano dos Santos Fernandes wrote: Hi! I do want to define default cache size. I'm thinking in 2M. Comments? As long as it's one_place_change constant - any value is OK for begining. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Build for WASM
On 2/24/22 03:45, Adriano dos Santos Fernandes wrote: On 23/02/2022 06:40, Jiří Činčura wrote: You mean, running in the browser? Yes. Or any other place where WASM is/will be supported. I had a (very) brief look at it once. Main difficulties should be: - Memory manager - no mmap - Shared libraries - File I/O - Lock manager All these points should have a workaround for a limited environment, but one must work on them - improving abstractions and much testing and fixes, I suppose. I believe it should be a long work. It would have a good usage: allow usage of Firebird directly in web pages, running in the client. That would be awesome for users wanting to try Firebird. That port promises to be much more problematic than Android one. Certainly, if someone voluteers for that job it will be good, but I doubt we should spend project resources on it. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2/15/22 18:20, Vlad Khorsun wrote: I'd vote to remove idx_numeric2 in favour of idx_decimal and look how to improve Decimal128::makeIndexKey() performance. For example, it stores every 3 decimal digits into 10 bits using relatively complex (computationally) algorithm with shifts and multiplications. Storing every digit in 4 bits will use 2 bits per 3 digits more but should save come CPU circles, I believe. Yes, it does help - but not too much. select dat from biTest order by dat; (I reference sample from #7133) becomes 1.1% faster with less dense encoding. Pay attention - engine does a lot of other job, i.e. speedup of this particular place is certainly much more. Certainly we can use that performance increase for new index type. But I doubt it's worth breaking existing DECFLOAT indices. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2/16/22 13:49, Alex Peshkoff via Firebird-devel wrote: On 2/16/22 12:54, Dmitry Yemanov wrote: 16.02.2022 12:19, Mark Rotteveel wrote: However, that was not my main point. My main point was that it sounds like an index format that was created for supporting DECFLOAT(34), and that it is not suitable for the full range of INT128 and NUMERIC/DECIMAL backed by INT128 (for the same reasons the DOUBLE PRECISION format is not suitable for BIGINT and NUMERIC/DECIMAL backed by BIGINT). It looks so. Unless we miss something (Alex?), This is possible way to fix a bug: https://github.com/FirebirdSQL/firebird/commit/7dd832f32e9669bcb3007dc675b3ee7cca6f6b7d New type of indexes is added and it works fine. But I wonder - what to do with existing databases? Rebuilding affected indexes is enough - but such database becomes unusable for previous versions of engine. Normally we should increase ODS minor. But last time when it was done in FB3 it gave serious negative feedback. Next, upgrading ODS requires gbak/restore cycle, which is problematic with big database. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2/16/22 12:54, Dmitry Yemanov wrote: 16.02.2022 12:19, Mark Rotteveel wrote: However, that was not my main point. My main point was that it sounds like an index format that was created for supporting DECFLOAT(34), and that it is not suitable for the full range of INT128 and NUMERIC/DECIMAL backed by INT128 (for the same reasons the DOUBLE PRECISION format is not suitable for BIGINT and NUMERIC/DECIMAL backed by BIGINT). It looks so. Unless we miss something (Alex?), That was my fault. Initially it was decided to represent big numerics with decfloat. But that works very slow, and later it was replaced with BIGINT. But about index key formay I've just forgotten. perhaps we need to add a runtime check that rejects key creation for INT128 values longer than 34 decimal digits. And find a better solution in some later release. Normalize coefficient appropriately adjusting exponent like it's done for BCDs in DECFLOAT case, but here that can be done in binary form. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2/16/22 13:34, Dmitry Yemanov wrote: 16.02.2022 13:28, Dmitry Yemanov wrote: It looks so. Unless we miss something (Alex?), perhaps we need to add a runtime check that rejects key creation for INT128 values longer than 34 decimal digits. Thinking twice, an overflow error should already been raised when a longish INT128 is converted into DECFLOAT(34) before composing the index key. No, it does not happen, that int128 is rounded. So no problems at all, except suboptimal index scan (possibly false matches) for longish values? At least ORDER BY via index should be broken for them, I suppose. Yes. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2/16/22 13:28, Dmitry Yemanov wrote: 16.02.2022 13:24, Alex Peshkoff via Firebird-devel wrote: It looks so. Unless we miss something (Alex?), perhaps we need to add a runtime check that rejects key creation for INT128 values longer than 34 decimal digits. Thinking twice, an overflow error should already been raised when a longish INT128 is converted into DECFLOAT(34) before composing the index key. No, it does not happen, that int128 is rounded. So no problems at all, except suboptimal index scan (possibly false matches) for longish values? Not exactly. When index is used in order by it may produce wrong sorts. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2/16/22 12:59, Dmitry Yemanov wrote: 16.02.2022 12:54, Dmitry Yemanov wrote: It looks so. Unless we miss something (Alex?), perhaps we need to add a runtime check that rejects key creation for INT128 values longer than 34 decimal digits. Thinking twice, an overflow error should already been raised when a longish INT128 is converted into DECFLOAT(34) before composing the index key. No, it does not happen, that int128 is rounded. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2/16/22 12:45, Alex Peshkoff via Firebird-devel wrote: To be precise - 9 digits into ULONG. No shifts (BTW what a problem with them, CPUs have appropriate fast support since first pentium or even more). There is one division of small integer (range from 0 to 33). It can be easily replaced with table lookup but I'm not sure is it worth doing or not - small numbers division if fast enough. Sorry - that's all about sort, not index key. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2/15/22 18:20, Vlad Khorsun wrote: I'd vote to remove idx_numeric2 in favour of idx_decimal Appears to be good idea. and look how to improve Decimal128::makeIndexKey() performance. For example, it stores every 3 decimal digits into 10 bits using relatively complex (computationally) algorithm with shifts and multiplications. To be precise - 9 digits into ULONG. No shifts (BTW what a problem with them, CPUs have appropriate fast support since first pentium or even more). There is one division of small integer (range from 0 to 33). It can be easily replaced with table lookup but I'm not sure is it worth doing or not - small numbers division if fast enough. For Decimal34 use of packed BCD will cause growth of index key (+1 DWORD). I doubt that can make overall result better. May be we can move 2 digits in the end of exponent's DWORD, in that case we do not loose in key size. But is that faster or slower compared with current should be checked. For Decimal16 I see no such problems - packed BCD can be used w/o problems. Storing every digit in 4 bits will use 2 bits per 3 digits more but should save come CPU circles, I believe. There is also extracting of BCD from internal decfloat representation. Also not too fast process. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] JRD_reschedule
On 2/14/22 15:33, Adriano dos Santos Fernandes wrote: Hi! What is the importance of JRD_reschedule today? More specifically speaking, verifying contention and "checking out"? - AST (yes, 'A'-async is wrong tday, but I use traditional term) processing - cancel, shutdown, etc. processing Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Compiled statement cache
On 2/10/22 19:21, Dimitry Sibiryakov wrote: Dmitry Yemanov wrote 10.02.2022 17:13: Cannot the UTF8-translated SQL text (which is currently inside Statement::sqlText) be the key, to avoid dependency on the charset? I don't think that semantically different statements may have the same UTF8 representation. However, it would make sense to have a single cached statement for the same statement executed from different client charsets. Only if such translation is made right. Remember charset introducers. But apparently to transform the query before using it as a cache key is a right idea. Two queries different only by some whitespaces, comments or case (unless in literals) should not miss the cache. I think that for the first time it\s enough to implement simple most cache. IMHO more precise detection (like different whitespaces, etc.) of same statements hardly makes big dufference in most cases. If we see solid effect from new cache - certainloy, it will be worth to make it better prepare search key. I.e. let's start from something simple and see does it make sense. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Compiled statement cache
On 2/10/22 15:57, Adriano dos Santos Fernandes wrote: On 10/02/2022 06:59, Alex Peshkoff via Firebird-devel wrote: If we need to take roles into an account - only for attachment with same USER. Even without shared cache, user can change its roles with SET ROLES and new prepared statements should work as before even when they were previously cached with different roles. Some of them should not wotk at all - provided access to some objects in them was granted to previous role only. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Compiled statement cache
On 2/10/22 12:44, Vlad Khorsun wrote: I've peek these: - statement text - clientDialect - isInternalRequest Why it is important ? Do we have internal DSQL requests now ? Yes we have - but not too much. Is it makes sense to have two caches - for internal and for user statements ? - current client charset (as external engines may change it) - active roles If\when shared metadata will be implemented - will it be possible to use cached compiled statement created in one attachment to use by another attachment ? If we need to take roles into an account - only for attachment with same USER. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Compiled statement cache
On 2/8/22 16:36, Adriano dos Santos Fernandes wrote: Hi! I have refactored DSQL statements/requests (continued worked of many time ago) to separate shared (statement) and specific (request) parts. It seems this ground work for compiled statement cache is complete and I have even did a (very very) initial version of a compiled statement cache working. Now I think it's better to discuss its semantics. First what should be the statement key in the cache? I've peek these: - statement text - clientDialect - isInternalRequest - current client charset (as external engines may change it) - active roles Do you see any thing more? Even if you missed something adding an item later should not be too awful problem. Then there is when statements should go out of cache? I see three approaches: - 1. Timeout after its put in the cache, updated when it is get from it - 2. LRU based on memory consumption and max cache size - 3. Both 1. and 2. I think we can start with 1. I like 2 better - we have much less chances to go OOM with it. Should it be enabled by default? I think yes. And it may have per database configuration of the timeout value. Or cache size value... Cache invalidation: Cached (and unused) statements should not lock objects preventing DDL changes. Assuming that cached statements never lock anything exclusively (there should be bug if that happens), then when someone tries to lock something exclusively it should ask caches to release the statements having that objects. Or, I think it's completely ok for an initial implementation, any try to exclusively lock an object may ask caches for complete invalidation. Agreed - that's ok, for first time definitely. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Slower database creation on Embedded in 4.0 compared to 2.5
On 2/8/22 12:23, Jiří Činčura wrote: First of all DPB, but conf files are also interesting. DPB: isc_dpb_dummy_packet_interval => 120 dummy packet interval makes no sense for embedded do not think that affects result - just strange isc_dpb_sql_dialect => 3 isc_dpb_user_name isc_dpb_set_db_charset isc_dpb_force_write => 0 isc_dpb_overwrite => 1 isc_dpb_page_size => 16k Config file is whatever is default as there's no explicit config file. Retried your test, using isql in this way (linux x64): time (echo "CREATE DATABASE '/mnt/db/xxx.gdb' PAGESIZE 16384;" | ./isql) /mnt/db is on sata SSD Resuting time is 0.086s for fb4 & 0.038s for 2.5. I.e. 4 is twice slower. But please compare database sizes: fb4: 3162112 fb2.5: 2359296 I.e. approximately half of a difference is due to more complex (and therefore bigger) database. The rest may depend on a lot of things - up to other ICU version or additional time for separate provider loading. Do not see it as critical. I compared what I get with your result: 0.042s (on average on my machine) on FB 2.5, but 0.162s on FB4 Almost 4 times slower is definitely hard to explain with 1.35 times bigger DB size, no idea why does it run so slow for you. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Slower database creation on Embedded in 4.0 compared to 2.5
On 2/8/22 11:27, Jiří Činčura wrote: Are you using default DB options? You mean DPB or settings in firebird.conf? First of all DPB, but conf files are also interesting. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Slower database creation on Embedded in 4.0 compared to 2.5
On 2/7/22 13:52, Jiří Činčura wrote: Hi *, For some tests I'm creating and dropping a database for each test. Doing just that, it takes around 0.042s (on average on my machine) on FB 2.5, but 0.162s on FB4. Is this expected? Are you using default DB options? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parameter block and database/service name encoding
On 2/4/22 16:44, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 04.02.2022 14:37: What to do with values of tags that are unknown (other version or provider-specific)? I see no way to tell on destination side if they arrived as is or were transliterated. Take a look at checkTag() functions in IntlParametersBlock.cpp. It's clear that items unknown to client library remain not transliterated. And thus two problems: 1. Client library is required to match server version while original design allowed a variety of back- and forward compatibility. As long as one does not use new string-items in DPB with non-ascii data, known to server but not known to client library, it's OK. In described bad case if one does not want to upgrade fbclient UTF8 should be used. 2. Provider-defined string items are in troubles. Yep, it's really a problem. They will be delivered to provider in client encoding. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parameter block and database/service name encoding
On 2/4/22 15:17, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 04.02.2022 12:59: I suppose you suggest to fallback to existing way when there is no isc_dpb_lc_ctype in DPB? I suggest nothing but wonder what to do if there is no isc_dpb_utf8 tag in DPB but isc_dpb_lc_ctype is. Is it right to still consider values to be in ANSI encoding? You know yourself that client may push any sheet in DPB. For client use of default ANSI codepage is good guess, not more. What to do with values of tags that are unknown (other version or provider-specific)? I see no way to tell on destination side if they arrived as is or were transliterated. Take a look at checkTag() functions in IntlParametersBlock.cpp. It's clear that items unknown to client library remain not transliterated. Host name is not sent over the wire. I doubt names resolver in system using (for example) cp1251 will accept host name in utf8. It is actually contrary: https://www.sql.ru/forum/actualthread.aspx?tid=1341790 According to documentation gethostbyname() & Co should work with current locale. But not sure what does it mean for windows. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parameter block and database/service name encoding
On 2/4/22 13:41, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 04.02.2022 9:53: Remember me please why isc_lc_ctype isn't used to determine encoding of database name and DPB values? Due to limitations of WideCharToMultiByte API. (with posix iconv this cabe easily done) Do you mean that client would have to have a table matching Firebird charset names and Windows codepages? Sorry, windows-specific part was implemented not by me. As gar as I know there were more serious problems. Table will certainly be needed, but IMO it's not too big deal. I suppose you suggest to fallback to existing way when there is no isc_dpb_lc_ctype in DPB? BTW, service name during service attach isn't converted into UTF-8 at all. It was always ascii :D Except host name part. Host name is not sent over the wire. I doubt names resolver in system using (for example) cp1251 will accept host name in utf8. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parameter block and database/service name encoding
On 2/3/22 18:39, Dimitry Sibiryakov wrote: Hello All. Remember me please why isc_lc_ctype isn't used to determine encoding of database name and DPB values? Due to limitations of WideCharToMultiByte API. (with posix iconv this cabe easily done) BTW, service name during service attach isn't converted into UTF-8 at all. It was always ascii :D Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Packaging FirebirdSQL for Conan
On 2/2/22 17:04, Horváth V. wrote: Okay, so from this it appears that fbclient 3.x has basically no required external dependencies? I'm not sure how tommath relates to threading and atomics, it appears to be a MPI library. Yes. tommath is the only requirement not in std libraries list. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Packaging FirebirdSQL for Conan
On 2/2/22 15:46, Adriano dos Santos Fernandes wrote: On 02/02/2022 09:21, Horváth V. wrote: Hello list! I have the intention of packaging FirebirdSQL 3.x for the Conan cross-platform package manager, so that I can start working on migrating PHP to being a CMake and Conan managed project. Details about that here: https://github.com/conan-io/conan-center-index/issues/8618 I was redirected to this mailing list regarding questions about FirebirdSQL: https://github.com/asfernandes/cloop/issues/8#issuecomment-1027859114 So, the thing I'm curious about is: What are the normal (runtime) requirements of FirebirdSQL? It turns out that cloop is used for developer workflows only and it's a build requirement, which I assume to be true also for btyacc yes - both not needed for runtime , both of which I already have pending PRs for in Conan Center Index (Conan's default repository). https://www.firebirdsql.org/en/building-the-code/ and the linked pages don't seem to have a list of categorized dependencies that would tell me what exactly I need to just build and distribute FirebirdSQL. It depends you are talking about client library only or also server and command line tools. I guess you are talking about client library only, so it must be zlib. zlib is not absolutely required in runtime for fb3. It's needed to support data compression over the wire, which is controlled by client settings and in 99% of usage cases not needed. If not mention absolutely standard thing like C/C++ libraries with threading (and may be atomic - depends on used compiler version) support there is tommath library, which may be present or missing in FB binary package depeding upon ./configure parameters - no idea what does cmake do for it. When used with package manager tommath is definitely external dependency. If embedded access is needed (embedded means engine is running inside calling process) that list will certainly grow. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] obj_database value in FB3 and FB4
On 1/26/22 17:48, Roman Simakov wrote: I hope it can be handled while restoring. I have no good idea how to deal with differences between 3 and 4. It's possible to make a gap in 5 and handle it while restoring as well. Let's make a gap only when have real need to change it. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] MacOS problem with dlclose
On 1/25/22 11:11, Paul Beach wrote: Should this be maintained in FB 3? No, Min version now for 3.0.8 is 10.15 Ahh - OK, have not known when suggested a fix a few minutes ago. This makes things just simpler. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
