Re: [Firebird-devel] Build firebird 2.5 problem
Hi Hugo, You sent this message to the old list at SourceForge. Please post to the firebird-devel Google Group (https://groups.google.com/g/firebird-devel, or firebird-de...@googlegroups.com). Mark On 05-08-2023 13:58, Hugo Larson via Firebird-devel wrote: Hello, I'm stuck for a while with FB 2.5 and need to build it on aarch64 Linux and getting the error below. I know too little about building and am using docker https://github.com/jacobalberty/firebird-docker/blob/2.5-sc/Dockerfile <https://github.com/jacobalberty/firebird-docker/blob/2.5-sc/Dockerfile> Thanks, Hugo. 100 12.9M 100 12.9M 0 0 14.1M 0 --:--:-- --:--:-- --:--:-- 40.9M 66.85 checking whether make sets $(MAKE)... yes 66.86 checking build system type... builds/make.new/config/config.guess: unable to guess system type 66.96 66.96 This script, last modified 2005-12-23, has failed to recognize 66.96 the operating system you are using. It is advised that you 66.96 download the most up to date version of the config scripts from 66.96 66.96 http://savannah.gnu.org/cgi-bin/viewcvs/*checkout*/config/config/config.guess 66.96 and 66.96 http://savannah.gnu.org/cgi-bin/viewcvs/*checkout*/config/config/config.sub 66.96 66.96 If the version you run (builds/make.new/config/config.guess) is already up to date, please 66.96 send the following data and any information you think might be 66.96 pertinent to in order to provide the needed 66.96 information to handle your system. 66.96 66.96 config.guess timestamp = 2005-12-23 66.96 66.96 uname -m = aarch64 66.96 uname -r = 5.15.0-1039-oracle 66.96 uname -s = Linux 66.96 uname -v = #45-Ubuntu SMP Thu Jul 13 19:41:22 UTC 2023 66.96 66.96 /usr/bin/uname -p = aarch64 66.96 /bin/uname -X = 66.96 66.96 hostinfo = 66.96 /bin/universe = 66.96 /usr/bin/arch -k = 66.96 /bin/arch = aarch64 66.96 /usr/bin/oslevel = 66.96 /usr/convex/getsysinfo = 66.96 66.96 UNAME_MACHINE = aarch64 66.96 UNAME_RELEASE = 5.15.0-1039-oracle 66.96 UNAME_SYSTEM = Linux 66.96 UNAME_VERSION = #45-Ubuntu SMP Thu Jul 13 19:41:22 UTC 2023 66.96 configure: error: cannot guess build type; you must specify one Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] nullOffset on vacation ?
Hi Norbert, We're moving the list. Please subscribe to the firebird-devel Google Group and post your question there (once subscribed, you can use firebird-de...@googlegroups.com or post through the website). See also the message with subject "[Firebird-devel] ACTION REQUIRED: This list is moving to Google Groups, please resubscribe on the new location" for details. Mark On 25-09-2022 12:48, Norbert Saint Georges wrote: Hi, FB4 iso8859_1 With a varchar(32765) field, the API returns a nullOffset outside the buffer length and the next field gives me an offset identical to the nullOffset of the previous field. Did I miss something? -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ACTION REQUIRED: This list is moving to Google Groups, please resubscribe on the new location
On 25-09-2022 11:29, Mark Rotteveel wrote: On 25-09-2022 11:19, Dimitry Sibiryakov via Firebird-devel wrote: Mark Rotteveel wrote 25.09.2022 11:15: This list is moving to Google Groups, and we'd like you to move with it. "Join" button in your invitation sends to 404 page. Do you mean the invite I sent you directly from Google Groups? That is odd as that is generated by Google itself. Maybe replying to the message works as well? Otherwise, I suggest subscribing on https://groups.google.com/g/firebird-devel or by sending an email to firebird-devel+subscr...@googlegroups.com It looks like the email generated when inviting people through the admin interface contains links that only work when you're logged in to Google. If you're not logged in, it generates a 404 :( In any case, above alternatives should work. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ACTION REQUIRED: This list is moving to Google Groups, please resubscribe on the new location
On 25-09-2022 11:19, Dimitry Sibiryakov via Firebird-devel wrote: Mark Rotteveel wrote 25.09.2022 11:15: This list is moving to Google Groups, and we'd like you to move with it. "Join" button in your invitation sends to 404 page. Do you mean the invite I sent you directly from Google Groups? That is odd as that is generated by Google itself. Maybe replying to the message works as well? Otherwise, I suggest subscribing on https://groups.google.com/g/firebird-devel or by sending an email to firebird-devel+subscr...@googlegroups.com Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] ACTION REQUIRED: This list is moving to Google Groups, please resubscribe on the new location
Hello everyone, This list is moving to Google Groups, and we'd like you to move with it. To do this, subscribe at our new location: https://groups.google.com/g/firebird-devel, or by sending an email to firebird-devel+subscr...@googlegroups.com Afterwards, we suggest you unsubscribe from the old list. You can do this at https://lists.sourceforge.net/lists/options/firebird-devel, or be sending an email to firebird-devel-requ...@lists.sourceforge.net with subject: unsubscribe NOTE: After subscribing on Google Groups, messages may be moderated, so please be patient if your message doesn't show up immediately. We'll try to manually clear moderation status for known subscribers. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Test (please ignore)
Test (please ignore) -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 15-09-2022 15:19, Alex Peshkoff via Firebird-devel wrote: On 9/15/22 14:36, Adriano dos Santos Fernandes wrote: On 15/09/2022 08:18, Alex Peshkoff via Firebird-devel wrote: This is solved by install script - it creates symlink to /inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory (provided it's missing). If we decide to _always_ provide own dynamic libraries (logically it's like using static one but does not require duplicating code) this trick becomes not needed any more. Let me remind you that this thread started with me noticing an inconsistency in how we build things when explicitly building with --with-builtin-tommath compared to the - in my opinion - more sane behaviour of --with-builtin-tomcrypt (putting things in gen/buildroot/usr/local/firebird/lib/.tm vs in gen/buildroot/usr/local/firebird/lib). I was not asking about always using our own libraries, but at least doing things in a sane way when a build is performed explicitly requesting to use our own version of libraries. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 15-09-2022 13:13, Adriano dos Santos Fernandes wrote: On 15/09/2022 07:52, Mark Rotteveel wrote: On 15-09-2022 03:52, Adriano dos Santos Fernandes wrote: On 05/09/2022 08:13, Mark Rotteveel wrote: That is not really embedded IMHO. Firebird Embedded with Firebird 3.0 has become unwieldy and even harder to use than earlier versions, especially if you compare it to other embedded database systems. How it become harder? Basically, in Firebird 2.5 and earlier, you needed fbembed.dll / libfbembed.so and the intl folder, and you were done. Now you need a whole constellation of files. Despite the problem with system libraries, which in many cases we could use static versions (I believe we cannot only with ICU), dealing with dozen of files has the same complexity of dealing with 2. In some ways yes, in others not. Other embedded database are basically on DLL/.so and you're done. Other that are *embedded only* or that can also be used (and focus on it) as a server like Firebird? Embedded only. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 15-09-2022 03:52, Adriano dos Santos Fernandes wrote: On 05/09/2022 08:13, Mark Rotteveel wrote: That is not really embedded IMHO. Firebird Embedded with Firebird 3.0 has become unwieldy and even harder to use than earlier versions, especially if you compare it to other embedded database systems. How it become harder? Basically, in Firebird 2.5 and earlier, you needed fbembed.dll / libfbembed.so and the intl folder, and you were done. Now you need a whole constellation of files. Other embedded database are basically on DLL/.so and you're done. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 05-09-2022 13:00, Alex Peshkoff via Firebird-devel wrote: When adding it I cared about an ability to fix secirtiy issues in 3d-party libraries not rebuilding and reinstalling FB packages. With used schema one can simply install fixed system package for tommath in a case of some issues in it. None were found - but that was when tommath was at pre-release stage. I find these inconsistencies quite confusing, but lets leave it at that. BTW, we always recommended to those who need embedded-only access full install (classic) and server stop. That is not really embedded IMHO. Firebird Embedded with Firebird 3.0 has become unwieldy and even harder to use than earlier versions, especially if you compare it to other embedded database systems. In any case, this solution would be for Java applications that want a Firebird Embedded without having to install Firebird. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 05-09-2022 12:06, Alex Peshkoff via Firebird-devel wrote: On 9/5/22 12:59, Mark Rotteveel wrote: On 05-09-2022 11:50, Alex Peshkoff via Firebird-devel wrote: On 9/5/22 12:31, Mark Rotteveel wrote: That is inconsistent compared to what is done for --with-builtin-tomcrypt. I explicitly requested --with-builtin-tommath, so I expect it to end up directly in the lib directory like tomcrypt. This switch works another way. It seems to me it does the same thing, but for a different library, so why does it do so in an entirely different way? I do not know what library do you talk about. There are many of them - and rules for them vary slightly. I explicitly mention it multiple times (see also above): tomcrypt. The behaviour for --with-builtin-tomcrypt (putting it directly in lib) seems more sane to me compared to what is done for --with-builtin-tommath (putting it in lib/.tm). Why is it handled differently? Yes, you need to change something not suitable for embedded, I even expect you to add configure switch for it - we never deployed FB for linux w/o install script. We'll see, if I can find the motivation and energy to figure this out. If you just shell scripts and send me changes I can do the rest myself... That is part of the figuring out. I don't normally write shell scripts either, but before that I would also need to figure out exactly what I need to do before I can even start on that. And since I consider this packaging Firebird embedded in a JAR for easy use from Java just a "nice to have", I will shelve this until I have more energy to tackle this. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 05-09-2022 11:50, Alex Peshkoff via Firebird-devel wrote: On 9/5/22 12:31, Mark Rotteveel wrote: On 05-09-2022 11:23, Alex Peshkoff via Firebird-devel wrote: This is solved by install script - it creates symlink to /inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory (provided it's missing). That's done this way in order to make it possible to use system tommath when present. That is inconsistent compared to what is done for --with-builtin-tomcrypt. I explicitly requested --with-builtin-tommath, so I expect it to end up directly in the lib directory like tomcrypt. This switch works another way. It seems to me it does the same thing, but for a different library, so why does it do so in an entirely different way? If I understand you correctly, I will need to massage what I have in buildroot even further to get a usable build. I won't be using the install script, because I don't want to install it, I want to package it for an embedded deployment. Yes, you need to change something not suitable for embedded, I even expect you to add configure switch for it - we never deployed FB for linux w/o install script. We'll see, if I can find the motivation and energy to figure this out. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath
On 05-09-2022 11:23, Alex Peshkoff via Firebird-devel wrote: This is solved by install script - it creates symlink to /inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory (provided it's missing). That's done this way in order to make it possible to use system tommath when present. That is inconsistent compared to what is done for --with-builtin-tomcrypt. I explicitly requested --with-builtin-tommath, so I expect it to end up directly in the lib directory like tomcrypt. If I understand you correctly, I will need to massage what I have in buildroot even further to get a usable build. I won't be using the install script, because I don't want to install it, I want to package it for an embedded deployment. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Building Firebird Linux with --with-builtin-tommath
I'm investigating ways of packaging Firebird Embedded for Java, so it can be used as drop-in Java library (which is then unpacked to a temporary directory automatically). Given Firebird complains about missing libtommath when I use the normal build, and I don't want to impose on users to install additional libraries, I tried a custom build (from current state of v4.0-release branch) with: ./autogen.sh --with-builtin-tommath --with-builtin-tomcrypt --enable-binreloc make make dist I have a problem with libtommath in this build: gen/buildroot/usr/local/firebird/lib contains libtomcrypt.so.1.0.1 (and symlinks libtomcrypt.so and libtomcrypt.so.1), but no libtommath files, these are in a .tm subdirectory of lib, so gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so.0 gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so.0.0.41 gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so I would have expected these files directly in gen/buildroot/usr/local/firebird/lib/, not in gen/buildroot/usr/local/firebird/lib/.tm/. Trying to load embedded this way complains about missing libtommath. Copying those files/symlinks from .tm to the parent directory allows embedded to be loaded. What is wrong here? Did I miss a configuration option somewhere? Is this a bug in the build? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 25-08-2022 15:13, Stefan Heymann wrote: I propose this name convention starting with Firebird 5: Firebird-5.0.0.2816-0-windows-x86-pdb.exe Firebird-5.0.0.2816-0-windows-x86-pdb.zip Firebird-5.0.0.2816-0-windows-x86.exe Firebird-5.0.0.2816-0-windows-x86.zip Firebird-5.0.0.2816-0-windows-x64-pdb.exe Firebird-5.0.0.2816-0-windows-x64-pdb.zip Firebird-5.0.0.2816-0-windows-x64.exe Firebird-5.0.0.2816-0-windows-x64.zip [...] I'd like to suggest you use win32 and win64 instead of x86 and x64. Or, it you want to keep the "windows" prefix, use window-32 and windows-64. For me as a Windows user this would be much more logical (the 86 comes from the 80*86 processor, the 64 comes from 64 bits, so it's not consistent from the beginning). The proposed naming has the benefit it is consistent across platforms. That is: Firebird---[-qualifier]. I don't see the need, nor the point to deviate from this for Windows. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 25-08-2022 12:38, Dimitry Sibiryakov wrote: Dmitry Yemanov wrote 25.08.2022 12:34: Firebird-5.0.0.2816-0-windows-x86-withDebugSymbols.exe Firebird-5.0.0.2816-0-windows-x86-withDebugSymbols.zip Firebird-5.0.0.2816-0-windows-x86.exe Firebird-5.0.0.2816-0-windows-x86.zip Firebird-5.0.0.2816-0-windows-x64-withDebugSymbols.exe Firebird-5.0.0.2816-0-windows-x64-withDebugSymbols.zip Firebird-5.0.0.2816-0-windows-x64.exe Firebird-5.0.0.2816-0-windows-x64.zip Firebird-5.0.0.2816-0-linux-x64.tar.gz Firebird-5.0.0.2816-0-linux-x64-debugSymbols.tar.gz Firebird-5.0.0.2816-0-android-arm.tar.gz (armv7, other?) Firebird-5.0.0.2816-0-android-arm-withDebugSymbols.tar.gz Firebird-5.0.0.2816-0-android-arm64.tar.gz Firebird-5.0.0.2816-0-android-arm64-withDebugSymbols.tar.gz Firebird-5.0.0.2816-0-linux-x86.tar.gz Firebird-5.0.0.2816-0-linux-x86-debugSymbols.tar.gz Firebird-5.0.0.2816-0-source.tar.xz Firebird-5.0.0.2816-0-macos-x64.pkg Looks good to me. Is this zero after build number necessary? We have had -1 releases, e.g. https://firebirdsql.org/en/firebird-4-0-0/#Win32 (Firebird-4.0.0.2496-1-Win32.exe), so for consistency, either it should always be included, or we should ditch this package revision (or whatever we should call it) entirely. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 25-08-2022 03:15, Adriano dos Santos Fernandes wrote: On 24/08/2022 08:45, Adriano dos Santos Fernandes wrote: I propose this name convention starting with Firebird 5: Here is my updated proposal based on the discussion so far: Firebird-5.0.0.2816-0-windows-x86-withDebugSymbols.exe Firebird-5.0.0.2816-0-windows-x86-withDebugSymbols.zip Firebird-5.0.0.2816-0-windows-x86.exe Firebird-5.0.0.2816-0-windows-x86.zip Firebird-5.0.0.2816-0-windows-x64-withDebugSymbols.exe Firebird-5.0.0.2816-0-windows-x64-withDebugSymbols.zip Firebird-5.0.0.2816-0-windows-x64.exe Firebird-5.0.0.2816-0-windows-x64.zip Firebird-5.0.0.2816-0-linux-x64.tar.gz Firebird-5.0.0.2816-0-linux-x64-debugSymbols.tar.gz Firebird-5.0.0.2816-0-android-arm.tar.gz (armv7, other?) Firebird-5.0.0.2816-0-android-arm-withDebugSymbols.tar.gz Firebird-5.0.0.2816-0-android-arm64.tar.gz Firebird-5.0.0.2816-0-android-arm64-withDebugSymbols.tar.gz Firebird-5.0.0.2816-0-linux-x86.tar.gz Firebird-5.0.0.2816-0-linux-x86-debugSymbols.tar.gz Firebird-5.0.0.2816-0-source.tar.xz Firebird-5.0.0.2816-0-macos-x64.pkg Sounds OK. On suggestion: use "with-debug-symbols" and "debug-symbols", but that is just because I'm a bit allergic to capitals in filenames ;) Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 24-08-2022 17:00, Alex Peshkoff via Firebird-devel wrote: On 8/24/22 17:10, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 24.08.2022 16:06: Same for linux - not stripped binaries provided. Nope. Firebird-debuginfo-4.0.2.2816-0.amd64.tar.gz contains only extracted debug symbols, not binaries. Sorry, but they are created by 'cp' command ;) One can run them directly. Nonetheless, those Linux archives are not complete builds. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 24-08-2022 16:06, Alex Peshkoff via Firebird-devel wrote: On 8/24/22 17:04, Mark Rotteveel wrote: On 24-08-2022 15:36, Dimitry Sibiryakov wrote: Mark Rotteveel wrote 24.08.2022 15:32: If the term is confusing or ambiguous, it already is so in its current form. Yes, it is. That's why I would suggest to change that. The Windows pdb packages are complete builds though. Same for linux - not stripped binaries provided. The Linux builds are not complete builds compared to Windows pdb builds (e.g. config files, documentation, security database, etc are missing) Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 24-08-2022 15:36, Dimitry Sibiryakov wrote: Mark Rotteveel wrote 24.08.2022 15:32: If the term is confusing or ambiguous, it already is so in its current form. Yes, it is. That's why I would suggest to change that. The Windows pdb packages are complete builds though. So maybe those should be "debug", while the Linux builds should be "debug-symbols" or something like that. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 24-08-2022 14:50, Dimitry Sibiryakov wrote: Dmitry Yemanov wrote 24.08.2022 14:47: I would unify the `pdb`, `debuginfo`, etc. into simple `debug` suffix. Agreed. And better to keep "debuginfo" verb, IMHO to prevent confusion after expectation to find "a debug build" in "debug" package. What confusion, debuginfo packages are already listed as a "Debug build" on the download page. If the term is confusing or ambiguous, it already is so in its current form. In other words, consolidating it under the term "debug" shouldn't be a problem. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Improve release filenames
On 24-08-2022 13:45, Adriano dos Santos Fernandes wrote: This is our Firebird 4.0.2 released files: [..] It's difficult to understand, they do not mention OS and hardware architectures are inconsistent. I propose this name convention starting with Firebird 5: [..] Firebird-5.0.0.2816-0-linux-x64.tar.gz Firebird-5.0.0.2816-0-linux-x64-debuginfo.tar.gz Firebird-5.0.0.2816-0-linux-arm.tar.gz (armv7, other?) Firebird-5.0.0.2816-0-linux-arm-debuginfo.tar.gz Firebird-5.0.0.2816-0-linux-arm64.tar.gz Firebird-5.0.0.2816-0-linux-arm64-debuginfo.tar.gz [..] In general I'm OK with this, but currently our download page claims the ARM32 and ARM64 build are for Android, and not generically Linux. So, are they actually generically Linux, or specifically for Android? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Purpose of op_batch_sync
On 19-08-2022 17:30, Alex Peshkoff via Firebird-devel wrote: On 8/19/22 18:24, Mark Rotteveel wrote: It looks like op_batch_sync is just used to force the client to send the deferred messages to the server, and the server to send the deferred responses. Is that correct? exactly So, just for confirmation, using op_ping (which I found earlier as a trick to get the server to send deferred messages) is a similar solution? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Purpose of op_batch_sync
It looks like op_batch_sync is just used to force the client to send the deferred messages to the server, and the server to send the deferred responses. Is that correct? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Build master branch with VS 2019
On 19-08-2022 14:04, Gabor Boros wrote: In output_x64_release directory I have common_test.exe and engine_test.exe. Is it normal? I cannot check the snapshot build because "Connection timed out" (http://web.firebirdsql.org/download/snapshot_builds/win/5.0). You can also download snapshots from https://github.com/FirebirdSQL/firebird/actions/workflows/main.yml (e.g. use one built from master) Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Build master branch with VS 2019
On 19-08-2022 12:31, Vlad Khorsun wrote: At last, ensure your .bat files contains Windows-style EOL's (crlf), not Unix style. The repository should contain a .gitattributes file that configures this. I was a bit surprised to see use of unix2dos in the GitHub actions workflow to ensure this for the batch files. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Named parameters in client libraries
On 19-08-2022 12:05, Adriano dos Santos Fernandes wrote: You prefer large over smaller VARCHARs, prefer TIMESTAMP over TIME - widest types But prefer TIMESTAMP and NUMBERS over VARCHAR - most specific I see no logic in your approach. Most specific in the sense that a datetime type is more specific than a string type, for the TIME+TIMESTAMP+VARCHAR example: a TIMESTAMP value contains TIME, so covers both TIME and TIMESTAMP, and a TIMESTAMP is convertible to string, so can also be used with VARCHAR. On the other hand, not all string values are convertible to a TIME or TIMESTAMP value. On the other hand, maybe that could lead to other weird issues (e.g. consider a query TIME+VARCHAR that works fine, but stops working when TIMESTAMP is added in the mix. So, maybe using VARCHAR instead for those cases is the least intrusive alternative after all :( Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Build master branch with VS 2019
On 19-08-2022 10:20, Gabor Boros wrote: 2022.07.03. 12:43 keltezéssel, Gabor Boros írta: I tried to build master branch with VS 2019 Community 2019 16.11.16 on an up to date Windows 10 Pro 64bit. At make_boot got fatal errors in isql_x64.log: Same errors with current state of master branch. Any idea? Have you ever been able to build Firebird locally? I'm not sure if it is still used, but in the past the build required GNU sed, which isn't available by default, and you would need to install it (e.g. from http://gnuwin32.sourceforge.net/packages/sed.htm). Also, given the errors reference the `gen` path, it is likely the preprocess phase failed, and this is just a subsequent error. That said, you probably need one of the core developers to respond to diagnose this better. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Named parameters in client libraries
On 19-08-2022 02:36, Adriano dos Santos Fernandes wrote: On 18/08/2022 04:37, Mark Rotteveel wrote: In other words, most specific type with the highest precision/scale/length. When confronted with combinations of non-string types and string/blob types, use the most specific non-string type. I don't think this is correct, specially the ones I listed above. 1: I would define it as VARCHAR with size using same rules of COALESCE with both types 5: same case as 1, VARCHAR 12: should be error, the two types are incompatible They shouldn't be according to section 4.6 of the SQL standard, specifically Table 3 — Datetime data type conversions. So it would be something like COALESCE. I think that is the wrong approach for parameters. When deriving a type for COALESCE, it makes sense to infer the type to the widest possible type (VARCHAR) to allow all possible stored values to be represented, but with parameters, it is IMHO better to use the most specific type, as that will yield useful information to the user through the metadata, and probably result in the best diagnostic information when specifying invalid values. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Named parameters in client libraries
On 18-08-2022 04:02, Adriano dos Santos Fernandes wrote: And what type (and the deduction rules) this parameter will be described as? I'd suggest looking at what other database systems that do have named parameters currently do. My guess is that they either: 1) don't allow mixing types 2) use the most specific type that is compatible with all positions, otherwise error 3) use the first position 4) use the last position I think rule 2 is probably the best, but I'm not really sure. Some examples for rule 2 1) TIME, TIMESTAMP, VARCHAR(..) => TIMESTAMP 2) CHAR(5), VARCHAR(50), VARCHAR(100) => VARCHAR(100) 3) INTEGER, BIGINT, DECFLOAT => DECFLOAT 4) INTEGER, DATE => error: incompatible types? 5) VARCHAR(50), DOUBLE PRECISION => DOUBLE PRECISION 6) INTEGER, DOUBLE PRECISION => DOUBLE PRECISION 7) NUMERIC(4,2), NUMERIC(38,6) => NUMERIC(38,6) 8) NUMERIC(4,1), NUMERIC(4,4) => NUMERIC(9,4) (note the change in precision to accommodate range of values) 9) SQL_NULL (e.g. :param IS NULL), INTEGER => INTEGER 10) VARCHAR(..), BLOB => BLOB 11) BLOB SUB_TYPE TEXT, BLOB SUB_TYPE BINARY => BLOB SUB_TYPE BINARY 12) TIME WITH TIME ZONE, TIMESTAMP WITHOUT TIME ZONE => TIMESTAMP WITH TIME ZONE ... In other words, most specific type with the highest precision/scale/length. When confronted with combinations of non-string types and string/blob types, use the most specific non-string type. -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Named parameters in client libraries
On 18-08-2022 04:02, Adriano dos Santos Fernandes wrote: How are client libraries (Jaybird, .NET Provider, Delphi ones) describing its named parameters to their users? I mean, given this SQL: select * from rdb$database where :param = 1 or :param = '2' We have here single name used in context with multiple types. I suppose you transform this to: select * from rdb$database where ? = 1 or ? = '2' Which will map to two Firebird parameters with different types. But for the user of the library, I suppose it's one parameter, correct? And what type (and the deduction rules) this parameter will be described as? JDBC doesn't define support for named parameters in java.sql.PreparedStatement, only in java.sql.CallableStatement, which is for calling stored procedures, and that support is optional, so Jaybird doesn't have any support for named parameters. So, if Firebird is going to implement real named parameters, Jaybird is probably not going to support them, and in that case I would really love if this is controlled through a DPB item so when disabled, use of a named parameter results in an error (e.g. "named parameters in DSQL are not enabled" or something). Maybe I can then allow users aware of named parameters to enable it explicitly to map it them themselves, for example if something like `column1 = :param1 and column2 = :param2 and colum3 = :param1` results in two parameters (param1 = position 1, and param2 = position 2), and setting by position still works. The reason Jaybird probably is not going to support this, is because most people don't use the JDBC API directly, but through things like Hibernate, so adding vendor extensions to the JDBC API is pretty much wasted effort because those libraries only use the JDBC API. I could add such methods to the FirebirdPreparedStatement interface, but that would then be a niche feature for people who do use JDBC, and are able to unwrap to the FirebirdPreparedStatement interface (which is not always possible when obtained from a connection pool). In the Java world, named parameters are usually emulated by layers like Hibernate, or Spring's NamedParameterJdbcTemplate, which will map names to positional parameters, and set the values of those positional parameters appropriately. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] op_que_events and database shutdown
On 16-08-2022 11:01, Vlad Khorsun wrote: 16.08.2022 11:26, Jiří Činčura wrote: I just want to know there's nothing on protocol level I can use to help this situation. No response. So I suppose nothing? I didn't found anything suitable. Are aux connections closed when a database is shutdown? In that case you should be able to detect the connection close. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] New statement: EXECUTE SQL
On 15-08-2022 19:42, Vlad Khorsun wrote: Also, I don't like 'sql' word, especially after 'execute statement' and 'execute block'. Too much, as for me :) Syntax with 'with' instead of 'execute sql' looks much better to me, but it is already used in CTE's, thus it seems as not the best choice :( Personally, I do like the EXECUTE SQL for its explicitness, and it fits in the EXECUTE PROCEDURE and EXECUTE BLOCK naming; it is slightly a mismatch/confusing with EXECUTE STATEMENT though. An alternative could be to extend the EXECUTE BLOCK syntax for this so it has a simplified form, without a RETURNS clause. So, it has two forms: EXECUTE BLOCK [(, ...)] [RETURNS (, ...)] AS [] BEGIN [ ...] END and the simplified form EXECUTE BLOCK [(, ...)]] AS [] DO Syntactically, the simplified form could be distinguished by the DO instead of the BEGIN. Presence of the RETURNS clause with DO would raise an error (maybe easiest is to allow it in the parse.y syntax, but raise an error when its present). A downside of this is that statement recognition by clients that currently just look for EXECUTE BLOCK might break. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] New statement: EXECUTE SQL
On 16-08-2022 03:25, Adriano dos Santos Fernandes wrote: On 15/08/2022 14:42, Vlad Khorsun wrote: I like the idea but not syntax. As already mentioned, there it will be hard for app\component devs to parse the whole statement looking for parameters. Note, semicolon usually mark "client" named parameters and it will be near to impossible for, say, Delphi components to correctly preprocess statement like below: execute sql (p1 integer = :p1) do select * from t where t.id = :p1 and t.name = :p2 Client parsers already need to stop at some place. But I understand what you mean, so just read below... You may expect after preprocessing by app it will be like: execute sql (p1 integer = ?) do select * from t where t.id = :p1 and t.name = ? but actually it will be like: execute sql (p1 integer = ?) do select * from t where t.id = ? and t.name = ? without complex re-writing of existing preprocessors. No, I expect that client parsers may not do unnecessary things, at least with this statement. We may report named parameters in the right way (currently EXECUTE BLOCK does not report then, do not know why). For the parsers using :name syntax, that's very easy. Just detect the new statement and do not pre-parse it. AFAIK .NET uses @name syntax. Since that is not a Firebird valid syntax, it should also not be difficult to detect and replace it. So this will not be processed by client at all. It will just read parameter names (p1, p2) from the described statement: execute sql (p1 integer = ?) do select * from t where t.id = :p1 and t.name = :p2 In fact the ugly "= ?" becomes completely unnecessary and better removed in this statement. And it's up to us to continue allowing unnamed parameters in this statement or not. I think we should and people using client libraries that require named (instead positional) parameters could just not use them. Am I reading this correctly that your proposal is to use the following syntax? execute sql (p1 integer, p2 varchar(50)) do select * from t where t.id = :p1 and t.name = :p2 So, no explicit positional parameter markers ('?')? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Is there a way to get the server to send deferred messages?
Is there a way to get the server to send deferred messages without having to read an additional response? Messages like the response to op_batch_create are deferred, meaning you have to trigger a non-deferred action to get the response. I tried to send op_dummy to the server, but that had no effect. I could send an op_ping or another simple request, but that requires consuming an additional response. This is mostly for test purposes, so if using a ping or similar is the only way, then so be it, but if there is something I missed, then that would be appreciated. -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Do repeated op_batch_create leak the batch?
On 15-08-2022 12:47, Alex Peshkoff via Firebird-devel wrote: On 8/13/22 14:59, Mark Rotteveel wrote: I'm implementing batch execution in Jaybird. Looking at the code of rem_port::batch_create(P_BATCH_CREATE* batch, PACKET* sendL) in server.cpp, it looks like sending multiple op_batch_create requests for the same statement handle could leak a previously created batch, as it doesn't call statement->rsr_batch->release() before assigning a new batch to statement->rsr_batch. Is my assessment correct? Seems to be so. In the engine there is a check avoiding opening batch when batch or cursor is already opened, sane should be added to remote server. Add a ticket please. https://github.com/FirebirdSQL/firebird/issues/7262 -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] New statement: EXECUTE SQL
On 15-08-2022 12:40, Simonov Denis via Firebird-devel wrote: I'm talking about the fact that EXECUTE STATEMENT supports named parameters, including those for EXECUTE BLOCK. There's even a special query preparser for that. Those who implement client libraries will have to do much the same. And if for EXECUTE BLOCK the prepaser is quite simple (named parameters need to be searched only between EXECUTE BLOCK and the first AS), then for such a syntax it will not be very trivial. In your version, I don't see how even EXECUTE STATEMENT can work with named parameters. The named parameters in the example are basically equivalent to stored procedure / execute block parameters (a.k.a. PSQL variables), and PSQL supports named parameters in SQL. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] New statement: EXECUTE SQL
On 14-08-2022 01:28, Adriano dos Santos Fernandes wrote: Hi! When one starts with a DSQL command and need to adapt it to EXECUTE BLOCK (for example to use sub routines or use a single parameter in many places), work is difficult when there are many parameters and output fields. Everything must be explicitly declared. I propose new DSQL statement that improve a lot this workflow (and others when not all power of EXECUTE BLOCK is necessary, but it's verbosity is inevitable). I'm calling it EXECUTE SQL, and it's to use with SELECT, UPDATE, DELETE and MERGE, with or without RETURNING. It seats between lack of resources + simplicity of direct SQL command and power + verbosity of EXECUTE BLOCK. Syntax: execute sql [ ( ) ] [ ] do Here is how it can be used: execute sql (p1 integer = ?, p2 integer = ?) declare function subfunc (i1 integer) returns integer as begin return i1; end declare procedure subproc (i1 integer) returns (o1 integer) as begin o1 = i1; suspend; end do select subfunc(:p1) + o1 from subproc(:p2 + ?) Note that parameters may be declared or directly (only in the DO command) used like now. Output is not declared. It's inferred from the DO command. Statement type of the DO command is returned. An interesting idea. I do find the ability to specify question mark parameters at the top and inside the body slightly confusing, but I understand why it is handy to allow this. I do think this statement will complicate the parser I have in Jaybird to detect statement types and whether or not it has RETURNING, but I can adjust. Would this still allow the full SELECT syntax (including WITH clauses, OFFSET/FETCH, GROUP BY, etc), and things like RETURNING for INSERT, etc? I assume this is a toplevel statement only, so it can't occur as a derived table or subquery inside another statement. Is that correct? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Do repeated op_batch_create leak the batch?
I'm implementing batch execution in Jaybird. Looking at the code of rem_port::batch_create(P_BATCH_CREATE* batch, PACKET* sendL) in server.cpp, it looks like sending multiple op_batch_create requests for the same statement handle could leak a previously created batch, as it doesn't call statement->rsr_batch->release() before assigning a new batch to statement->rsr_batch. Is my assessment correct? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parallel workers in isc_action_svc_repair + isc_spb_rpr_sweep_db
On 12-08-2022 15:13, Vlad Khorsun wrote: 12.08.2022 15:31, Jiří Činčura wrote: Hi *, does the isc_dpb_parallel_workers apply when running sweep via isc_action_svc_repair + isc_spb_rpr_sweep_db? Sure. There is no isc_spb_parallel_workers, so I would think it doesn't apply to service actions. What am I missing? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] BLOB_APPEND and NULL
On 11-08-2022 18:09, Alex Peshkoff via Firebird-devel wrote: On 8/11/22 19:04, Vlad Khorsun wrote: So far I don't see it as mistake. +1 Function is anyway damned non-standard. And as soon as we have documented behavior - no problems at all. Time will tell. I think deviating NULL behaviour can lead to confusion, even if it is documented. In any case, I updated the language reference (I used the release notes as a starting point): https://www.firebirdsql.org/file/documentation/html/en/refdocs/fblangref40/firebird-40-language-reference.html#fblangref40-scalarfuncs-blob-append Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] BLOB_APPEND and NULL
On 11-08-2022 18:04, Vlad Khorsun wrote: 11.08.2022 17:46, Jiří Činčura wrote: I was thinking the same when reading the discussion on GH. There was a LOT of time to write something at that discussion. Nobody asked about NULL's there, while it was documented since a very beginning. Unfortunately I don't always have time or energy to monitor all GitHub updates, and there was no previous discussion on this list about BLOB_APPEND, so I simply had not noticed its existence before. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] BLOB_APPEND and NULL
On 11-08-2022 17:58, Vlad Khorsun wrote: 11.08.2022 17:26, Mark Rotteveel wrote: On 11-08-2022 16:21, Vlad Khorsun wrote: 11.08.2022 17:10, Mark Rotteveel wrote: Why was this NULL behaviour chosen? To make BLOB_APPEND more convenient for users. I don't understand how using a different NULL behaviour then standard for operations in SQL/Firebird is convenient. There is no standard defined operation for appending blobs, afaik. I am talking about the general expectations raised by the SQL standard (and Firebird itself) regarding behaviour when faced with NULL, and that is, in general, that when one of the inputs of a function, operation, or other expression is NULL, the result is NULL. And given concatenation is defined for BLOBs in the standard, with the normal NULL behaviour, I would expect this appending to follow the same basic rules. To me it sounds like something that will result in confusion because of the difference with normal concatenation, but OK. The BLOB_APPEND is not CONCATENATION, it is non-standard function with custom semantics. It is a non-standard function that is intended to be used in places where you'd otherwise use concatenation. To me that means semantics of concatenation should be applied. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] BLOB_APPEND and returned SUB_TYPE
I noticed that BLOB_APPEND always returns a blob of SUB_TYPE TEXT, even if the first blob is binary or other type of blob. Is that expected? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] BLOB_APPEND and NULL
On 11-08-2022 16:21, Vlad Khorsun wrote: 11.08.2022 17:10, Mark Rotteveel wrote: Why was this NULL behaviour chosen? To make BLOB_APPEND more convenient for users. I don't understand how using a different NULL behaviour then standard for operations in SQL/Firebird is convenient. To me it sounds like something that will result in confusion because of the difference with normal concatenation, but OK. I had hoped for something more solid, so I could add that as an explanation when documenting it in the language reference. PS You mailed this to me privately instead of to the list. -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] BLOB_APPEND and NULL
The new BLOB_APPEND function has a NULL behaviour that is not consistent with the normal NULL behaviour. Using NULL in BLOB_APPEND behaves as an empty string, while the normal behaviour for functions and operations involving NULL is to result in NULL. For example, normal concatenation: select cast(null as blob sub_type text) || cast('a' as blob sub_type text) || cast(null as blob sub_type text) || cast('b' as blob sub_type text) from rdb$database; results in NULL, while BLOB_APPEND: select blob_append(null, 'a', null, 'b') from rdb$database; results in 'ab'. If the "normal" NULL-behaviour had been used, it would be possible to easily rewrite normal concatenation involving blobs to BLOB_APPEND, but now that requires careful consideration. In addition, this closes an avenue for the engine to - at some point in the future, as an engine optimization - rewrite concatenation to BLOB_APPEND internally. Why was this NULL behaviour chosen? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Ability to use non-local protocol to create DB which alias is declared as self-security
On 09-08-2022 06:33, Alex Peshkoff via Firebird-devel wrote: PPS. If one has access to database.conf to create new alias he definitely has embedded access to server, i.e. problem appears to be rather artificial. Not necessarily. One could have access to the filesystem or even only the configuration files remotely (e.g. using some configuration management tool like Puppet), but not necessarily be able to run process on the host, or create databases. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Is batch release necessary in the wire protocol?
On 16-07-2022 10:54, Mark Rotteveel wrote: Is sending op_batch_rls necessary in the wire protocol, or is closing the statement itself sufficient? Similar question, is a batch automatically released when unpreparing the statement, or preparing a new statement text? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Wrong doc RDB$INDICES.RDB$INDEX_TYPE ?
On 18-07-2022 17:35, Ariel Álvarez wrote: Great! But I see the table format broken in langref 3.0. Check again. Aargh, thanks, and fixed :) Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Wrong doc RDB$INDICES.RDB$INDEX_TYPE ?
On 16-07-2022 11:15, Mark Rotteveel wrote: On 14-07-2022 18:23, Ariel Álvarez wrote: In language reference it is documented: Table RDB$INDICES Column RDB$INDEX_TYPE: Distinguishes between an expression index (1) and a regular index (0 or null). Not used in databases created before Firebird 2.0; hence, regular indexes in upgraded databases are more more likely to store null in this column But, according to this discussion: https://github.com/FirebirdSQL/firebird/issues/4085 <https://github.com/FirebirdSQL/firebird/issues/4085> RDB$INDEX_TYPE identifies a descending or ascending index. It appears the doc is wrong It looks like it. The error also occurs in The Firebird Book, Second Edition. I guess the error derives from there. The first edition of The Firebird Book says "Not currently used; likely to distinguish regular indexes from expression indexes when the feature is implemented", so I guess when expression indexes were added in Firebird 2, the assumption was codified in the book, and then copied when the Language Reference was written. I'll double check this and fix the documentation. I published a new version of the docs with this fixed. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Wrong doc RDB$INDICES.RDB$INDEX_TYPE ?
On 14-07-2022 18:23, Ariel Álvarez wrote: In language reference it is documented: Table RDB$INDICES Column RDB$INDEX_TYPE: Distinguishes between an expression index (1) and a regular index (0 or null). Not used in databases created before Firebird 2.0; hence, regular indexes in upgraded databases are more more likely to store null in this column But, according to this discussion: https://github.com/FirebirdSQL/firebird/issues/4085 <https://github.com/FirebirdSQL/firebird/issues/4085> RDB$INDEX_TYPE identifies a descending or ascending index. It appears the doc is wrong It looks like it. The error also occurs in The Firebird Book, Second Edition. I guess the error derives from there. The first edition of The Firebird Book says "Not currently used; likely to distinguish regular indexes from expression indexes when the feature is implemented", so I guess when expression indexes were added in Firebird 2, the assumption was codified in the book, and then copied when the Language Reference was written. I'll double check this and fix the documentation. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Is batch release necessary in the wire protocol?
Is sending op_batch_rls necessary in the wire protocol, or is closing the statement itself sufficient? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] More attention to preventing breaking changes in point releases
I think https://github.com/FirebirdSQL/firebird/issues/6987 shows that we should be more careful with introducing (or better, preventing) breaking changes in point releases. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] SIO_LOOPBACK_FAST_PATH deprecated
On 02-02-2021 00:45, Vlad Khorsun wrote: 02.02.2021 1:35, Leyne, Sean wrote: https://www.gitmemory.com/issue/grpc/grpc/18057/486312183 There is no mention of SIO_LOOPBACK_FAST_PATH in this link Read carefully, please. https://support.microsoft.com/en-us/topic/stop-error-0xd1-when-you-set- the-sio-loopback-fast-path-flag-in-windows-8-or-windows-server-2012- 14399334-f3a8-b731-3799-12899a79bf35 The HotFix is from 2012, with the last update in 2015. I think that the underlying issue has been resolved/mitigated. We can only guess. Therefore I don't offer to remove the feature, but consider to deactivate it by default. I want to revisit this discussion. I think that given Microsoft deprecated SIO_LOOPBACK_FAST_PATH and warns against it, we should either remove this entirely from Firebird 5.0, or at minimum disable it by default. Thoughts? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Add scroll fetch support in legacy API?
On 04-06-2022 15:17, Dmitry Yemanov wrote: 04.06.2022 15:07, Mark Rotteveel wrote: Currently scroll fetch is only exposed in the OO API. Any plans for exposing it in the legacy API Nope, we have an agreement to not extending the legacy API with new features. Ok, too bad. (e.g. isc_dsql_fetch_scroll / fb_dsql_fetch_scroll)? That's not enough, some method is also needed to mark the cursor as scrollable before opening. The new cursor's getInfo() should also be duplicated. And maybe isBof/isEof too. Ah, right. Though isBof/isEof is technically not necessary. You can infer that from the status return value and the last fetch direction (fetching relative 0 when before-first requires some of your own bookkeeping though) Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Add scroll fetch support in legacy API?
Currently scroll fetch is only exposed in the OO API. Any plans for exposing it in the legacy API (e.g. isc_dsql_fetch_scroll / fb_dsql_fetch_scroll)? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Strange CAST result
On 25-05-2022 10:08, artyom.abakumov wrote: The CAST function returns a strange result in the following case: SQL> SELECT CAST(345.12e-2 as VARCHAR(10)) from RDB$DATABASE; CAST == 3.45 Is it a correct behavior? LI-V4.0.2.2770 Firebird 4.0 It does seem off, I would expect it to output 3.4512, same as happens for cast(3.4512e0 as varchar(10)). There are other oddities, e.g. cast(x as varchar(10)) where x is a literal gives weird results: 34512e-4: 3.4512000 3451.2e-3 : 3.5 34.512e-1 : 3.451 3.4512e0: 3.4512 0.34512e1 : 3.45120 0.034512e2 : 3.451200 0.0034512e3 : 3.4512000 These should basically be the same value Also select cast(x as varchar(10)) from (select cast(345.12e-2 as double precision) as x from rdb$database); results in 3.3412000 while select cast(x as varchar(10)) from (select 345.12e-2 as x from rdb$database); results in 3.45, which is odd as 345.12e-2 is a double precision literal, but it looks like it gets coerced to something else in the presence of the cast to varchar. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Incorrect string right truncation errors in Firebird 5
On 17-05-2022 15:07, Adriano dos Santos Fernandes wrote: On 17/05/2022 09:54, Mark Rotteveel wrote: Thanks! I will change Jaybird to switch to using blr_varying2/blr_text2 for Jaybird 4.0.7 and Jaybird 5. When you did it for test, did you used the charset number or CS_dynamic? I'm asking because the current code seems to have the same problem if test2/varying2 is used with CS_dynamic instead of the actual number. It used the character set id of the column as obtained from statement information isc_info_sql_bind, isc_info_sql_describe_vars (specifically isc_info_sql_sub_type), which in the case of this test is 4 (UTF8). Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Incorrect string right truncation errors in Firebird 5
On 17-05-2022 14:47, Adriano dos Santos Fernandes wrote: On 17/05/2022 08:30, Mark Rotteveel wrote: Is this happening with fbclient library too? Good question: no it doesn't. Which suggests Jaybird is doing something different. Jaybird uses blr_varying/blr_text, not blr_varying2/blr_text2 when sending the BLR of the execute. Could that make a difference? Changing Jaybird to write blr_varying2/blr_text2 with the sub type (character set) fixes the issue. However, I thought that using blr_varying/blr_text (judging by MetadataFromBlr::MetadataFromBlr in common\classes\InternalMessageBuffer.cpp, Firebird selects CS_dynamic, which should use the connection character set (UTF8) in this case. Is it possible that `CharSet* const fromCharSet = INTL_charset_lookup(tdbb, from_cs);` doesn't handle CS_dynamic properly? Or that this happens in a codepath where BLR decoding doesn't assign CS_dynamic, but it remains zero (NONE)? The engine is receiving the string using NONE charset, which makes things bad there. This happens due to this place: - // ASF: Older than 2.5 engine hasn't validating strings in DSQL. After this has been // implemented in 2.5, selecting a NONE column with UTF-8 attachment charset started // failing. The real problem is that the client encodes SQL_TEXT/SQL_VARYING using // blr_text/blr_varying (i.e. with the connection charset). I'm reseting the charset // here at the server as a way to make older (and not yet changed) client work // correctly. if (desc.isText() && desc.getTextType() == ttype_dynamic) desc.setTextType(ttype_none); - So it is happening for a long time and may make things incorrect in Jaybird. Anyway, it seems I'll need to change the fix for #7179 to take care of real NONE/BINARY source strings. Thanks! I will change Jaybird to switch to using blr_varying2/blr_text2 for Jaybird 4.0.7 and Jaybird 5. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Incorrect string right truncation errors in Firebird 5
On 16-05-2022 21:00, Adriano dos Santos Fernandes wrote: On 16/05/2022 12:07, Mark Rotteveel wrote: On 16-05-2022 16:50, Mark Rotteveel wrote: I was running some tests against Firebird-5.0.0.494-0_x64 (latest snapshot, from last Saturday), and I notice that I get incorrect string right truncation errors with CHAR/VARCHAR. I currently cannot dive deeper into it, but as a datapoint, the error does not occur with Firebird-5.0.0.488-0_x64. Based solely on the commit message, maybe this commit is at fault? https://github.com/FirebirdSQL/firebird/commit/dd18a3b11b28c3ed8126a6f54b829989954bfa03 The Jaybird test that triggers this is org.firebirdsql.jdbc.TestFBPreparedStatementUTF8, specifically tests: - connectionUtf8_insertMultiByte_inWin1252_char_1_win1252_succeeds (fails with "expected length 1, actual 2") [..] - connectionUtf8_insertMultiple_lessThanMax_varchar_5_utf8_succeeds (fails with "expected length 5, actual 6") For CHAR, it seems to count 1 more character, for VARCHAR 2-3 more characters. Is this happening with fbclient library too? Good question: no it doesn't. Which suggests Jaybird is doing something different. Jaybird uses blr_varying/blr_text, not blr_varying2/blr_text2 when sending the BLR of the execute. Could that make a difference? Changing Jaybird to write blr_varying2/blr_text2 with the sub type (character set) fixes the issue. However, I thought that using blr_varying/blr_text (judging by MetadataFromBlr::MetadataFromBlr in common\classes\InternalMessageBuffer.cpp, Firebird selects CS_dynamic, which should use the connection character set (UTF8) in this case. Is it possible that `CharSet* const fromCharSet = INTL_charset_lookup(tdbb, from_cs);` doesn't handle CS_dynamic properly? Or that this happens in a codepath where BLR decoding doesn't assign CS_dynamic, but it remains zero (NONE)? Is the error in insert or select? The error happens on insert. I'm failing to reproduce it in isql, for example for connectionUtf8_insertMultipleInWin1252_varchar_5_win1252_succeeds: - CREATE TABLE utf8table ( id INTEGER, char_1_none CHAR(1) CHARACTER SET NONE, char_1_utf8 CHAR(1) CHARACTER SET UTF8, char_1_win1252 CHAR(1) CHARACTER SET WIN1252, char_5_none CHAR(5) CHARACTER SET NONE, char_5_utf8 CHAR(5) CHARACTER SET UTF8, char_5_win1252 CHAR(5) CHARACTER SET WIN1252, varchar_5_none VARCHAR(5) CHARACTER SET NONE, varchar_5_utf8 VARCHAR(5) CHARACTER SET UTF8, varchar_5_win1252 VARCHAR(5) CHARACTER SET WIN1252 ); /* select unicode_char(0x00FE) || unicode_char(0x00A3) || 'a' || unicode_char(0x0160) || ',' from rdb$database; */ set bulk_insert INSERT INTO utf8table (id, char_5_win1252) VALUES (?, ?); (1, 'þ£aŠ,') stop ; select char_5_win1252 from utf8table where id = 1; - The problem seems to be it is counting characters UTF8 using WIN1252 or maybe NONE, so UTF8 characters that take two bytes are counted as two characters. As an example, if I change a failing test to use "12345" it works, if I change it to "\u00fe2345" (\u00fe is two bytes in UTF-8), the error is "expected length 5, actual 6". If I change it to "\u20ac2345" (\u20ac is three bytes in UTF-8), the error is "expected length 5, actual 7". Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Incorrect string right truncation errors in Firebird 5
On 16-05-2022 16:50, Mark Rotteveel wrote: I was running some tests against Firebird-5.0.0.494-0_x64 (latest snapshot, from last Saturday), and I notice that I get incorrect string right truncation errors with CHAR/VARCHAR. I currently cannot dive deeper into it, but as a datapoint, the error does not occur with Firebird-5.0.0.488-0_x64. Based solely on the commit message, maybe this commit is at fault? https://github.com/FirebirdSQL/firebird/commit/dd18a3b11b28c3ed8126a6f54b829989954bfa03 The Jaybird test that triggers this is org.firebirdsql.jdbc.TestFBPreparedStatementUTF8, specifically tests: - connectionUtf8_insertMultiByte_inWin1252_char_1_win1252_succeeds (fails with "expected length 1, actual 2") - connectionUtf8_insertMultiByte_notInWin1252_char_1_win1252_fails (fails with "expected length 1, actual 2", should fail with "Cannot transliterate character between character sets") - connectionUtf8_insertMultipleInWin1252_char_5_win1252_succeeds (fails with "expected length 5, actual 8") - connectionUtf8_insertMultipleInWin1252_varchar_5_win1252_succeeds (fails with "expected length 5, actual 8") - connectionUtf8_insertMultipleInWin1252_lessThanMax_varchar_5_win1252_succeeds (fails with "expected length 5, actual 6") -connectionUtf8_insertMultiByte_char_1_utf8_succeeds (fails with "expected length 1, actual 2") - connectionUtf8_insertMultiple_char_5_utf8_succeeds (fails with "expected length 5, actual 8") - connectionUtf8_insertMultiple_varchar_5_utf8_succeeds (fails with "expected length 5, actual 8") - connectionUtf8_insertMultiple_lessThanMax_varchar_5_utf8_succeeds (fails with "expected length 5, actual 6") For CHAR, it seems to count 1 more character, for VARCHAR 2-3 more characters. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Incorrect string right truncation errors in Firebird 5
I was running some tests against Firebird-5.0.0.494-0_x64 (latest snapshot, from last Saturday), and I notice that I get incorrect string right truncation errors with CHAR/VARCHAR. I currently cannot dive deeper into it, but as a datapoint, the error does not occur with Firebird-5.0.0.488-0_x64. Based solely on the commit message, maybe this commit is at fault? https://github.com/FirebirdSQL/firebird/commit/dd18a3b11b28c3ed8126a6f54b829989954bfa03 Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Refresh roles
On 11-05-2022 18:56, Alex Peshkoff via Firebird-devel wrote: On the other hand I see no problems if we have dedicated command peforming desired action. Suggest syntax: SET ROLE without parameters. (but not insist on it) To me, `SET ROLE` without a role name doesn't feel correct, and doesn't properly convey intent. Given we already have `ALTER SESSION RESET`, I would suggest `ALTER SESSION UPDATE PRIVILEGES` or something like that. However, I'm wondering how other database systems behave in a scenario like this. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] DEFAULT grants to PUBLIC
On 10-05-2022 12:10, Dimitry Sibiryakov wrote: Roman Simakov wrote 10.05.2022 10:21: It makes sense. Could you create a ticket? But what's the point to grant object rights to the role and then grant the role to public instead of granting the rights to public directly? Just like with normal default roles: granularity. You can group related permissions together, and make is easier to revoke those permissions en bloc if you no longer want everyone to have them. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_spb_version Call Without Global User?
On 03-05-2022 18:56, Dimitry Sibiryakov wrote: Scott Morgan via Firebird-devel wrote 03.05.2022 18:29: What can you do to get the server's info if you only have per-DB user accounts? Use isc_dpb_expected_db with your database name to inform Firebird which setting from databases.conf should be used during service attach (including security database). I assume you mean isc_spb_expected_db, right? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Remove travis-ci?
On 30-04-2022 12:05, Mark Rotteveel wrote: On 28-04-2022 11:51, Adriano dos Santos Fernandes wrote: On 28/04/2022 05:37, Mark Rotteveel wrote: Our travis-ci builds haven't actually run for the past 10 months as we ran out of credits. As no one has noticed this in that time, it looks like the travis-ci builds serve no actual purpose. Any objections if I remove the travis-ci build definitions, and revoke the travis-ci permissions from github.com/FirebirdSQL ? No objections. They seem to had a security problem. I have been notified by GitHub that my account was used to "list my organizations" in a broader attack: https://github.blog/2022-04-15-security-alert-stolen-oauth-user-tokens/ I have removed the Travis related things from B3_0_Release, v4.0-release and master and revoked the permission to the repository. I didn't touch B2_5_Release. I still need to check two of the Java repositories before I can fully remove Travis. All Travis access has been revoked. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Remove travis-ci?
On 28-04-2022 11:51, Adriano dos Santos Fernandes wrote: On 28/04/2022 05:37, Mark Rotteveel wrote: Our travis-ci builds haven't actually run for the past 10 months as we ran out of credits. As no one has noticed this in that time, it looks like the travis-ci builds serve no actual purpose. Any objections if I remove the travis-ci build definitions, and revoke the travis-ci permissions from github.com/FirebirdSQL ? No objections. They seem to had a security problem. I have been notified by GitHub that my account was used to "list my organizations" in a broader attack: https://github.blog/2022-04-15-security-alert-stolen-oauth-user-tokens/ I have removed the Travis related things from B3_0_Release, v4.0-release and master and revoked the permission to the repository. I didn't touch B2_5_Release. I still need to check two of the Java repositories before I can fully remove Travis. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors
On 22-04-2022 17:51, Dmitry Yemanov wrote: 22.04.2022 14:49, Mark Rotteveel wrote: I was wondering if this - fetching rowcount of a scrollable cursor - was already implemented. If so, how do I get it? And if not, when can I expect it to be implemented? PR #7083, was left for review but not merged yet. I see it has conflicts now, I will address them tomorrow and then merge the PR. I notice that if I get the INF_RECORD_COUNT with op_info_cursor before the first fetch, I will get the record count, but the subsequent fetch will fail with a Dynamic SQL Error; SQLDA error; Data type unknown; at SQLVAR index 0 [SQLState:07002, ISC error code:335544583]. If I get the record count after the first fetch, everything is OK. This seems to indicate that retrieving the record count somehow sets an empty SQLDA or something like that, and ignores the one from the first fetch. Is this a bug? Should retrieving cursor info before the first fetch be disallowed by the server? Or maybe something else? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Remove travis-ci?
Our travis-ci builds haven't actually run for the past 10 months as we ran out of credits. As no one has noticed this in that time, it looks like the travis-ci builds serve no actual purpose. Any objections if I remove the travis-ci build definitions, and revoke the travis-ci permissions from github.com/FirebirdSQL ? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors
On 08-12-2021 10:13, Dmitry Yemanov wrote: 28.11.2021 14:45, Mark Rotteveel wrote: 3) "row count" makes it possible to know the position after fetchLast() and everything else could be calculated locally by the client library, thus making the server-supported "current position" totally unnecessary. Do I miss anything? Could we agree on having only "row count" returned via op_info_cursor and leaving "cursor position" (getRow() in Java API) up the connectivity library developers? I was wondering if this - fetching rowcount of a scrollable cursor - was already implemented. If so, how do I get it? And if not, when can I expect it to be implemented? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Security vulnerability in zlib library
On 2022-03-31 15:39, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 31.03.2022 15:21: Note that the crash happen on compression so it doesn't affect Firebird security. Did not catch why - we use zlib compression on the wire (since fb3) and in gbak (since fb4). Both cases are not default but anyway not good. The crash happen when a stream of definite data is tried to be compressed. IMHO, it is hard (if possible at all) to purposefully construct such stream *from* server to crash or exploit it. That is a very dangerous assumption. Things people think "that is not possible to get exploited in our case" always seem to get exploited by people with sufficient motivation and drive. And even if it is not exploitable in the case of Firebird, that is not a reason not to update the dependency in a next release. It costs nearly nothing to do, and it avoids the potential for vulnerabilities, and the *perception* of being vulnerable. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Security vulnerability in zlib library
A security vulnerability was found in zlib: https://nakedsecurity.sophos.com/2022/03/29/zlib-data-compressor-fixes-17-year-old-security-bug-patch-errr-now/ Will we include an updated version in the next release? Can people just drop in a replacement? Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_dpb_lc_ctype and isc_dpb_set_db_charset and isc_dpb_utf8_filename
On 2022-03-01 12:47, Alex Peshkoff via Firebird-devel wrote: On 2/28/22 21:39, Jiří Činčura wrote: Thanks. I suppose not only TPB, but also buffers for batch, events, etc., right? Yes. Basically only DPB and SPB are "UTF8-ready". If one wants everyting to be utf8 why not have appropriate attachment? Having the connection character set UTF8 is IMHO a separate concern from driver/protocol-related communication. The benefits of using UTF8 for driver/protocol communications doesn't always work well with the limitations that UTF8 enforces on the size of (VAR)CHAR columns. If we ever design a new protocol, things like this need to be taken into account. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2022-02-16 09:56, Vlad Khorsun wrote: 16.02.2022 10:35, Mark Rotteveel wrote: On 2022-02-15 20:08, Vlad Khorsun wrote: 15.02.2022 20:32, Mark Rotteveel wrote: On 2022-02-15 16:20, Vlad Khorsun wrote: I'd vote to remove idx_numeric2 in favour of idx_decimal and look how to improve Decimal128::makeIndexKey() performance. For example, it stores every 3 decimal digits into 10 bits using relatively complex (computationally) algorithm with shifts and multiplications. Storing every digit in 4 bits will use 2 bits per 3 digits more but should save come CPU circles, I believe. But that is the storage format of Decimal128 (DECFLOAT(34)). No. It looks more like packed BCD. DECFLOAT uses a more complex internals, it is not BCD inside. Decimal128 uses densely packed BCD (with some oddities to encode sign, exponent and first digit). It is not BCD and definitely not what uses Decimal128::makeIndexKey(). See http://speleotrove.com/decimal/dbspec.html and http://speleotrove.com/decimal/DPDecimal.html Also, note, Decimal encodings is not directly sortable. From that first link under 'coefficient continuation': "Each 10-bit group represents three decimal digits, using Densely Packed Decimal encoding." and at the footnote (and the top of your second link): "Chen-Ho encoding is a lossless compression of three Binary Coded Decimal digits into 10 bits using an algorithm which can be applied or reversed using only simple Boolean operations". However, that was not my main point. My main point was that it sounds like an index format that was created for supporting DECFLOAT(34), and that it is not suitable for the full range of INT128 and NUMERIC/DECIMAL backed by INT128 (for the same reasons the DOUBLE PRECISION format is not suitable for BIGINT and NUMERIC/DECIMAL backed by BIGINT). Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2022-02-15 20:08, Vlad Khorsun wrote: 15.02.2022 20:32, Mark Rotteveel wrote: On 2022-02-15 16:20, Vlad Khorsun wrote: I'd vote to remove idx_numeric2 in favour of idx_decimal and look how to improve Decimal128::makeIndexKey() performance. For example, it stores every 3 decimal digits into 10 bits using relatively complex (computationally) algorithm with shifts and multiplications. Storing every digit in 4 bits will use 2 bits per 3 digits more but should save come CPU circles, I believe. But that is the storage format of Decimal128 (DECFLOAT(34)). No. It looks more like packed BCD. DECFLOAT uses a more complex internals, it is not BCD inside. Decimal128 uses densely packed BCD (with some oddities to encode sign, exponent and first digit). Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] INT64 and index keys
On 2022-02-15 16:20, Vlad Khorsun wrote: I'd vote to remove idx_numeric2 in favour of idx_decimal and look how to improve Decimal128::makeIndexKey() performance. For example, it stores every 3 decimal digits into 10 bits using relatively complex (computationally) algorithm with shifts and multiplications. Storing every digit in 4 bits will use 2 bits per 3 digits more but should save come CPU circles, I believe. But that is the storage format of Decimal128 (DECFLOAT(34)). It makes no sense to replace that with something else for DECFLOAT(34) columns. So maybe a different index format is necessary for INT128 (and NUMERIC(38, ...), because Decimal128 doesn't sound suitable for those types as it has less precision (34 digits instead of 38) (similar like the double precision index format is not suitable for INT64/BIGINT). Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Statement type for INSERT ... RETURNING
On 11-02-2022 17:46, Mark Rotteveel wrote: On 2022-02-11 12:54, Jiří Činčura wrote: Is there a way to get as a response to isc_info_sql_stmt_type for "INSERT ... RETURNING" isc_info_sql_stmt_insert instead of isc_info_sql_stmt_exec_procedure? Or something similar that would specify it's an "insert"? No. In fact, with Firebird 5, statements with RETURNING can even identify as isc_info_sql_stmt_select. The statement types are more about how you should handle statement execution than anything else. When it was discussed last time, I believe the idea was raised to add a new info type to get this information, but that it would first need a real use case to do so. I'll see if I can find that discussion. See discussions: - subject "isc_info_sql_stmt_type/isc_info_sql_stmt_flags" on the 19th and 20th of August 2021 - subject "Firebird 5 and Update...Returning" on the 26th of November 2021 Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Statement type for INSERT ... RETURNING
On 2022-02-11 12:54, Jiří Činčura wrote: Is there a way to get as a response to isc_info_sql_stmt_type for "INSERT ... RETURNING" isc_info_sql_stmt_insert instead of isc_info_sql_stmt_exec_procedure? Or something similar that would specify it's an "insert"? No. In fact, with Firebird 5, statements with RETURNING can even identify as isc_info_sql_stmt_select. The statement types are more about how you should handle statement execution than anything else. When it was discussed last time, I believe the idea was raised to add a new info type to get this information, but that it would first need a real use case to do so. I'll see if I can find that discussion. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] User name case
On 2022-01-19 16:46, Dimitry Sibiryakov wrote: Alex Peshkoff via Firebird-devel wrote 18.01.2022 18:03: fb_utils::dpbItemUpper(userName); That's quite strange routine. It will uppercase the string even if it contains ASCII letters disallowed for not-delimited SQL identifier (which user name is). A user name is a delimited identifier since Firebird 3.0, but if it was delimited, it wouldn't be uppercased. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_usrname_too_long
On 14-01-2022 17:10, Dimitry Sibiryakov wrote: Mark Rotteveel wrote 14.01.2022 17:06: Also shouldn't it to be returned if user name received in DPB (after conversion into UTF-8) doesn't fit as well? I'm not sure what you mean with that. Currently if someone put into DPB non-ASCII user name and during conversion into UTF-8 it doesn't fit 255 bytes the error "attempt to store %d bytes in a clumplet with maximum size 255 bytes" is thrown. Shouldn't the error to be a little more specific and comprehensive...? Personally, I don't think so, as I think that is messy to conflate logic for populating the DPB with selecting appropriate messages specific to the select DPB item. Also when you use a DPB v2, it is wide, so accepts far larger values than you'll ever logically need for a user name. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] isc_usrname_too_long
On 14-01-2022 16:41, Dimitry Sibiryakov wrote: Hello All. Shouldn't subj text to be corrected? I probably should. Also shouldn't it to be returned if user name received in DPB (after conversion into UTF-8) doesn't fit as well? I'm not sure what you mean with that. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Sub routines accessing outer variables, parameters and cursors
On 2022-01-11 20:15, Adriano dos Santos Fernandes wrote: Hi! Currently sub routines cannot access the outer (main routine/block) variables, parameters and cursors - #4769. There is some details to make that happen safely. Currently a variable cannot have its value changed while its being used. var = var + No expression can reset var. As soon we let sub routines change outer variables, that will be possible. Currently VariableNode just returns a descriptor pointing to the variable's impure_value present in the variable declaration. To make outer access safe, variables accessed in outer scope would be flagged and its VariableNode (even in the main routine/block) should copy the value to its own impure_value. It will be a bit slower, but I don't see a problem. What worried me more is how to represent things in BLR. A simple approach would be to have blr_outer_variable. But it is not only variables. There is also parameters, fields, dbkeys, cursor statements. What seems for me the way to go is to implement single blr_outer_map that maps main routine's things to sub routines in they own number space. blr_outer_map, blr_outer_map_variable, 0,0, 3,0 It maps main's variable 0,0 to this sub routine's 3,0, so it can access with blr_variable, 3,0. blr_outer_map, blr_outer_map_cursor, 1,0, 2,0 Maps main's cursor 1,0 to this sub routine's 2,0, so it can access with blr_cursor_stmt, 2,0. Comments? It is not entirely clear to me what you're proposing and what the motivation is for the proposal. Are you proposing to allow local subroutines to modify their parents variable, or are you proposing something to allow reading a local variable, but prevent modification? Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Commit (un)certainity
On 2022-01-05 21:48, Jiří Činčura wrote: Yes. You probably have the same problem in your provider, no? The problem exists there. Same as with fbclient. But I don't explicitly solve it, application developer is responsible for handling such scenario (if ever). I agree. If the application developer thinks this is a problem, they should switch to using 2-phase commits and having a scavenger to handle limbo transactions. BTW: I think in most cases you should be able to infer from the socket error if the problem occurred on sending or on reading data (at least, when using blocking socket reads/writes, not sure about non-blocking). Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Content for p_batch_create.p_batch_pb
On 26-12-2021 09:59, Alex Peshkoff via Firebird-devel wrote: On 12/16/21 11:51, Jiří Činčura wrote: Or maybe more broadly, is there some high level description of the batching? I'm failing to find something, except batch interface description in Using_OO_API.html, but that's about API, not about the "protocol". Added chapter to https://github.com/FirebirdSQL/firebird-documentation/blob/master/src/docs/asciidoc/en/firebirddocs/wireprotocol/firebird-wire-protocol.adoc#wireprotocol-batches-release I did some minor copy editing and published it. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Content for p_batch_create.p_batch_pb
On 26-12-2021 09:59, Alex Peshkoff via Firebird-devel wrote: On 12/16/21 11:51, Jiří Činčura wrote: Or maybe more broadly, is there some high level description of the batching? I'm failing to find something, except batch interface description in Using_OO_API.html, but that's about API, not about the "protocol". Added chapter to https://github.com/FirebirdSQL/firebird-documentation/blob/master/src/docs/asciidoc/en/firebirddocs/wireprotocol/firebird-wire-protocol.adoc#wireprotocol-batches-release Thanks Alex! Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Documentation of SET DEBUG OPTION
On 24-12-2021 13:14, Adriano dos Santos Fernandes wrote: On 24/12/2021 07:12, Mark Rotteveel wrote: Checking the changes of Firebird 4.0.1 to update the docs, I noticed that a new statement `SET DEBUG OPTION` has been introduced, but I can't find much about it, except for "To make isc_info_sql_exec_path_blr_* work, session must be first execute set debug option dsql_keep_blr = true." in https://github.com/FirebirdSQL/firebird/issues/6910 Questions: 1. Is dsql_keep_blr the only option at the moment, or are there more? (the code suggests not) It's the only one. 2. Is SET DEBUG OPTION general for an entire connection, or does it need to be executed on the statement handle you want to retrieve one of the isc_info_sql_exec_path_blr_* options from? (checking the commit, it seems to be attachment level, but I'd like confirmation It's for the attachment. 3. How do you clear an option, set value to false, can you set to null? SET DEBUG OPTION valid_symbol_name '=' constant The general command accepts any constant (NULL is not included). DSQL_KEEP_BLR requires a boolean constant. But it can also be implicit converted via strings ('true' or 'false'). Also, is there more to it than this? It's good to reproduce the warning put in the issue: Warning: this feature is very tied to engine internals and its usage is discouraged if you do not understand very well how these internals are subject to change between versions. Thanks, I added the following: - ISQL: https://www.firebirdsql.org/file/documentation/html/en/firebirddocs/isql/firebird-isql.html#isql-set-exec-path-display - langref: https://www.firebirdsql.org/file/documentation/chunk/en/refdocs/fblangref40/fblangref40-management-debug.html#fblangref40-management-setdebugoption Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Documentation of SET DEBUG OPTION
Checking the changes of Firebird 4.0.1 to update the docs, I noticed that a new statement `SET DEBUG OPTION` has been introduced, but I can't find much about it, except for "To make isc_info_sql_exec_path_blr_* work, session must be first execute set debug option dsql_keep_blr = true." in https://github.com/FirebirdSQL/firebird/issues/6910 Questions: 1. Is dsql_keep_blr the only option at the moment, or are there more? (the code suggests not) 2. Is SET DEBUG OPTION general for an entire connection, or does it need to be executed on the statement handle you want to retrieve one of the isc_info_sql_exec_path_blr_* options from? (checking the commit, it seems to be attachment level, but I'd like confirmation 3. How do you clear an option, set value to false, can you set to null? Also, is there more to it than this? -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] varbinary
On 2021-12-20 14:04, Norbert Saint Georges wrote: Thank you for your two responses. unfortunately I have a problem with an old project, following the update of the netprovider 8.5 client. Several varchar (x) character set bytes are declared there which crashes on an expected byte [] even though I have been sending it strings for several months :-) but ok :-) That sounds like something to ask about on https://groups.google.com/g/firebird-net-provider Make sure to include the necessary code and the actual error you receive. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors
On 2021-12-20 11:51, Dmitry Yemanov wrote: Mark et al, Looked at it again, and being able to get the total row count will work for me. Is this information already available, or does this still need to be implemented? What would you expect from the "row count" requested for a non-scrollable cursor? It cannot return the true count, as already explained here. Should it return some magic number (0? 1? -1?) or would error be appropriate? If the latter, should it be isc_infinap (information type inappropriate for object specified) or isc_infona (no information of this type available for object specified)? I'd say -1 as a magic value to signal "unknown". Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] varbinary
On 2021-12-20 12:40, Norbert Saint Georges wrote: Hello, a varbinary has a subtype of 1 and not 0 (binary) normal? You're thinking of BLOB SUB_TYPE, and this is unrelated to BLOB SUB_TYPE, as it isn't a BLOB type. In Firebird 4, VARBINARY is introduced as *an alias* for VARCHAR CHARACTER SET OCTETS (and BINARY for CHAR CHARACTER SET OCTETS), and "normal" VARCHAR (or CHAR) has sub_type 0. The character set OCTETS has id 1, and some parts of Firebird communicate the character set in the sub_type field (e.g. of XSQLDA), so to make this backwards compatible, the character set id of OCTETS (1) is also used as its RDB$FIELD_SUB_TYPE value. In that way, "old" tools handle it correctly as (VAR)CHAR CHARACTER SET OCTETS, while Firebird 4.0 tools can identify it as (VAR)BINARY. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors
On 08-12-2021 10:13, Dmitry Yemanov wrote: 28.11.2021 14:45, Mark Rotteveel wrote: We don't have anything like this. Theoretically, we could extend IRecordSet with something similar (although it would also require a protocol change), but the question is whether it's really needed. Personally, I don't see it useful per se. If users want to know a number of rows, then perhaps an explicit getRowCount() method would be more useful (*). But AFAIU the Java API does not mention it. (*) not applicable to uni-directional cursors? The intent of the method is to report the position of the current row. The example it is abused by some to get the total size was just an illustration of why people would expect a value after requesting the last row. But the same would be a problem if people went to last, scrolled around and then want to know the current row position. I was thinking about providing both "current position" and "row count" information values for a cursor. Both are doable, however the former has some issues: [..] 3) "row count" makes it possible to know the position after fetchLast() and everything else could be calculated locally by the client library, thus making the server-supported "current position" totally unnecessary. Do I miss anything? Could we agree on having only "row count" returned via op_info_cursor and leaving "cursor position" (getRow() in Java API) up the connectivity library developers? Looked at it again, and being able to get the total row count will work for me. Is this information already available, or does this still need to be implemented? One last thing I was wondering about (but haven't had time to verify): what happens with positional updates or deletes when scrolling? Specifically: - After an update: does the cursor have the original row content or the updated content? - After a delete: does the row disappear from the cursor, does the cursor retain the original content of the row, or does an 'empty' (e.g. all NULL) row exist in the cursor? Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Content for p_batch_create.p_batch_pb
On 2021-12-16 14:23, Alex Peshkoff via Firebird-devel wrote: On 12/16/21 15:12, Mark Rotteveel wrote: On 2021-12-16 10:28, Alex Peshkoff via Firebird-devel wrote: be segmented and swap bytes in segment length when needed. Afraid there is - as always :( - no protocol description except source code. Key ops are op_batch_msg, op_batch_blob_stream & also op_batch_cs (retuning batch execution state to client). That is not entirely true, there is [1] which was basically reverse-engineerd by Carlos Guzman Alvarez, and I updated some of it for protocol 11. Personally, I really don't like the current situation that the core developers don't document the actual protocol, because reverse engineering from the code and by using things like WireShark is extremely painful, confusing and error prone. With the batch API it's even worse IMHO, because even the usage examples seem to be incomplete, and too basic/simple and glosses over things. Mark [1]: https://firebirdsql.org/file/documentation/html/en/firebirddocs/wireprotocol/firebird-wire-protocol.html And where is the source of this docs? It's in the firebird-documentation project[1], specifically https://github.com/FirebirdSQL/firebird-documentation/blob/master/src/docs/asciidoc/en/firebirddocs/wireprotocol/firebird-wire-protocol.adoc Mark [1]: https://github.com/FirebirdSQL/firebird-documentation Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Content for p_batch_create.p_batch_pb
On 2021-12-16 10:28, Alex Peshkoff via Firebird-devel wrote: be segmented and swap bytes in segment length when needed. Afraid there is - as always :( - no protocol description except source code. Key ops are op_batch_msg, op_batch_blob_stream & also op_batch_cs (retuning batch execution state to client). That is not entirely true, there is [1] which was basically reverse-engineerd by Carlos Guzman Alvarez, and I updated some of it for protocol 11. Personally, I really don't like the current situation that the core developers don't document the actual protocol, because reverse engineering from the code and by using things like WireShark is extremely painful, confusing and error prone. With the batch API it's even worse IMHO, because even the usage examples seem to be incomplete, and too basic/simple and glosses over things. Mark [1]: https://firebirdsql.org/file/documentation/html/en/firebirddocs/wireprotocol/firebird-wire-protocol.html Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Fwd: [FirebirdSQL/firebird] Connection hangs after delivery of 2**32 - 1 packets (Issue #7065)
I strongly disagree with the chosen fix to make the counter size configurable. ChaCha20 is standardized in RFC-7539 with a 32-bit counter size[1]. Making the counter size configurable has two problems: 1) It is harder to support (as non-standard forms of ChaCha are not always available) 2) The client has no way to know which counter variant the server expects, and this needs to be explicitly configured both by the client and the server, which is really not ideal, and will lead to hard to diagnose connection problems. The proper way to fix this is to define a separate encryption plugin name for the variant with a 64-bit counter, so that client and server can negotiate the appropriate plugin that is supported. Alternatively, re-keying could be supported, so that client and server can change keys during a connection, but this comes with additional challenges. Mark [1]: https://datatracker.ietf.org/doc/html/rfc7539#section-2.4 Original Message Subject: [FirebirdSQL/firebird] Connection hangs after delivery of 2**32 - 1 packets (Issue #7065) Date: 2021-12-12 18:26 From: Alexander Peshkov To: FirebirdSQL/firebird Cc: Subscribed Reply-To: FirebirdSQL/firebird ChaCha wire encryption, used by default since FB4, is using 32-bit counter. When counter overflows secure packets delivery becomes impossible without reconnect. -- You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. Triage notifications on the go with GitHub Mobile for iOS [3] or Android [4]. Links: -- [1] https://github.com/FirebirdSQL/firebird/issues/7065 [2] https://github.com/notifications/unsubscribe-auth/ABI2Z4J2M42KMEBDP2KTWQDUQTLNJANCNFSM5J4PIROA [3] https://apps.apple.com/app/apple-store/id1477376905?ct=notification-emailmt=8pt=524675 [4] https://play.google.com/store/apps/details?id=com.github.androidreferrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors
On 2021-12-08 10:13, Dmitry Yemanov wrote: 28.11.2021 14:45, Mark Rotteveel wrote: We don't have anything like this. Theoretically, we could extend IRecordSet with something similar (although it would also require a protocol change), but the question is whether it's really needed. Personally, I don't see it useful per se. If users want to know a number of rows, then perhaps an explicit getRowCount() method would be more useful (*). But AFAIU the Java API does not mention it. (*) not applicable to uni-directional cursors? The intent of the method is to report the position of the current row. The example it is abused by some to get the total size was just an illustration of why people would expect a value after requesting the last row. But the same would be a problem if people went to last, scrolled around and then want to know the current row position. I was thinking about providing both "current position" and "row count" information values for a cursor. Both are doable, however the former has some issues: 1) Adjustments due to prefetch should be taken into account at both client and server sides, thus complicating the code (information response buffer must be parsed, value received from the engine/server must be replaced with the adjusted one). 2) If used "with purpose", it may be requested by the client after every fetch, thus multiplying the round-trips and killing the prefetch benefits. So it could make sense to request the position (send op_info_cursor) internally after every op_fetch_scroll packet. As every fetch request may return multiple rows, sending many positions is unnecessary and only position of the first row could be returned. This puts additional requirements to the client-side protocol implementations. 3) "row count" makes it possible to know the position after fetchLast() and everything else could be calculated locally by the client library, thus making the server-supported "current position" totally unnecessary. Do I miss anything? Could we agree on having only "row count" returned via op_info_cursor and leaving "cursor position" (getRow() in Java API) up the connectivity library developers? That sounds doable, but I won't have time to really look and think about this until the weekend. I'll reply then if I have more concerns. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Protocol 15
On 2021-12-06 19:10, Jiří Činčura wrote: IIRC, the only real change in protocol 14 was Yeah. Not really worth the effort I meant, it will be "solved" by 15 implementation. Yeah, I did the same in Jaybird: skipped implementation of protocol 14. Mark Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Protocol 15
On 06-12-2021 15:12, Jiří Činčura wrote: In a nutshell, what changed in protocol 15 compared to 13? Is it only the early crypt key callback? And I suppose there's not much into protocol 14, right? IIRC, the only real change in protocol 14 was the inclusion of the int32 p_cc_reply in the op_crypt_key_callback. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors
On 01-12-2021 16:06, Dmitry Yemanov wrote: 01.12.2021 16:07, Dimitry Sibiryakov wrote: Also if IResultSet is returned from IAttachment::openCursor() there is no (visible) IStatement at all. And I see the same problem for setCursorName() which is available only through IStatement. An oversight? You need to set the cursor name before you open it, IIRC. So - assuming I'm not mistaken in this - adding it to IResultSet doesn't make sense, as that will only return once the cursor has been opened. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] WNET future
On 30-11-2021 15:26, Adriano dos Santos Fernandes wrote: On 30/11/2021 10:24, Jiří Činčura wrote: My vote goes to dropping it. Agreed. I have no opinion on this. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors
On 28-11-2021 12:55, Dimitry Sibiryakov wrote: Dmitry Yemanov wrote 28.11.2021 12:51: 28.11.2021 14:47, Dimitry Sibiryakov wrote: RDB$DB_KEY for the current record as well. It's not a property of the cursor. Consider joins, unions, procedures, views, etc. It is a property of a current record the same as position. It is ok to return wide concatenated DB_KEY (up to 255 bytes due limitation of info buffer format). The statement info uses 16-bit size integers, so generally it would 65535, but AFAIK, it's contextually sized (e.g. isc_info_sql_select and isc_info_sql_bind don't have their own size). Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel