Re: [Firebird-devel] Build firebird 2.5 problem

2023-08-05 Thread Mark Rotteveel via Firebird-devel

Hi Hugo,

You sent this message to the old list at SourceForge. Please post to the 
firebird-devel Google Group (https://groups.google.com/g/firebird-devel, 
or firebird-de...@googlegroups.com).


Mark

On 05-08-2023 13:58, Hugo Larson via Firebird-devel wrote:

Hello,

I'm stuck for a while with FB 2.5 and need to build it on aarch64 Linux 
and getting the error below.
I know too little about building and am using docker 
https://github.com/jacobalberty/firebird-docker/blob/2.5-sc/Dockerfile 
<https://github.com/jacobalberty/firebird-docker/blob/2.5-sc/Dockerfile>


Thanks,
Hugo.


100 12.9M  100 12.9M    0     0  14.1M      0 --:--:-- --:--:-- --:--:-- 
40.9M

66.85 checking whether make sets $(MAKE)... yes
66.86 checking build system type... builds/make.new/config/config.guess: 
unable to guess system type

66.96
66.96 This script, last modified 2005-12-23, has failed to recognize
66.96 the operating system you are using. It is advised that you
66.96 download the most up to date version of the config scripts from
66.96
66.96  
  http://savannah.gnu.org/cgi-bin/viewcvs/*checkout*/config/config/config.guess

66.96 and
66.96  
  http://savannah.gnu.org/cgi-bin/viewcvs/*checkout*/config/config/config.sub

66.96
66.96 If the version you run (builds/make.new/config/config.guess) is 
already up to date, please

66.96 send the following data and any information you think might be
66.96 pertinent to  in order to provide the needed
66.96 information to handle your system.
66.96
66.96 config.guess timestamp = 2005-12-23
66.96
66.96 uname -m = aarch64
66.96 uname -r = 5.15.0-1039-oracle
66.96 uname -s = Linux
66.96 uname -v = #45-Ubuntu SMP Thu Jul 13 19:41:22 UTC 2023
66.96
66.96 /usr/bin/uname -p = aarch64
66.96 /bin/uname -X     =
66.96
66.96 hostinfo               =
66.96 /bin/universe          =
66.96 /usr/bin/arch -k       =
66.96 /bin/arch              = aarch64
66.96 /usr/bin/oslevel       =
66.96 /usr/convex/getsysinfo =
66.96
66.96 UNAME_MACHINE = aarch64
66.96 UNAME_RELEASE = 5.15.0-1039-oracle
66.96 UNAME_SYSTEM  = Linux
66.96 UNAME_VERSION = #45-Ubuntu SMP Thu Jul 13 19:41:22 UTC 2023
66.96 configure: error: cannot guess build type; you must specify one



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] nullOffset on vacation ?

2022-09-25 Thread Mark Rotteveel

Hi Norbert,

We're moving the list. Please subscribe to the firebird-devel Google 
Group and post your question there (once subscribed, you can use 
firebird-de...@googlegroups.com or post through the website).


See also the message with subject "[Firebird-devel] ACTION REQUIRED: 
This list is moving to Google Groups, please resubscribe on the new 
location" for details.


Mark

On 25-09-2022 12:48, Norbert Saint Georges wrote:

Hi,

FB4
iso8859_1

With a varchar(32765) field, the API returns a nullOffset outside the 
buffer length and the next field gives me an offset identical to the 
nullOffset of the previous field.


Did I miss something?



--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] ACTION REQUIRED: This list is moving to Google Groups, please resubscribe on the new location

2022-09-25 Thread Mark Rotteveel

On 25-09-2022 11:29, Mark Rotteveel wrote:

On 25-09-2022 11:19, Dimitry Sibiryakov via Firebird-devel wrote:

Mark Rotteveel wrote 25.09.2022 11:15:

This list is moving to Google Groups, and we'd like you to move with it.


   "Join" button in your invitation sends to 404 page.



Do you mean the invite I sent you directly from Google Groups? That is 
odd as that is generated by Google itself. Maybe replying to the message 
works as well? Otherwise, I suggest subscribing on 
https://groups.google.com/g/firebird-devel or by sending an email to 
firebird-devel+subscr...@googlegroups.com


It looks like the email generated when inviting people through the admin 
interface contains links that only work when you're logged in to Google. 
If you're not logged in, it generates a 404 :(


In any case, above alternatives should work.

Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] ACTION REQUIRED: This list is moving to Google Groups, please resubscribe on the new location

2022-09-25 Thread Mark Rotteveel

On 25-09-2022 11:19, Dimitry Sibiryakov via Firebird-devel wrote:

Mark Rotteveel wrote 25.09.2022 11:15:

This list is moving to Google Groups, and we'd like you to move with it.


   "Join" button in your invitation sends to 404 page.



Do you mean the invite I sent you directly from Google Groups? That is 
odd as that is generated by Google itself. Maybe replying to the message 
works as well? Otherwise, I suggest subscribing on 
https://groups.google.com/g/firebird-devel or by sending an email to 
firebird-devel+subscr...@googlegroups.com


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] ACTION REQUIRED: This list is moving to Google Groups, please resubscribe on the new location

2022-09-25 Thread Mark Rotteveel

Hello everyone,

This list is moving to Google Groups, and we'd like you to move with it.

To do this, subscribe at our new location: 
https://groups.google.com/g/firebird-devel, or by sending an email to 
firebird-devel+subscr...@googlegroups.com


Afterwards, we suggest you unsubscribe from the old list. You can do 
this at https://lists.sourceforge.net/lists/options/firebird-devel, or 
be sending an email to firebird-devel-requ...@lists.sourceforge.net with 
subject: unsubscribe


NOTE: After subscribing on Google Groups, messages may be moderated, so 
please be patient if your message doesn't show up immediately. We'll try 
to manually clear moderation status for known subscribers.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Test (please ignore)

2022-09-21 Thread Mark Rotteveel

Test (please ignore)
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath

2022-09-15 Thread Mark Rotteveel

On 15-09-2022 15:19, Alex Peshkoff via Firebird-devel wrote:

On 9/15/22 14:36, Adriano dos Santos Fernandes wrote:

On 15/09/2022 08:18, Alex Peshkoff via Firebird-devel wrote:

This is solved by install script - it creates symlink to

/inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory
(provided it's missing).


If we decide to _always_ provide own dynamic libraries (logically it's 
like using static one but does not require duplicating code) this trick 
becomes not needed any more.


Let me remind you that this thread started with me noticing an 
inconsistency in how we build things when explicitly building with 
--with-builtin-tommath compared to the - in my opinion - more sane 
behaviour of --with-builtin-tomcrypt (putting things in 
gen/buildroot/usr/local/firebird/lib/.tm vs in 
gen/buildroot/usr/local/firebird/lib).


I was not asking about always using our own libraries, but at least 
doing things in a sane way when a build is performed explicitly 
requesting to use our own version of libraries.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath

2022-09-15 Thread Mark Rotteveel

On 15-09-2022 13:13, Adriano dos Santos Fernandes wrote:

On 15/09/2022 07:52, Mark Rotteveel wrote:

On 15-09-2022 03:52, Adriano dos Santos Fernandes wrote:

On 05/09/2022 08:13, Mark Rotteveel wrote:

That is not really embedded IMHO. Firebird Embedded with Firebird 3.0
has become unwieldy and even harder to use than earlier versions,
especially if you compare it to other embedded database systems.


How it become harder?


Basically, in Firebird 2.5 and earlier, you needed fbembed.dll /
libfbembed.so and the intl folder, and you were done. Now you need a
whole constellation of files.



Despite the problem with system libraries, which in many cases we could
use static versions (I believe we cannot only with ICU), dealing with
dozen of files has the same complexity of dealing with 2.


In some ways yes, in others not.


Other embedded database are basically on DLL/.so and you're done.



Other that are *embedded only* or that can also be used (and focus on
it) as a server like Firebird?


Embedded only.

Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath

2022-09-15 Thread Mark Rotteveel

On 15-09-2022 03:52, Adriano dos Santos Fernandes wrote:

On 05/09/2022 08:13, Mark Rotteveel wrote:

That is not really embedded IMHO. Firebird Embedded with Firebird 3.0
has become unwieldy and even harder to use than earlier versions,
especially if you compare it to other embedded database systems.


How it become harder?


Basically, in Firebird 2.5 and earlier, you needed fbembed.dll / 
libfbembed.so and the intl folder, and you were done. Now you need a 
whole constellation of files.


Other embedded database are basically on DLL/.so and you're done.

Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath

2022-09-05 Thread Mark Rotteveel

On 05-09-2022 13:00, Alex Peshkoff via Firebird-devel wrote:
When adding it I cared about an ability to fix secirtiy issues in 
3d-party libraries not rebuilding and reinstalling FB packages. With 
used schema one can simply install fixed system package for tommath in a 
case of some issues in it. None were found - but that was when tommath 
was at pre-release stage.


I find these inconsistencies quite confusing, but lets leave it at that.

BTW, we always recommended to those who need embedded-only access full 
install (classic) and server stop.


That is not really embedded IMHO. Firebird Embedded with Firebird 3.0 
has become unwieldy and even harder to use than earlier versions, 
especially if you compare it to other embedded database systems.


In any case, this solution would be for Java applications that want a 
Firebird Embedded without having to install Firebird.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath

2022-09-05 Thread Mark Rotteveel

On 05-09-2022 12:06, Alex Peshkoff via Firebird-devel wrote:

On 9/5/22 12:59, Mark Rotteveel wrote:

On 05-09-2022 11:50, Alex Peshkoff via Firebird-devel wrote:

On 9/5/22 12:31, Mark Rotteveel wrote:
That is inconsistent compared to what is done for 
--with-builtin-tomcrypt. I explicitly requested 
--with-builtin-tommath, so I expect it to end up directly in the lib 
directory like tomcrypt.


This switch works another way.


It seems to me it does the same thing, but for a different library, so 
why does it do so in an entirely different way?


I do not know what library do you talk about.
There are many of them - and rules for them vary slightly.


I explicitly mention it multiple times (see also above): tomcrypt. The 
behaviour for --with-builtin-tomcrypt (putting it directly in lib) seems 
more sane to me compared to what is done for --with-builtin-tommath 
(putting it in lib/.tm). Why is it handled differently?


Yes, you need to change something not suitable for embedded, I even 
expect you to add configure switch for it - we never deployed FB for 
linux w/o install script.


We'll see, if I can find the motivation and energy to figure this out.



If you just shell scripts and send me changes I can do the rest myself...


That is part of the figuring out. I don't normally write shell scripts 
either, but before that I would also need to figure out exactly what I 
need to do before I can even start on that.


And since I consider this packaging Firebird embedded in a JAR for easy 
use from Java just a "nice to have", I will shelve this until I have 
more energy to tackle this.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath

2022-09-05 Thread Mark Rotteveel

On 05-09-2022 11:50, Alex Peshkoff via Firebird-devel wrote:

On 9/5/22 12:31, Mark Rotteveel wrote:

On 05-09-2022 11:23, Alex Peshkoff via Firebird-devel wrote:
This is solved by install script - it creates symlink to 
/inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory 
(provided it's missing).
That's done this way in order to make it possible to use system 
tommath when present.


That is inconsistent compared to what is done for 
--with-builtin-tomcrypt. I explicitly requested 
--with-builtin-tommath, so I expect it to end up directly in the lib 
directory like tomcrypt.


This switch works another way.


It seems to me it does the same thing, but for a different library, so 
why does it do so in an entirely different way?


If I understand you correctly, I will need to massage what I have in 
buildroot even further to get a usable build. I won't be using the 
install script, because I don't want to install it, I want to package 
it for an embedded deployment.


Yes, you need to change something not suitable for embedded, I even 
expect you to add configure switch for it - we never deployed FB for 
linux w/o install script.


We'll see, if I can find the motivation and energy to figure this out.

Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Building Firebird Linux with --with-builtin-tommath

2022-09-05 Thread Mark Rotteveel

On 05-09-2022 11:23, Alex Peshkoff via Firebird-devel wrote:
This is solved by install script - it creates symlink to 
/inst/path/firebird/lib/.tm/libtommath.so.0 in system lib directory 
(provided it's missing).
That's done this way in order to make it possible to use system tommath 
when present.


That is inconsistent compared to what is done for 
--with-builtin-tomcrypt. I explicitly requested --with-builtin-tommath, 
so I expect it to end up directly in the lib directory like tomcrypt.


If I understand you correctly, I will need to massage what I have in 
buildroot even further to get a usable build. I won't be using the 
install script, because I don't want to install it, I want to package it 
for an embedded deployment.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Building Firebird Linux with --with-builtin-tommath

2022-09-03 Thread Mark Rotteveel
I'm investigating ways of packaging Firebird Embedded for Java, so it 
can be used as drop-in Java library (which is then unpacked to a 
temporary directory automatically).


Given Firebird complains about missing libtommath when I use the normal 
build, and I don't want to impose on users to install additional 
libraries, I tried a custom build (from current state of v4.0-release 
branch) with:


./autogen.sh --with-builtin-tommath --with-builtin-tomcrypt 
--enable-binreloc

make
make dist

I have a problem with libtommath in this build:

gen/buildroot/usr/local/firebird/lib contains libtomcrypt.so.1.0.1 (and 
symlinks libtomcrypt.so and libtomcrypt.so.1), but no libtommath files, 
these are in a .tm subdirectory of lib, so


gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so.0
gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so.0.0.41
gen/buildroot/usr/local/firebird/lib/.tm/libtommath.so

I would have expected these files directly in 
gen/buildroot/usr/local/firebird/lib/, not in 
gen/buildroot/usr/local/firebird/lib/.tm/. Trying to load embedded this 
way complains about missing libtommath.


Copying those files/symlinks from .tm to the parent directory allows 
embedded to be loaded.


What is wrong here? Did I miss a configuration option somewhere? Is this 
a bug in the build?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Improve release filenames

2022-08-25 Thread Mark Rotteveel

On 25-08-2022 15:13, Stefan Heymann wrote:

I propose this name convention starting with Firebird 5:



Firebird-5.0.0.2816-0-windows-x86-pdb.exe
Firebird-5.0.0.2816-0-windows-x86-pdb.zip
Firebird-5.0.0.2816-0-windows-x86.exe
Firebird-5.0.0.2816-0-windows-x86.zip
Firebird-5.0.0.2816-0-windows-x64-pdb.exe
Firebird-5.0.0.2816-0-windows-x64-pdb.zip
Firebird-5.0.0.2816-0-windows-x64.exe
Firebird-5.0.0.2816-0-windows-x64.zip
[...]


I'd like to suggest you use win32 and win64 instead of x86 and x64. Or, it you want to 
keep the "windows" prefix, use window-32 and windows-64.

For me as a Windows user this would be much more logical (the 86 comes from the 
80*86 processor, the 64 comes from 64 bits, so it's not consistent from the 
beginning).


The proposed naming has the benefit it is consistent across platforms. 
That is:

Firebird---[-qualifier].

I don't see the need, nor the point to deviate from this for Windows.

Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Improve release filenames

2022-08-25 Thread Mark Rotteveel

On 25-08-2022 12:38, Dimitry Sibiryakov wrote:

Dmitry Yemanov wrote 25.08.2022 12:34:

Firebird-5.0.0.2816-0-windows-x86-withDebugSymbols.exe
Firebird-5.0.0.2816-0-windows-x86-withDebugSymbols.zip
Firebird-5.0.0.2816-0-windows-x86.exe
Firebird-5.0.0.2816-0-windows-x86.zip
Firebird-5.0.0.2816-0-windows-x64-withDebugSymbols.exe
Firebird-5.0.0.2816-0-windows-x64-withDebugSymbols.zip
Firebird-5.0.0.2816-0-windows-x64.exe
Firebird-5.0.0.2816-0-windows-x64.zip
Firebird-5.0.0.2816-0-linux-x64.tar.gz
Firebird-5.0.0.2816-0-linux-x64-debugSymbols.tar.gz
Firebird-5.0.0.2816-0-android-arm.tar.gz (armv7, other?)
Firebird-5.0.0.2816-0-android-arm-withDebugSymbols.tar.gz
Firebird-5.0.0.2816-0-android-arm64.tar.gz
Firebird-5.0.0.2816-0-android-arm64-withDebugSymbols.tar.gz
Firebird-5.0.0.2816-0-linux-x86.tar.gz
Firebird-5.0.0.2816-0-linux-x86-debugSymbols.tar.gz
Firebird-5.0.0.2816-0-source.tar.xz
Firebird-5.0.0.2816-0-macos-x64.pkg


Looks good to me.


   Is this zero after build number necessary?


We have had -1 releases, e.g. 
https://firebirdsql.org/en/firebird-4-0-0/#Win32 
(Firebird-4.0.0.2496-1-Win32.exe), so for consistency, either it should 
always be included, or we should ditch this package revision (or 
whatever we should call it) entirely.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Improve release filenames

2022-08-25 Thread Mark Rotteveel

On 25-08-2022 03:15, Adriano dos Santos Fernandes wrote:

On 24/08/2022 08:45, Adriano dos Santos Fernandes wrote:


I propose this name convention starting with Firebird 5:



Here is my updated proposal based on the discussion so far:

Firebird-5.0.0.2816-0-windows-x86-withDebugSymbols.exe
Firebird-5.0.0.2816-0-windows-x86-withDebugSymbols.zip
Firebird-5.0.0.2816-0-windows-x86.exe
Firebird-5.0.0.2816-0-windows-x86.zip
Firebird-5.0.0.2816-0-windows-x64-withDebugSymbols.exe
Firebird-5.0.0.2816-0-windows-x64-withDebugSymbols.zip
Firebird-5.0.0.2816-0-windows-x64.exe
Firebird-5.0.0.2816-0-windows-x64.zip
Firebird-5.0.0.2816-0-linux-x64.tar.gz
Firebird-5.0.0.2816-0-linux-x64-debugSymbols.tar.gz
Firebird-5.0.0.2816-0-android-arm.tar.gz (armv7, other?)
Firebird-5.0.0.2816-0-android-arm-withDebugSymbols.tar.gz
Firebird-5.0.0.2816-0-android-arm64.tar.gz
Firebird-5.0.0.2816-0-android-arm64-withDebugSymbols.tar.gz
Firebird-5.0.0.2816-0-linux-x86.tar.gz
Firebird-5.0.0.2816-0-linux-x86-debugSymbols.tar.gz
Firebird-5.0.0.2816-0-source.tar.xz
Firebird-5.0.0.2816-0-macos-x64.pkg


Sounds OK. On suggestion: use "with-debug-symbols" and "debug-symbols", 
but that is just because I'm a bit allergic to capitals in filenames ;)


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Improve release filenames

2022-08-24 Thread Mark Rotteveel

On 24-08-2022 17:00, Alex Peshkoff via Firebird-devel wrote:

On 8/24/22 17:10, Dimitry Sibiryakov wrote:

Alex Peshkoff via Firebird-devel wrote 24.08.2022 16:06:

Same for linux - not stripped binaries provided.


  Nope. Firebird-debuginfo-4.0.2.2816-0.amd64.tar.gz contains only 
extracted debug symbols, not binaries.




Sorry, but they are created by 'cp' command ;) One can run them directly.


Nonetheless, those Linux archives are not complete builds.

Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Improve release filenames

2022-08-24 Thread Mark Rotteveel

On 24-08-2022 16:06, Alex Peshkoff via Firebird-devel wrote:

On 8/24/22 17:04, Mark Rotteveel wrote:

On 24-08-2022 15:36, Dimitry Sibiryakov wrote:

Mark Rotteveel wrote 24.08.2022 15:32:
If the term is confusing or ambiguous, it already is so in its 
current form.


   Yes, it is. That's why I would suggest to change that.


The Windows pdb packages are complete builds though. 


Same for linux - not stripped binaries provided.


The Linux builds are not complete builds compared to Windows pdb builds 
(e.g. config files, documentation, security database, etc are missing)


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Improve release filenames

2022-08-24 Thread Mark Rotteveel

On 24-08-2022 15:36, Dimitry Sibiryakov wrote:

Mark Rotteveel wrote 24.08.2022 15:32:
If the term is confusing or ambiguous, it already is so in its current 
form.


   Yes, it is. That's why I would suggest to change that.


The Windows pdb packages are complete builds though. So maybe those 
should be "debug", while the Linux builds should be "debug-symbols" or 
something like that.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Improve release filenames

2022-08-24 Thread Mark Rotteveel

On 24-08-2022 14:50, Dimitry Sibiryakov wrote:

Dmitry Yemanov wrote 24.08.2022 14:47:

I would unify the `pdb`, `debuginfo`, etc. into simple `debug` suffix.


Agreed.


   And better to keep "debuginfo" verb, IMHO to prevent confusion after 
expectation to find "a debug build" in "debug" package.


What confusion, debuginfo packages are already listed as a "Debug build" 
on the download page. If the term is confusing or ambiguous, it already 
is so in its current form. In other words, consolidating it under the 
term "debug" shouldn't be a problem.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Improve release filenames

2022-08-24 Thread Mark Rotteveel

On 24-08-2022 13:45, Adriano dos Santos Fernandes wrote:

This is our Firebird 4.0.2 released files:


[..]

It's difficult to understand, they do not mention OS and hardware
architectures are inconsistent.

I propose this name convention starting with Firebird 5:

[..]

Firebird-5.0.0.2816-0-linux-x64.tar.gz
Firebird-5.0.0.2816-0-linux-x64-debuginfo.tar.gz
Firebird-5.0.0.2816-0-linux-arm.tar.gz (armv7, other?)
Firebird-5.0.0.2816-0-linux-arm-debuginfo.tar.gz
Firebird-5.0.0.2816-0-linux-arm64.tar.gz
Firebird-5.0.0.2816-0-linux-arm64-debuginfo.tar.gz

[..]

In general I'm OK with this, but currently our download page claims the 
ARM32 and ARM64 build are for Android, and not generically Linux.


So, are they actually generically Linux, or specifically for Android?

Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Purpose of op_batch_sync

2022-08-19 Thread Mark Rotteveel

On 19-08-2022 17:30, Alex Peshkoff via Firebird-devel wrote:

On 8/19/22 18:24, Mark Rotteveel wrote:
It looks like op_batch_sync is just used to force the client to send 
the deferred messages to the server, and the server to send the 
deferred responses. Is that correct?




exactly


So, just for confirmation, using op_ping (which I found earlier as a 
trick to get the server to send deferred messages) is a similar solution?


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Purpose of op_batch_sync

2022-08-19 Thread Mark Rotteveel
It looks like op_batch_sync is just used to force the client to send the 
deferred messages to the server, and the server to send the deferred 
responses. Is that correct?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Build master branch with VS 2019

2022-08-19 Thread Mark Rotteveel

On 19-08-2022 14:04, Gabor Boros wrote:
In output_x64_release directory I have common_test.exe and 
engine_test.exe. Is it normal? I cannot check the snapshot build because 
"Connection timed out" 
(http://web.firebirdsql.org/download/snapshot_builds/win/5.0).


You can also download snapshots from 
https://github.com/FirebirdSQL/firebird/actions/workflows/main.yml (e.g. 
use one built from master)


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Build master branch with VS 2019

2022-08-19 Thread Mark Rotteveel

On 19-08-2022 12:31, Vlad Khorsun wrote:
   At last, ensure your .bat files contains Windows-style EOL's (crlf), 
not Unix style.


The repository should contain a .gitattributes file that configures 
this. I was a bit surprised to see use of unix2dos in the GitHub actions 
workflow to ensure this for the batch files.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Named parameters in client libraries

2022-08-19 Thread Mark Rotteveel

On 19-08-2022 12:05, Adriano dos Santos Fernandes wrote:

You prefer large over smaller VARCHARs, prefer TIMESTAMP over TIME -
widest types

But prefer TIMESTAMP and NUMBERS over VARCHAR - most specific

I see no logic in your approach.


Most specific in the sense that a datetime type is more specific than a 
string type, for the TIME+TIMESTAMP+VARCHAR example: a TIMESTAMP value 
contains TIME, so covers both TIME and TIMESTAMP, and a TIMESTAMP is 
convertible to string, so can also be used with VARCHAR. On the other 
hand, not all string values are convertible to a TIME or TIMESTAMP value.


On the other hand, maybe that could lead to other weird issues (e.g. 
consider a query TIME+VARCHAR that works fine, but stops working when 
TIMESTAMP is added in the mix.


So, maybe using VARCHAR instead for those cases is the least intrusive 
alternative after all :(


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Build master branch with VS 2019

2022-08-19 Thread Mark Rotteveel

On 19-08-2022 10:20, Gabor Boros wrote:

2022.07.03. 12:43 keltezéssel, Gabor Boros írta:
I tried to build master branch with VS 2019 Community 2019 16.11.16 on 
an up to date Windows 10 Pro 64bit.


At make_boot got fatal errors in isql_x64.log:



Same errors with current state of master branch.

Any idea?


Have you ever been able to build Firebird locally?

I'm not sure if it is still used, but in the past the build required GNU 
sed, which isn't available by default, and you would need to install it 
(e.g. from http://gnuwin32.sourceforge.net/packages/sed.htm). Also, 
given the errors reference the `gen` path, it is likely the preprocess 
phase failed, and this is just a subsequent error.


That said, you probably need one of the core developers to respond to 
diagnose this better.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Named parameters in client libraries

2022-08-19 Thread Mark Rotteveel

On 19-08-2022 02:36, Adriano dos Santos Fernandes wrote:

On 18/08/2022 04:37, Mark Rotteveel wrote:

In other words, most specific type with the highest
precision/scale/length. When confronted with combinations of non-string
types and string/blob types, use the most specific non-string type.


I don't think this is correct, specially the ones I listed above.

1: I would define it as VARCHAR with size using same rules of COALESCE
with both types

5: same case as 1, VARCHAR

12: should be error, the two types are incompatible


They shouldn't be according to section 4.6 of the SQL standard, 
specifically Table 3 — Datetime data type conversions.



So it would be something like COALESCE.


I think that is the wrong approach for parameters. When deriving a type 
for COALESCE, it makes sense to infer the type to the widest possible 
type (VARCHAR) to allow all possible stored values to be represented, 
but with parameters, it is IMHO better to use the most specific type, as 
that will yield useful information to the user through the metadata, and 
probably result in the best diagnostic information when specifying 
invalid values.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Named parameters in client libraries

2022-08-18 Thread Mark Rotteveel

On 18-08-2022 04:02, Adriano dos Santos Fernandes wrote:

And what type (and the deduction rules) this parameter will be described as?


I'd suggest looking at what other database systems that do have named 
parameters currently do.


My guess is that they either:

1) don't allow mixing types
2) use the most specific type that is compatible with all positions, 
otherwise error

3) use the first position
4) use the last position

I think rule 2 is probably the best, but I'm not really sure.

Some examples for rule 2
1) TIME, TIMESTAMP, VARCHAR(..) => TIMESTAMP
2) CHAR(5), VARCHAR(50), VARCHAR(100) => VARCHAR(100)
3) INTEGER, BIGINT, DECFLOAT => DECFLOAT
4) INTEGER, DATE => error: incompatible types?
5) VARCHAR(50), DOUBLE PRECISION => DOUBLE PRECISION
6) INTEGER, DOUBLE PRECISION => DOUBLE PRECISION
7) NUMERIC(4,2), NUMERIC(38,6) => NUMERIC(38,6)
8) NUMERIC(4,1), NUMERIC(4,4) => NUMERIC(9,4) (note the change in 
precision to accommodate range of values)

9) SQL_NULL (e.g. :param IS NULL), INTEGER => INTEGER
10) VARCHAR(..), BLOB => BLOB
11) BLOB SUB_TYPE TEXT, BLOB SUB_TYPE BINARY => BLOB SUB_TYPE BINARY
12) TIME WITH TIME ZONE, TIMESTAMP WITHOUT TIME ZONE => TIMESTAMP WITH 
TIME ZONE

...

In other words, most specific type with the highest 
precision/scale/length. When confronted with combinations of non-string 
types and string/blob types, use the most specific non-string type.

--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Named parameters in client libraries

2022-08-18 Thread Mark Rotteveel

On 18-08-2022 04:02, Adriano dos Santos Fernandes wrote:

How are client libraries (Jaybird, .NET Provider, Delphi ones)
describing its named parameters to their users?

I mean, given this SQL:

select *
   from rdb$database
   where :param = 1 or :param = '2'

We have here single name used in context with multiple types.

I suppose you transform this to:

select *
   from rdb$database
   where ? = 1 or ? = '2'

Which will map to two Firebird parameters with different types.

But for the user of the library, I suppose it's one parameter, correct?

And what type (and the deduction rules) this parameter will be described as?


JDBC doesn't define support for named parameters in 
java.sql.PreparedStatement, only in java.sql.CallableStatement, which is 
for calling stored procedures, and that support is optional, so Jaybird 
doesn't have any support for named parameters.


So, if Firebird is going to implement real named parameters, Jaybird is 
probably not going to support them, and in that case I would really love 
if this is controlled through a DPB item so when disabled, use of a 
named parameter results in an error (e.g. "named parameters in DSQL are 
not enabled" or something). Maybe I can then allow users aware of named 
parameters to enable it explicitly to map it them themselves, for 
example if something like `column1 = :param1 and column2 = :param2 and 
colum3 = :param1` results in two parameters (param1 = position 1, and 
param2 = position 2), and setting by position still works.


The reason Jaybird probably is not going to support this, is because 
most people don't use the JDBC API directly, but through things like 
Hibernate, so adding vendor extensions to the JDBC API is pretty much 
wasted effort because those libraries only use the JDBC API. I could add 
such methods to the FirebirdPreparedStatement interface, but that would 
then be a niche feature for people who do use JDBC, and are able to 
unwrap to the FirebirdPreparedStatement interface (which is not always 
possible when obtained from a connection pool).


In the Java world, named parameters are usually emulated by layers like 
Hibernate, or Spring's NamedParameterJdbcTemplate, which will map names 
to positional parameters, and set the values of those positional 
parameters appropriately.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] op_que_events and database shutdown

2022-08-16 Thread Mark Rotteveel

On 16-08-2022 11:01, Vlad Khorsun wrote:

16.08.2022 11:26, Jiří Činčura wrote:

I just want to know
there's nothing on protocol level I can use to help this situation.


No response. So I suppose nothing?


   I didn't found anything suitable.


Are aux connections closed when a database is shutdown? In that case you 
should be able to detect the connection close.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] New statement: EXECUTE SQL

2022-08-16 Thread Mark Rotteveel

On 15-08-2022 19:42, Vlad Khorsun wrote:
   Also, I don't like 'sql' word, especially after 'execute statement' 
and 'execute
block'. Too much, as for me :) Syntax with 'with' instead of 'execute 
sql' looks
much better to me, but it is already used in CTE's, thus it seems as not 
the best

choice :(


Personally, I do like the EXECUTE SQL for its explicitness, and it fits 
in the EXECUTE PROCEDURE and EXECUTE BLOCK naming; it is slightly a 
mismatch/confusing with EXECUTE STATEMENT though.


An alternative could be to extend the EXECUTE BLOCK syntax for this so 
it has a simplified form, without a RETURNS clause.


So, it has two forms:

EXECUTE BLOCK [(, ...)]
  [RETURNS (, ...)]
AS
[]
BEGIN
  [ ...]
END

and the simplified form

EXECUTE BLOCK [(, ...)]]
AS
[]
DO
  

Syntactically, the simplified form could be distinguished by the DO 
instead of the BEGIN. Presence of the RETURNS clause with DO would raise 
an error (maybe easiest is to allow it in the parse.y syntax, but raise 
an error when its present).


A downside of this is that statement recognition by clients that 
currently just look for EXECUTE BLOCK might break.


Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] New statement: EXECUTE SQL

2022-08-16 Thread Mark Rotteveel

On 16-08-2022 03:25, Adriano dos Santos Fernandes wrote:

On 15/08/2022 14:42, Vlad Khorsun wrote:


   I like the idea but not syntax. As already mentioned, there it will be
hard for
app\component devs to parse the whole statement looking for parameters.
Note,
semicolon usually mark "client" named parameters and it will be near to
impossible
for, say, Delphi components to correctly preprocess statement like below:

execute sql (p1 integer = :p1)
do
   select * from t where t.id = :p1 and t.name = :p2



Client parsers already need to stop at some place.

But I understand what you mean, so just read below...




You may expect after preprocessing by app it will be like:

execute sql (p1 integer = ?)
do
   select * from t where t.id = :p1 and t.name = ?


but actually it will be like:

execute sql (p1 integer = ?)
do
   select * from t where t.id = ? and t.name = ?

without complex re-writing of existing preprocessors.



No, I expect that client parsers may not do unnecessary things, at least
with this statement.

We may report named parameters in the right way (currently EXECUTE BLOCK
does not report then, do not know why).

For the parsers using :name syntax, that's very easy. Just detect the
new statement and do not pre-parse it.

AFAIK .NET uses @name syntax. Since that is not a Firebird valid syntax,
it should also not be difficult to detect and replace it.

So this will not be processed by client at all. It will just read
parameter names (p1, p2) from the described statement:

execute sql (p1 integer = ?)
do
   select * from t where t.id = :p1 and t.name = :p2

In fact the ugly "= ?" becomes completely unnecessary and better removed
in this statement.

And it's up to us to continue allowing unnamed parameters in this
statement or not.

I think we should and people using client libraries that require named
(instead positional) parameters could just not use them.



Am I reading this correctly that your proposal is to use the following 
syntax?


execute sql (p1 integer, p2 varchar(50))
do
  select * from t where t.id = :p1 and t.name = :p2

So, no explicit positional parameter markers ('?')?

Mark
--
Mark Rotteveel



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Is there a way to get the server to send deferred messages?

2022-08-15 Thread Mark Rotteveel
Is there a way to get the server to send deferred messages without 
having to read an additional response? Messages like the response to 
op_batch_create are deferred, meaning you have to trigger a non-deferred 
action to get the response.


I tried to send op_dummy to the server, but that had no effect. I could 
send an op_ping or another simple request, but that requires consuming 
an additional response.


This is mostly for test purposes, so if using a ping or similar is the 
only way, then so be it, but if there is something I missed, then that 
would be appreciated.

--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Do repeated op_batch_create leak the batch?

2022-08-15 Thread Mark Rotteveel

On 15-08-2022 12:47, Alex Peshkoff via Firebird-devel wrote:

On 8/13/22 14:59, Mark Rotteveel wrote:
I'm implementing batch execution in Jaybird. Looking at the code of 
rem_port::batch_create(P_BATCH_CREATE* batch, PACKET* sendL) in 
server.cpp, it looks like sending multiple op_batch_create requests 
for the same statement handle could leak a previously created batch, 
as it doesn't call statement->rsr_batch->release() before assigning a 
new batch to statement->rsr_batch.


Is my assessment correct?



Seems to be so. In the engine there is a check avoiding opening batch 
when batch or cursor is already opened, sane should be added to remote 
server. Add a ticket please.


https://github.com/FirebirdSQL/firebird/issues/7262

--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] New statement: EXECUTE SQL

2022-08-15 Thread Mark Rotteveel

On 15-08-2022 12:40, Simonov Denis via Firebird-devel wrote:
I'm talking about the fact that EXECUTE STATEMENT supports named 
parameters, including those for EXECUTE BLOCK. There's even a special 
query preparser for that. Those who implement client libraries will have 
to do much the same. And if for EXECUTE BLOCK the prepaser is quite 
simple (named parameters need to be searched only between EXECUTE BLOCK 
and the first AS), then for such a syntax it will not be very trivial.


In your version, I don't see how even EXECUTE STATEMENT can work with 
named parameters.


The named parameters in the example are basically equivalent to stored 
procedure / execute block parameters (a.k.a. PSQL variables), and PSQL 
supports named parameters in SQL.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] New statement: EXECUTE SQL

2022-08-14 Thread Mark Rotteveel

On 14-08-2022 01:28, Adriano dos Santos Fernandes wrote:

Hi!

When one starts with a DSQL command and need to adapt it to EXECUTE 
BLOCK (for example to use sub routines or use a single parameter in many 
places), work is difficult when there are many parameters and output 
fields. Everything must be explicitly declared.


I propose new DSQL statement that improve a lot this workflow (and 
others when not all power of EXECUTE BLOCK is necessary, but it's 
verbosity is inevitable).


I'm calling it EXECUTE SQL, and it's to use with SELECT, UPDATE, DELETE 
and MERGE, with or without RETURNING. It seats between lack of resources 
+ simplicity of direct SQL command and power + verbosity of EXECUTE BLOCK.


Syntax:

execute sql [ (  ) ]
     [  ]
do 

Here is how it can be used:

execute sql (p1 integer = ?, p2 integer = ?)
     declare function subfunc (i1 integer) returns integer
     as
     begin
         return i1;
     end

     declare procedure subproc (i1 integer) returns (o1 integer)
     as
     begin
         o1 = i1;
         suspend;
     end
do
select subfunc(:p1) + o1
     from subproc(:p2 + ?)

Note that parameters may be declared or directly (only in the DO 
command) used like now.


Output is not declared. It's inferred from the DO command.

Statement type of the DO command is returned.


An interesting idea. I do find the ability to specify question mark 
parameters at the top and inside the body slightly confusing, but I 
understand why it is handy to allow this.


I do think this statement will complicate the parser I have in Jaybird 
to detect statement types and whether or not it has RETURNING, but I can 
adjust.


Would this still allow the full SELECT syntax (including WITH clauses, 
OFFSET/FETCH, GROUP BY, etc), and things like RETURNING for INSERT, etc?


I assume this is a toplevel statement only, so it can't occur as a 
derived table or subquery inside another statement. Is that correct?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Do repeated op_batch_create leak the batch?

2022-08-13 Thread Mark Rotteveel
I'm implementing batch execution in Jaybird. Looking at the code of 
rem_port::batch_create(P_BATCH_CREATE* batch, PACKET* sendL) in 
server.cpp, it looks like sending multiple op_batch_create requests for 
the same statement handle could leak a previously created batch, as it 
doesn't call statement->rsr_batch->release() before assigning a new 
batch to statement->rsr_batch.


Is my assessment correct?

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Parallel workers in isc_action_svc_repair + isc_spb_rpr_sweep_db

2022-08-12 Thread Mark Rotteveel

On 12-08-2022 15:13, Vlad Khorsun wrote:

12.08.2022 15:31, Jiří Činčura wrote:

Hi *,

does the isc_dpb_parallel_workers apply when running sweep via 
isc_action_svc_repair + isc_spb_rpr_sweep_db?


   Sure.


There is no isc_spb_parallel_workers, so I would think it doesn't apply 
to service actions. What am I missing?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] BLOB_APPEND and NULL

2022-08-11 Thread Mark Rotteveel

On 11-08-2022 18:09, Alex Peshkoff via Firebird-devel wrote:

On 8/11/22 19:04, Vlad Khorsun wrote:


  So far I don't see it as mistake.



+1
Function is anyway damned non-standard.
And as soon as we have documented behavior - no problems at all.


Time will tell. I think deviating NULL behaviour can lead to confusion, 
even if it is documented.


In any case, I updated the language reference (I used the release notes 
as a starting point):


https://www.firebirdsql.org/file/documentation/html/en/refdocs/fblangref40/firebird-40-language-reference.html#fblangref40-scalarfuncs-blob-append

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] BLOB_APPEND and NULL

2022-08-11 Thread Mark Rotteveel

On 11-08-2022 18:04, Vlad Khorsun wrote:

11.08.2022 17:46, Jiří Činčura wrote:
I was thinking the same when reading the discussion on GH. 


   There was a LOT of time to write something at that discussion.
Nobody asked about NULL's there, while it was documented since a very 
beginning.


Unfortunately I don't always have time or energy to monitor all GitHub 
updates, and there was no previous discussion on this list about 
BLOB_APPEND, so I simply had not noticed its existence before.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] BLOB_APPEND and NULL

2022-08-11 Thread Mark Rotteveel

On 11-08-2022 17:58, Vlad Khorsun wrote:

11.08.2022 17:26, Mark Rotteveel wrote:

On 11-08-2022 16:21, Vlad Khorsun wrote:

11.08.2022 17:10, Mark Rotteveel wrote:

Why was this NULL behaviour chosen?


   To make BLOB_APPEND more convenient for users.


I don't understand how using a different NULL behaviour then standard 
for operations in SQL/Firebird is convenient. 


   There is no standard defined operation for appending blobs, afaik.


I am talking about the general expectations raised by the SQL standard 
(and Firebird itself) regarding behaviour when faced with NULL, and that 
is, in general, that when one of the inputs of a function, operation, or 
other expression is NULL, the result is NULL.


And given concatenation is defined for BLOBs in the standard, with the 
normal NULL behaviour, I would expect this appending to follow the same 
basic rules.


To me it sounds like something that will result in confusion because 
of the difference with normal concatenation, but OK.


   The BLOB_APPEND is not CONCATENATION, it is non-standard function 
with custom semantics.


It is a non-standard function that is intended to be used in places 
where you'd otherwise use concatenation. To me that means semantics of 
concatenation should be applied.


Mark

--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] BLOB_APPEND and returned SUB_TYPE

2022-08-11 Thread Mark Rotteveel
I noticed that BLOB_APPEND always returns a blob of SUB_TYPE TEXT, even 
if the first blob is binary or other type of blob. Is that expected?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] BLOB_APPEND and NULL

2022-08-11 Thread Mark Rotteveel

On 11-08-2022 16:21, Vlad Khorsun wrote:

11.08.2022 17:10, Mark Rotteveel wrote:

Why was this NULL behaviour chosen?


   To make BLOB_APPEND more convenient for users.


I don't understand how using a different NULL behaviour then standard 
for operations in SQL/Firebird is convenient. To me it sounds like 
something that will result in confusion because of the difference with 
normal concatenation, but OK.


I had hoped for something more solid, so I could add that as an 
explanation when documenting it in the language reference.


PS You mailed this to me privately instead of to the list.

--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] BLOB_APPEND and NULL

2022-08-11 Thread Mark Rotteveel
The new BLOB_APPEND function has a NULL behaviour that is not consistent 
with the normal NULL behaviour. Using NULL in BLOB_APPEND behaves as an 
empty string, while the normal behaviour for functions and operations 
involving NULL is to result in NULL.


For example, normal concatenation:

select cast(null as blob sub_type text) || cast('a' as blob sub_type 
text) || cast(null as blob sub_type text) || cast('b' as blob sub_type 
text)  from rdb$database;


results in NULL, while BLOB_APPEND:

select blob_append(null, 'a', null, 'b') from rdb$database;

results in 'ab'.

If the "normal" NULL-behaviour had been used, it would be possible to 
easily rewrite normal concatenation involving blobs to BLOB_APPEND, but 
now that requires careful consideration. In addition, this closes an 
avenue for the engine to - at some point in the future, as an engine 
optimization - rewrite concatenation to BLOB_APPEND internally.


Why was this NULL behaviour chosen?

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Ability to use non-local protocol to create DB which alias is declared as self-security

2022-08-09 Thread Mark Rotteveel

On 09-08-2022 06:33, Alex Peshkoff via Firebird-devel wrote:
PPS. If one has access to database.conf to create new alias he 
definitely has embedded access to server, i.e. problem appears to be 
rather artificial.


Not necessarily. One could have access to the filesystem or even only 
the configuration files remotely (e.g. using some configuration 
management tool like Puppet), but not necessarily be able to run process 
on the host, or create databases.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Is batch release necessary in the wire protocol?

2022-07-26 Thread Mark Rotteveel

On 16-07-2022 10:54, Mark Rotteveel wrote:
Is sending op_batch_rls necessary in the wire protocol, or is closing 
the statement itself sufficient?


Similar question, is a batch automatically released when unpreparing the 
statement, or preparing a new statement text?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Wrong doc RDB$INDICES.RDB$INDEX_TYPE ?

2022-07-18 Thread Mark Rotteveel

On 18-07-2022 17:35, Ariel Álvarez wrote:

Great! But I see the table format broken in langref 3.0. Check again.


Aargh, thanks, and fixed :)

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Wrong doc RDB$INDICES.RDB$INDEX_TYPE ?

2022-07-18 Thread Mark Rotteveel

On 16-07-2022 11:15, Mark Rotteveel wrote:

On 14-07-2022 18:23, Ariel Álvarez wrote:

In language reference it is documented:
Table RDB$INDICES
Column RDB$INDEX_TYPE:
Distinguishes between an expression index (1) and a regular index (0 
or null). Not used in databases created before Firebird 2.0; hence, 
regular indexes in upgraded databases are more more likely to store 
null in this column


But, according to this discussion:
https://github.com/FirebirdSQL/firebird/issues/4085 
<https://github.com/FirebirdSQL/firebird/issues/4085>


RDB$INDEX_TYPE identifies a descending or ascending index.

It appears the doc is wrong


It looks like it. The error also occurs in The Firebird Book, Second 
Edition. I guess the error derives from there. The first edition of The 
Firebird Book says "Not currently used; likely to distinguish regular 
indexes from expression indexes when the feature is implemented", so I 
guess when expression indexes were added in Firebird 2, the assumption 
was codified in the book, and then copied when the Language Reference 
was written.


I'll double check this and fix the documentation.


I published a new version of the docs with this fixed.

Mark

--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Wrong doc RDB$INDICES.RDB$INDEX_TYPE ?

2022-07-16 Thread Mark Rotteveel

On 14-07-2022 18:23, Ariel Álvarez wrote:

In language reference it is documented:
Table RDB$INDICES
Column RDB$INDEX_TYPE:
Distinguishes between an expression index (1) and a regular index (0 or 
null). Not used in databases created before Firebird 2.0; hence, regular 
indexes in upgraded databases are more more likely to store null in this 
column


But, according to this discussion:
https://github.com/FirebirdSQL/firebird/issues/4085 
<https://github.com/FirebirdSQL/firebird/issues/4085>


RDB$INDEX_TYPE identifies a descending or ascending index.

It appears the doc is wrong


It looks like it. The error also occurs in The Firebird Book, Second 
Edition. I guess the error derives from there. The first edition of The 
Firebird Book says "Not currently used; likely to distinguish regular 
indexes from expression indexes when the feature is implemented", so I 
guess when expression indexes were added in Firebird 2, the assumption 
was codified in the book, and then copied when the Language Reference 
was written.


I'll double check this and fix the documentation.

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Is batch release necessary in the wire protocol?

2022-07-16 Thread Mark Rotteveel
Is sending op_batch_rls necessary in the wire protocol, or is closing 
the statement itself sufficient?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] More attention to preventing breaking changes in point releases

2022-07-13 Thread Mark Rotteveel
I think https://github.com/FirebirdSQL/firebird/issues/6987 shows that 
we should be more careful with introducing (or better, preventing) 
breaking changes in point releases.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] SIO_LOOPBACK_FAST_PATH deprecated

2022-07-10 Thread Mark Rotteveel

On 02-02-2021 00:45, Vlad Khorsun wrote:

02.02.2021 1:35, Leyne, Sean wrote:

https://www.gitmemory.com/issue/grpc/grpc/18057/486312183


There is no mention of SIO_LOOPBACK_FAST_PATH in this link


   Read carefully, please.


https://support.microsoft.com/en-us/topic/stop-error-0xd1-when-you-set-
the-sio-loopback-fast-path-flag-in-windows-8-or-windows-server-2012-
14399334-f3a8-b731-3799-12899a79bf35


The HotFix is from 2012, with the last update in 2015.  I think that 
the underlying issue has been resolved/mitigated.


   We can only guess. Therefore I don't offer to remove the feature, but 
consider

to deactivate it by default.


I want to revisit this discussion. I think that given Microsoft 
deprecated SIO_LOOPBACK_FAST_PATH and warns against it, we should either 
remove this entirely from Firebird 5.0, or at minimum disable it by default.


Thoughts?

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Add scroll fetch support in legacy API?

2022-06-04 Thread Mark Rotteveel

On 04-06-2022 15:17, Dmitry Yemanov wrote:

04.06.2022 15:07, Mark Rotteveel wrote:

Currently scroll fetch is only exposed in the OO API. Any plans for 
exposing it in the legacy API 


Nope, we have an agreement to not extending the legacy API with new 
features.


Ok, too bad.


(e.g. isc_dsql_fetch_scroll / fb_dsql_fetch_scroll)?


That's not enough, some method is also needed to mark the cursor as 
scrollable before opening. The new cursor's getInfo() should also be 
duplicated. And maybe isBof/isEof too.


Ah, right. Though isBof/isEof is technically not necessary. You can 
infer that from the status return value and the last fetch direction 
(fetching relative 0 when before-first requires some of your own 
bookkeeping though)


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Add scroll fetch support in legacy API?

2022-06-04 Thread Mark Rotteveel
Currently scroll fetch is only exposed in the OO API. Any plans for 
exposing it in the legacy API (e.g. isc_dsql_fetch_scroll / 
fb_dsql_fetch_scroll)?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Strange CAST result

2022-05-25 Thread Mark Rotteveel

On 25-05-2022 10:08, artyom.abakumov wrote:

The CAST function returns a strange result in the following case:

SQL> SELECT CAST(345.12e-2 as VARCHAR(10)) from RDB$DATABASE;

CAST
==
3.45

Is it a correct behavior?
LI-V4.0.2.2770 Firebird 4.0


It does seem off, I would expect it to output 3.4512, same as happens 
for cast(3.4512e0 as varchar(10)).


There are other oddities, e.g. cast(x as varchar(10)) where x is a 
literal gives weird results:



34512e-4: 3.4512000
3451.2e-3   : 3.5
34.512e-1   : 3.451
3.4512e0: 3.4512
0.34512e1   : 3.45120
0.034512e2  : 3.451200
0.0034512e3 : 3.4512000

These should basically be the same value

Also
select cast(x as varchar(10)) from (select cast(345.12e-2 as double 
precision) as x from rdb$database);


results in 3.3412000

while
select cast(x as varchar(10)) from (select 345.12e-2 as x from 
rdb$database);


results in 3.45, which is odd as 345.12e-2 is a double precision 
literal, but it looks like it gets coerced to something else in the 
presence of the cast to varchar.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Incorrect string right truncation errors in Firebird 5

2022-05-17 Thread Mark Rotteveel

On 17-05-2022 15:07, Adriano dos Santos Fernandes wrote:

On 17/05/2022 09:54, Mark Rotteveel wrote:


Thanks! I will change Jaybird to switch to using blr_varying2/blr_text2
for Jaybird 4.0.7 and Jaybird 5.



When you did it for test, did you used the charset number or CS_dynamic?

I'm asking because the current code seems to have the same problem if
test2/varying2 is used with CS_dynamic instead of the actual number.


It used the character set id of the column as obtained from statement 
information isc_info_sql_bind, isc_info_sql_describe_vars (specifically 
isc_info_sql_sub_type), which in the case of this test is 4 (UTF8).


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Incorrect string right truncation errors in Firebird 5

2022-05-17 Thread Mark Rotteveel

On 17-05-2022 14:47, Adriano dos Santos Fernandes wrote:

On 17/05/2022 08:30, Mark Rotteveel wrote:

Is this happening with fbclient library too?


Good question: no it doesn't. Which suggests Jaybird is doing something
different. Jaybird uses blr_varying/blr_text, not blr_varying2/blr_text2
when sending the BLR of the execute. Could that make a difference?

Changing Jaybird to write blr_varying2/blr_text2 with the sub type
(character set) fixes the issue.

However, I thought that using blr_varying/blr_text (judging by
MetadataFromBlr::MetadataFromBlr in
common\classes\InternalMessageBuffer.cpp, Firebird selects CS_dynamic,
which should use the connection character set (UTF8) in this case. Is it
possible that `CharSet* const fromCharSet = INTL_charset_lookup(tdbb,
from_cs);` doesn't handle CS_dynamic properly? Or that this happens in a
codepath where BLR decoding doesn't assign CS_dynamic, but it remains
zero (NONE)?



The engine is receiving the string using NONE charset, which makes
things bad there.

This happens due to this place:

-
// ASF: Older than 2.5 engine hasn't validating strings in DSQL. After
this has been
// implemented in 2.5, selecting a NONE column with UTF-8 attachment
charset started
// failing. The real problem is that the client encodes
SQL_TEXT/SQL_VARYING using
// blr_text/blr_varying (i.e. with the connection charset). I'm reseting
the charset
// here at the server as a way to make older (and not yet changed)
client work
// correctly.
if (desc.isText() && desc.getTextType() == ttype_dynamic)
desc.setTextType(ttype_none);
-

So it is happening for a long time and may make things incorrect in Jaybird.

Anyway, it seems I'll need to change the fix for #7179 to take care of
real NONE/BINARY source strings.


Thanks! I will change Jaybird to switch to using blr_varying2/blr_text2 
for Jaybird 4.0.7 and Jaybird 5.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Incorrect string right truncation errors in Firebird 5

2022-05-17 Thread Mark Rotteveel

On 16-05-2022 21:00, Adriano dos Santos Fernandes wrote:

On 16/05/2022 12:07, Mark Rotteveel wrote:

On 16-05-2022 16:50, Mark Rotteveel wrote:

I was running some tests against Firebird-5.0.0.494-0_x64 (latest
snapshot, from last Saturday), and I notice that I get incorrect
string right truncation errors with CHAR/VARCHAR.

I currently cannot dive deeper into it, but as a datapoint, the error
does not occur with Firebird-5.0.0.488-0_x64.

Based solely on the commit message, maybe this commit is at fault?
https://github.com/FirebirdSQL/firebird/commit/dd18a3b11b28c3ed8126a6f54b829989954bfa03



The Jaybird test that triggers this is
org.firebirdsql.jdbc.TestFBPreparedStatementUTF8, specifically tests:

- connectionUtf8_insertMultiByte_inWin1252_char_1_win1252_succeeds
(fails with "expected length 1, actual 2")

[..]

- connectionUtf8_insertMultiple_lessThanMax_varchar_5_utf8_succeeds
(fails with "expected length 5, actual 6")

For CHAR, it seems to count 1 more character, for VARCHAR 2-3 more
characters.



Is this happening with fbclient library too?


Good question: no it doesn't. Which suggests Jaybird is doing something 
different. Jaybird uses blr_varying/blr_text, not blr_varying2/blr_text2 
when sending the BLR of the execute. Could that make a difference?


Changing Jaybird to write blr_varying2/blr_text2 with the sub type 
(character set) fixes the issue.


However, I thought that using blr_varying/blr_text (judging by 
MetadataFromBlr::MetadataFromBlr in 
common\classes\InternalMessageBuffer.cpp, Firebird selects CS_dynamic, 
which should use the connection character set (UTF8) in this case. Is it 
possible that `CharSet* const fromCharSet = INTL_charset_lookup(tdbb, 
from_cs);` doesn't handle CS_dynamic properly? Or that this happens in a 
codepath where BLR decoding doesn't assign CS_dynamic, but it remains 
zero (NONE)?



Is the error in insert or select?


The error happens on insert.


I'm failing to reproduce it in isql, for example for
connectionUtf8_insertMultipleInWin1252_varchar_5_win1252_succeeds:

-
CREATE TABLE utf8table (
  id INTEGER,
  char_1_none CHAR(1) CHARACTER SET NONE,
  char_1_utf8 CHAR(1) CHARACTER SET UTF8,
  char_1_win1252 CHAR(1) CHARACTER SET WIN1252,
  char_5_none CHAR(5) CHARACTER SET NONE,
  char_5_utf8 CHAR(5) CHARACTER SET UTF8,
  char_5_win1252 CHAR(5) CHARACTER SET WIN1252,
  varchar_5_none VARCHAR(5) CHARACTER SET NONE,
  varchar_5_utf8 VARCHAR(5) CHARACTER SET UTF8,
  varchar_5_win1252 VARCHAR(5) CHARACTER SET WIN1252
);

/*
select unicode_char(0x00FE) || unicode_char(0x00A3) || 'a' ||
unicode_char(0x0160) || ',' from rdb$database;
*/

set bulk_insert INSERT INTO utf8table (id, char_5_win1252) VALUES (?, ?);
(1, 'þ£aŠ,')
stop
;

select char_5_win1252 from utf8table where id = 1;
-


The problem seems to be it is counting characters UTF8 using WIN1252 or 
maybe NONE, so UTF8 characters that take two bytes are counted as two 
characters.


As an example, if I change a failing test to use "12345" it works, if I 
change it to "\u00fe2345" (\u00fe is two bytes in UTF-8), the error is 
"expected length 5, actual 6".


If I change it to "\u20ac2345" (\u20ac is three bytes in UTF-8), the 
error is "expected length 5, actual 7".


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Incorrect string right truncation errors in Firebird 5

2022-05-16 Thread Mark Rotteveel

On 16-05-2022 16:50, Mark Rotteveel wrote:
I was running some tests against Firebird-5.0.0.494-0_x64 (latest 
snapshot, from last Saturday), and I notice that I get incorrect string 
right truncation errors with CHAR/VARCHAR.


I currently cannot dive deeper into it, but as a datapoint, the error 
does not occur with Firebird-5.0.0.488-0_x64.


Based solely on the commit message, maybe this commit is at fault? 
https://github.com/FirebirdSQL/firebird/commit/dd18a3b11b28c3ed8126a6f54b829989954bfa03 


The Jaybird test that triggers this is 
org.firebirdsql.jdbc.TestFBPreparedStatementUTF8, specifically tests:


- connectionUtf8_insertMultiByte_inWin1252_char_1_win1252_succeeds 
(fails with "expected length 1, actual 2")
- connectionUtf8_insertMultiByte_notInWin1252_char_1_win1252_fails 
(fails with "expected length 1, actual 2", should fail with "Cannot 
transliterate character between character sets")
- connectionUtf8_insertMultipleInWin1252_char_5_win1252_succeeds (fails 
with "expected length 5, actual 8")
- connectionUtf8_insertMultipleInWin1252_varchar_5_win1252_succeeds 
(fails with "expected length 5, actual 8")
- 
connectionUtf8_insertMultipleInWin1252_lessThanMax_varchar_5_win1252_succeeds 
(fails with "expected length 5, actual 6")
-connectionUtf8_insertMultiByte_char_1_utf8_succeeds (fails with 
"expected length 1, actual 2")
- connectionUtf8_insertMultiple_char_5_utf8_succeeds (fails with 
"expected length 5, actual 8")
- connectionUtf8_insertMultiple_varchar_5_utf8_succeeds (fails with 
"expected length 5, actual 8")
- connectionUtf8_insertMultiple_lessThanMax_varchar_5_utf8_succeeds 
(fails with "expected length 5, actual 6")


For CHAR, it seems to count 1 more character, for VARCHAR 2-3 more 
characters.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Incorrect string right truncation errors in Firebird 5

2022-05-16 Thread Mark Rotteveel
I was running some tests against Firebird-5.0.0.494-0_x64 (latest 
snapshot, from last Saturday), and I notice that I get incorrect string 
right truncation errors with CHAR/VARCHAR.


I currently cannot dive deeper into it, but as a datapoint, the error 
does not occur with Firebird-5.0.0.488-0_x64.


Based solely on the commit message, maybe this commit is at fault? 
https://github.com/FirebirdSQL/firebird/commit/dd18a3b11b28c3ed8126a6f54b829989954bfa03


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Refresh roles

2022-05-11 Thread Mark Rotteveel

On 11-05-2022 18:56, Alex Peshkoff via Firebird-devel wrote:
On the other hand I see no problems if we have dedicated command 
peforming desired action. Suggest syntax:

SET ROLE
without parameters. (but not insist on it)


To me, `SET ROLE` without a role name doesn't feel correct, and doesn't 
properly convey intent. Given we already have `ALTER SESSION RESET`, I 
would suggest `ALTER SESSION UPDATE PRIVILEGES` or something like that.


However, I'm wondering how other database systems behave in a scenario 
like this.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] DEFAULT grants to PUBLIC

2022-05-10 Thread Mark Rotteveel

On 10-05-2022 12:10, Dimitry Sibiryakov wrote:

Roman Simakov wrote 10.05.2022 10:21:

It makes sense. Could you create a ticket?


   But what's the point to grant object rights to the role and then 
grant the role to public instead of granting the rights to public directly?


Just like with normal default roles: granularity. You can group related 
permissions together, and make is easier to revoke those permissions en 
bloc if you no longer want everyone to have them.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] isc_spb_version Call Without Global User?

2022-05-04 Thread Mark Rotteveel

On 03-05-2022 18:56, Dimitry Sibiryakov wrote:

Scott Morgan via Firebird-devel wrote 03.05.2022 18:29:
What can you do to get the server's info if you only have per-DB user 
accounts?


   Use isc_dpb_expected_db with your database name to inform Firebird 
which setting from databases.conf should be used during service attach 
(including security database).


I assume you mean isc_spb_expected_db, right?

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Remove travis-ci?

2022-04-30 Thread Mark Rotteveel

On 30-04-2022 12:05, Mark Rotteveel wrote:

On 28-04-2022 11:51, Adriano dos Santos Fernandes wrote:

On 28/04/2022 05:37, Mark Rotteveel wrote:

Our travis-ci builds haven't actually run for the past 10 months as we
ran out of credits. As no one has noticed this in that time, it looks
like the travis-ci builds serve no actual purpose.

Any objections if I remove the travis-ci build definitions, and revoke
the travis-ci permissions from github.com/FirebirdSQL ?



No objections.

They seem to had a security problem. I have been notified by GitHub that
my account was used to "list my organizations" in a broader attack:

https://github.blog/2022-04-15-security-alert-stolen-oauth-user-tokens/


I have removed the Travis related things from B3_0_Release, v4.0-release 
and master and revoked the permission to the repository. I didn't touch 
B2_5_Release.


I still need to check two of the Java repositories before I can fully 
remove Travis.


All Travis access has been revoked.

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Remove travis-ci?

2022-04-30 Thread Mark Rotteveel

On 28-04-2022 11:51, Adriano dos Santos Fernandes wrote:

On 28/04/2022 05:37, Mark Rotteveel wrote:

Our travis-ci builds haven't actually run for the past 10 months as we
ran out of credits. As no one has noticed this in that time, it looks
like the travis-ci builds serve no actual purpose.

Any objections if I remove the travis-ci build definitions, and revoke
the travis-ci permissions from github.com/FirebirdSQL ?



No objections.

They seem to had a security problem. I have been notified by GitHub that
my account was used to "list my organizations" in a broader attack:

https://github.blog/2022-04-15-security-alert-stolen-oauth-user-tokens/


I have removed the Travis related things from B3_0_Release, v4.0-release 
and master and revoked the permission to the repository. I didn't touch 
B2_5_Release.


I still need to check two of the Java repositories before I can fully 
remove Travis.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors

2022-04-28 Thread Mark Rotteveel

On 22-04-2022 17:51, Dmitry Yemanov wrote:

22.04.2022 14:49, Mark Rotteveel wrote:
I was wondering if this - fetching rowcount of a scrollable cursor - 
was already implemented. If so, how do I get it? And if not, when can 
I expect it to be implemented?


PR #7083, was left for review but not merged yet. I see it has conflicts 
now, I will address them tomorrow and then merge the PR.


I notice that if I get the INF_RECORD_COUNT with op_info_cursor before 
the first fetch, I will get the record count, but the subsequent fetch 
will fail with a Dynamic SQL Error; SQLDA error; Data type unknown; at 
SQLVAR index 0 [SQLState:07002, ISC error code:335544583].


If I get the record count after the first fetch, everything is OK.

This seems to indicate that retrieving the record count somehow sets an 
empty SQLDA or something like that, and ignores the one from the first 
fetch.


Is this a bug? Should retrieving cursor info before the first fetch be 
disallowed by the server? Or maybe something else?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Remove travis-ci?

2022-04-28 Thread Mark Rotteveel
Our travis-ci builds haven't actually run for the past 10 months as we 
ran out of credits. As no one has noticed this in that time, it looks 
like the travis-ci builds serve no actual purpose.


Any objections if I remove the travis-ci build definitions, and revoke 
the travis-ci permissions from github.com/FirebirdSQL ?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors

2022-04-22 Thread Mark Rotteveel

On 08-12-2021 10:13, Dmitry Yemanov wrote:

28.11.2021 14:45, Mark Rotteveel wrote:
3) "row count" makes it possible to know the position after fetchLast() 
and everything else could be calculated locally by the client library, 
thus making the server-supported "current position" totally unnecessary.


Do I miss anything? Could we agree on having only "row count" returned 
via op_info_cursor and leaving "cursor position" (getRow() in Java API) 
up the connectivity library developers?


I was wondering if this - fetching rowcount of a scrollable cursor - was 
already implemented. If so, how do I get it? And if not, when can I 
expect it to be implemented?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Security vulnerability in zlib library

2022-03-31 Thread Mark Rotteveel

On 2022-03-31 15:39, Dimitry Sibiryakov wrote:

Alex Peshkoff via Firebird-devel wrote 31.03.2022 15:21:


  Note that the crash happen on compression so it doesn't affect 
Firebird security.


Did not catch why - we use zlib compression on the wire (since fb3) 
and in gbak (since fb4). Both cases are not default but anyway not 
good.


  The crash happen when a stream of definite data is tried to be
compressed. IMHO, it is hard (if possible at all) to purposefully
construct such stream *from* server to crash or exploit it.


That is a very dangerous assumption. Things people think "that is not 
possible to get exploited in our case" always seem to get exploited by 
people with sufficient motivation and drive. And even if it is not 
exploitable in the case of Firebird, that is not a reason not to update 
the dependency in a next release. It costs nearly nothing to do, and it 
avoids the potential for vulnerabilities, and the *perception* of being 
vulnerable.


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Security vulnerability in zlib library

2022-03-31 Thread Mark Rotteveel
A security vulnerability was found in zlib: 
https://nakedsecurity.sophos.com/2022/03/29/zlib-data-compressor-fixes-17-year-old-security-bug-patch-errr-now/


Will we include an updated version in the next release?
Can people just drop in a replacement?

Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] isc_dpb_lc_ctype and isc_dpb_set_db_charset and isc_dpb_utf8_filename

2022-03-01 Thread Mark Rotteveel

On 2022-03-01 12:47, Alex Peshkoff via Firebird-devel wrote:

On 2/28/22 21:39, Jiří Činčura wrote:
Thanks. I suppose not only TPB, but also buffers for batch, events, 
etc., right?


Yes.



Basically only DPB and SPB are "UTF8-ready".



If one wants everyting to be utf8 why not have appropriate attachment?


Having the connection character set UTF8 is IMHO a separate concern from 
driver/protocol-related communication. The benefits of using UTF8 for 
driver/protocol communications doesn't always work well with the 
limitations that UTF8 enforces on the size of (VAR)CHAR columns. If we 
ever design a new protocol, things like this need to be taken into 
account.


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] INT64 and index keys

2022-02-16 Thread Mark Rotteveel

On 2022-02-16 09:56, Vlad Khorsun wrote:

16.02.2022 10:35, Mark Rotteveel wrote:

On 2022-02-15 20:08, Vlad Khorsun wrote:

15.02.2022 20:32, Mark Rotteveel wrote:

On 2022-02-15 16:20, Vlad Khorsun wrote:

  I'd vote to remove idx_numeric2 in favour of idx_decimal and look
how to improve
Decimal128::makeIndexKey() performance. For example, it stores 
every 3 decimal
digits into 10 bits using relatively complex (computationally) 
algorithm with
shifts and multiplications. Storing every digit in 4 bits will use 
2 bits per 3

digits more but should save come CPU circles, I believe.


But that is the storage format of Decimal128 (DECFLOAT(34)).


  No. It looks more like packed BCD. DECFLOAT uses a more complex 
internals,

it is not BCD inside.


Decimal128 uses densely packed BCD (with some oddities to encode sign, 
exponent and first digit).


  It is not BCD and definitely not what uses 
Decimal128::makeIndexKey().

See http://speleotrove.com/decimal/dbspec.html
and http://speleotrove.com/decimal/DPDecimal.html

Also, note, Decimal encodings is not directly sortable.


From that first link under 'coefficient continuation': "Each 10-bit 
group represents three decimal digits, using Densely Packed Decimal 
encoding." and at the footnote (and the top of your second link): 
"Chen-Ho encoding is a lossless compression of three Binary Coded 
Decimal digits into 10 bits using an algorithm which can be applied or 
reversed using only simple Boolean operations".


However, that was not my main point. My main point was that it sounds 
like an index format that was created for supporting DECFLOAT(34), and 
that it is not suitable for the full range of INT128 and NUMERIC/DECIMAL 
backed by INT128 (for the same reasons the DOUBLE PRECISION format is 
not suitable for BIGINT and NUMERIC/DECIMAL backed by BIGINT).


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] INT64 and index keys

2022-02-16 Thread Mark Rotteveel

On 2022-02-15 20:08, Vlad Khorsun wrote:

15.02.2022 20:32, Mark Rotteveel wrote:

On 2022-02-15 16:20, Vlad Khorsun wrote:

  I'd vote to remove idx_numeric2 in favour of idx_decimal and look
how to improve
Decimal128::makeIndexKey() performance. For example, it stores every 
3 decimal
digits into 10 bits using relatively complex (computationally) 
algorithm with
shifts and multiplications. Storing every digit in 4 bits will use 2 
bits per 3

digits more but should save come CPU circles, I believe.


But that is the storage format of Decimal128 (DECFLOAT(34)).


  No. It looks more like packed BCD. DECFLOAT uses a more complex 
internals,

it is not BCD inside.


Decimal128 uses densely packed BCD (with some oddities to encode sign, 
exponent and first digit).


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] INT64 and index keys

2022-02-15 Thread Mark Rotteveel

On 2022-02-15 16:20, Vlad Khorsun wrote:

  I'd vote to remove idx_numeric2 in favour of idx_decimal and look
how to improve
Decimal128::makeIndexKey() performance. For example, it stores every 3 
decimal
digits into 10 bits using relatively complex (computationally) 
algorithm with
shifts and multiplications. Storing every digit in 4 bits will use 2 
bits per 3

digits more but should save come CPU circles, I believe.


But that is the storage format of Decimal128 (DECFLOAT(34)). It makes no 
sense to replace that with something else for DECFLOAT(34) columns. So 
maybe a different index format is necessary for INT128 (and NUMERIC(38, 
...), because Decimal128 doesn't sound suitable for those types as it 
has less precision (34 digits instead of 38) (similar like the double 
precision index format is not suitable for INT64/BIGINT).


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Statement type for INSERT ... RETURNING

2022-02-12 Thread Mark Rotteveel

On 11-02-2022 17:46, Mark Rotteveel wrote:

On 2022-02-11 12:54, Jiří Činčura wrote:

Is there a way to get as a response to isc_info_sql_stmt_type for
"INSERT ... RETURNING" isc_info_sql_stmt_insert instead of
isc_info_sql_stmt_exec_procedure? Or something similar that would
specify it's an "insert"?


No. In fact, with Firebird 5, statements with RETURNING can even 
identify as isc_info_sql_stmt_select. The statement types are more about 
how you should handle statement execution than anything else. When it 
was discussed last time, I believe the idea was raised to add a new info 
type to get this information, but that it would first need a real use 
case to do so. I'll see if I can find that discussion.


See discussions:
- subject "isc_info_sql_stmt_type/isc_info_sql_stmt_flags" on the 19th 
and 20th of August 2021

- subject "Firebird 5 and Update...Returning" on the 26th of November 2021

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Statement type for INSERT ... RETURNING

2022-02-11 Thread Mark Rotteveel

On 2022-02-11 12:54, Jiří Činčura wrote:

Is there a way to get as a response to isc_info_sql_stmt_type for
"INSERT ... RETURNING" isc_info_sql_stmt_insert instead of
isc_info_sql_stmt_exec_procedure? Or something similar that would
specify it's an "insert"?


No. In fact, with Firebird 5, statements with RETURNING can even 
identify as isc_info_sql_stmt_select. The statement types are more about 
how you should handle statement execution than anything else. When it 
was discussed last time, I believe the idea was raised to add a new info 
type to get this information, but that it would first need a real use 
case to do so. I'll see if I can find that discussion.


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] User name case

2022-01-20 Thread Mark Rotteveel

On 2022-01-19 16:46, Dimitry Sibiryakov wrote:

Alex Peshkoff via Firebird-devel wrote 18.01.2022 18:03:

     fb_utils::dpbItemUpper(userName);


  That's quite strange routine. It will uppercase the string even if
it contains ASCII letters disallowed for not-delimited SQL identifier
(which user name is).


A user name is a delimited identifier since Firebird 3.0, but if it was 
delimited, it wouldn't be uppercased.


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] isc_usrname_too_long

2022-01-14 Thread Mark Rotteveel

On 14-01-2022 17:10, Dimitry Sibiryakov wrote:

Mark Rotteveel wrote 14.01.2022 17:06:
   Also shouldn't it to be returned if user name received in DPB 
(after conversion into UTF-8) doesn't fit as well?


I'm not sure what you mean with that.


   Currently if someone put into DPB non-ASCII user name and during 
conversion into UTF-8 it doesn't fit 255 bytes the error "attempt to 
store %d bytes in a clumplet with maximum size 255 bytes" is thrown. 
Shouldn't the error to be a little more specific and comprehensive...?


Personally, I don't think so, as I think that is messy to conflate logic 
for populating the DPB with selecting appropriate messages specific to 
the select DPB item.


Also when you use a DPB v2, it is wide, so accepts far larger values 
than you'll ever logically need for a user name.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] isc_usrname_too_long

2022-01-14 Thread Mark Rotteveel

On 14-01-2022 16:41, Dimitry Sibiryakov wrote:

   Hello All.

   Shouldn't subj text to be corrected?


I probably should.

   Also shouldn't it to be returned if user name received in DPB (after 
conversion into UTF-8) doesn't fit as well?


I'm not sure what you mean with that.

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Sub routines accessing outer variables, parameters and cursors

2022-01-11 Thread Mark Rotteveel

On 2022-01-11 20:15, Adriano dos Santos Fernandes wrote:

Hi!

Currently sub routines cannot access the outer (main routine/block)
variables, parameters and cursors - #4769.

There is some details to make that happen safely.

Currently a variable cannot have its value changed while its being 
used.


var = var + 

No expression can reset var.

As soon we let sub routines change outer variables, that will be 
possible.


Currently VariableNode just returns a descriptor pointing to the
variable's impure_value present in the variable declaration.

To make outer access safe, variables accessed in outer scope would be
flagged and its VariableNode (even in the main routine/block) should
copy the value to its own impure_value. It will be a bit slower, but I
don't see a problem.

What worried me more is how to represent things in BLR. A simple
approach would be to have blr_outer_variable.

But it is not only variables.

There is also parameters, fields, dbkeys, cursor statements.

What seems for me the way to go is to implement single blr_outer_map
that maps main routine's things to sub routines in they own number 
space.


blr_outer_map,
blr_outer_map_variable,
0,0,
3,0

It maps main's variable 0,0 to this sub routine's 3,0, so it can access
with blr_variable, 3,0.

blr_outer_map,
blr_outer_map_cursor,
1,0,
2,0

Maps main's cursor 1,0 to this sub routine's 2,0, so it can access with
blr_cursor_stmt, 2,0.

Comments?


It is not entirely clear to me what you're proposing and what the 
motivation is for the proposal. Are you proposing to allow local 
subroutines to modify their parents variable, or are you proposing 
something to allow reading a local variable, but prevent modification?


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Commit (un)certainity

2022-01-06 Thread Mark Rotteveel

On 2022-01-05 21:48, Jiří Činčura wrote:

   Yes. You probably have the same problem in your provider, no?


The problem exists there. Same as with fbclient. But I don't
explicitly solve it, application developer is responsible for handling
such scenario (if ever).


I agree. If the application developer thinks this is a problem, they 
should switch to using 2-phase commits and having a scavenger to handle 
limbo transactions.


BTW: I think in most cases you should be able to infer from the socket 
error if the problem occurred on sending or on reading data (at least, 
when using blocking socket reads/writes, not sure about non-blocking).


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Content for p_batch_create.p_batch_pb

2021-12-26 Thread Mark Rotteveel

On 26-12-2021 09:59, Alex Peshkoff via Firebird-devel wrote:

On 12/16/21 11:51, Jiří Činčura wrote:

Or maybe more broadly, is there some high level description of the 
batching? I'm failing to find something, except batch interface 
description in Using_OO_API.html, but that's about API, not about the 
"protocol".




Added chapter to
https://github.com/FirebirdSQL/firebird-documentation/blob/master/src/docs/asciidoc/en/firebirddocs/wireprotocol/firebird-wire-protocol.adoc#wireprotocol-batches-release 


I did some minor copy editing and published it.

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Content for p_batch_create.p_batch_pb

2021-12-26 Thread Mark Rotteveel

On 26-12-2021 09:59, Alex Peshkoff via Firebird-devel wrote:

On 12/16/21 11:51, Jiří Činčura wrote:

Or maybe more broadly, is there some high level description of the 
batching? I'm failing to find something, except batch interface 
description in Using_OO_API.html, but that's about API, not about the 
"protocol".




Added chapter to
https://github.com/FirebirdSQL/firebird-documentation/blob/master/src/docs/asciidoc/en/firebirddocs/wireprotocol/firebird-wire-protocol.adoc#wireprotocol-batches-release 


Thanks Alex!

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Documentation of SET DEBUG OPTION

2021-12-24 Thread Mark Rotteveel

On 24-12-2021 13:14, Adriano dos Santos Fernandes wrote:

On 24/12/2021 07:12, Mark Rotteveel wrote:

Checking the changes of Firebird 4.0.1 to update the docs, I noticed
that a new statement `SET DEBUG OPTION` has been introduced, but I can't
find much about it, except for "To make isc_info_sql_exec_path_blr_*
work, session must be first execute set debug option dsql_keep_blr =
true." in https://github.com/FirebirdSQL/firebird/issues/6910

Questions:

1. Is dsql_keep_blr the only option at the moment, or are there more?
(the code suggests not)


It's the only one.



2. Is SET DEBUG OPTION general for an entire connection, or does it need
to be executed on the statement handle you want to retrieve one of the
isc_info_sql_exec_path_blr_* options from? (checking the commit, it
seems to be attachment level, but I'd like confirmation


It's for the attachment.



3. How do you clear an option, set value to false, can you set to null?



SET DEBUG OPTION valid_symbol_name '=' constant

The general command accepts any constant (NULL is not included).

DSQL_KEEP_BLR requires a boolean constant. But it can also be implicit
converted via strings ('true' or 'false').


Also, is there more to it than this?


It's good to reproduce the warning put in the issue:


Warning: this feature is very tied to engine internals and its usage is
discouraged
if you do not understand very well how these internals are subject to
change between
versions.



Thanks, I added the following:

- ISQL: 
https://www.firebirdsql.org/file/documentation/html/en/firebirddocs/isql/firebird-isql.html#isql-set-exec-path-display
- langref: 
https://www.firebirdsql.org/file/documentation/chunk/en/refdocs/fblangref40/fblangref40-management-debug.html#fblangref40-management-setdebugoption


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Documentation of SET DEBUG OPTION

2021-12-24 Thread Mark Rotteveel
Checking the changes of Firebird 4.0.1 to update the docs, I noticed 
that a new statement `SET DEBUG OPTION` has been introduced, but I can't 
find much about it, except for "To make isc_info_sql_exec_path_blr_* 
work, session must be first execute set debug option dsql_keep_blr = 
true." in https://github.com/FirebirdSQL/firebird/issues/6910


Questions:

1. Is dsql_keep_blr the only option at the moment, or are there more? 
(the code suggests not)
2. Is SET DEBUG OPTION general for an entire connection, or does it need 
to be executed on the statement handle you want to retrieve one of the 
isc_info_sql_exec_path_blr_* options from? (checking the commit, it 
seems to be attachment level, but I'd like confirmation

3. How do you clear an option, set value to false, can you set to null?

Also, is there more to it than this?
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] varbinary

2021-12-20 Thread Mark Rotteveel

On 2021-12-20 14:04, Norbert Saint Georges wrote:

Thank you for your two responses.
unfortunately I have a problem with an old project, following the
update of the netprovider 8.5 client.
Several varchar (x) character set bytes are declared there which
crashes on an expected byte [] even though I have been sending it
strings for several months :-)
but ok :-)


That sounds like something to ask about on 
https://groups.google.com/g/firebird-net-provider


Make sure to include the necessary code and the actual error you 
receive.


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors

2021-12-20 Thread Mark Rotteveel

On 2021-12-20 11:51, Dmitry Yemanov wrote:

Mark et al,

Looked at it again, and being able to get the total row count will 
work

for me. Is this information already available, or does this still need
to be implemented?

What would you expect from the "row count" requested for a
non-scrollable cursor? It cannot return the true count, as already
explained here. Should it return some magic number (0? 1? -1?) or
would error be appropriate? If the latter, should it be isc_infinap
(information type inappropriate for object specified) or isc_infona
(no information of this type available for object specified)?


I'd say -1 as a magic value to signal "unknown".

Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] varbinary

2021-12-20 Thread Mark Rotteveel

On 2021-12-20 12:40, Norbert Saint Georges wrote:

Hello,


a varbinary has a subtype of 1 and not 0 (binary) normal?


You're thinking of BLOB SUB_TYPE, and this is unrelated to BLOB 
SUB_TYPE, as it isn't a BLOB type.


In Firebird 4, VARBINARY is introduced as *an alias* for VARCHAR 
CHARACTER SET OCTETS (and BINARY for CHAR CHARACTER SET OCTETS), and 
"normal" VARCHAR (or CHAR) has sub_type 0. The character set OCTETS has 
id 1, and some parts of Firebird communicate the character set in the 
sub_type field (e.g. of XSQLDA), so to make this backwards compatible, 
the character set id of OCTETS (1) is also used as its 
RDB$FIELD_SUB_TYPE value. In that way, "old" tools handle it correctly 
as (VAR)CHAR CHARACTER SET OCTETS, while Firebird 4.0 tools can identify 
it as (VAR)BINARY.


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors

2021-12-19 Thread Mark Rotteveel

On 08-12-2021 10:13, Dmitry Yemanov wrote:

28.11.2021 14:45, Mark Rotteveel wrote:


We don't have anything like this. Theoretically, we could extend 
IRecordSet with something similar (although it would also require a 
protocol change), but the question is whether it's really needed. 
Personally, I don't see it useful per se. If users want to know a 
number of rows, then perhaps an explicit getRowCount() method would 
be more useful (*). But AFAIU the Java API does not mention it.


(*) not applicable to uni-directional cursors?


The intent of the method is to report the position of the current row. 
The example it is abused by some to get the total size was just an 
illustration of why people would expect a value after requesting the 
last row. But the same would be a problem if people went to last, 
scrolled around and then want to know the current row position.


I was thinking about providing both "current position" and "row count" 
information values for a cursor. Both are doable, however the former has 
some issues:

[..]
3) "row count" makes it possible to know the position after fetchLast() 
and everything else could be calculated locally by the client library, 
thus making the server-supported "current position" totally unnecessary.


Do I miss anything? Could we agree on having only "row count" returned 
via op_info_cursor and leaving "cursor position" (getRow() in Java API) 
up the connectivity library developers?


Looked at it again, and being able to get the total row count will work 
for me. Is this information already available, or does this still need 
to be implemented?


One last thing I was wondering about (but haven't had time to verify): 
what happens with positional updates or deletes when scrolling?


Specifically:
- After an update: does the cursor have the original row content or the 
updated content?
- After a delete: does the row disappear from the cursor, does the 
cursor retain the original content of the row, or does an 'empty' (e.g. 
all NULL) row exist in the cursor?


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Content for p_batch_create.p_batch_pb

2021-12-16 Thread Mark Rotteveel

On 2021-12-16 14:23, Alex Peshkoff via Firebird-devel wrote:

On 12/16/21 15:12, Mark Rotteveel wrote:

On 2021-12-16 10:28, Alex Peshkoff via Firebird-devel wrote:

be segmented and swap bytes in segment length when needed. Afraid
there is - as always :( - no protocol description except source code.
Key ops are op_batch_msg, op_batch_blob_stream & also op_batch_cs
(retuning batch execution state to client).


That is not entirely true, there is [1] which was basically 
reverse-engineerd by Carlos Guzman Alvarez, and I updated some of it 
for protocol 11. Personally, I really don't like the current situation 
that the core developers don't document the actual protocol, because 
reverse engineering from the code and by using things like WireShark 
is extremely painful, confusing and error prone. With the batch API 
it's even worse IMHO, because even the usage examples seem to be 
incomplete, and too basic/simple and glosses over things.


Mark

 [1]: 
https://firebirdsql.org/file/documentation/html/en/firebirddocs/wireprotocol/firebird-wire-protocol.html




And where is the source of this docs?



It's in the firebird-documentation project[1], specifically 
https://github.com/FirebirdSQL/firebird-documentation/blob/master/src/docs/asciidoc/en/firebirddocs/wireprotocol/firebird-wire-protocol.adoc


Mark

 [1]: https://github.com/FirebirdSQL/firebird-documentation


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Content for p_batch_create.p_batch_pb

2021-12-16 Thread Mark Rotteveel

On 2021-12-16 10:28, Alex Peshkoff via Firebird-devel wrote:

be segmented and swap bytes in segment length when needed. Afraid
there is - as always :( - no protocol description except source code.
Key ops are op_batch_msg, op_batch_blob_stream & also op_batch_cs
(retuning batch execution state to client).


That is not entirely true, there is [1] which was basically 
reverse-engineerd by Carlos Guzman Alvarez, and I updated some of it for 
protocol 11. Personally, I really don't like the current situation that 
the core developers don't document the actual protocol, because reverse 
engineering from the code and by using things like WireShark is 
extremely painful, confusing and error prone. With the batch API it's 
even worse IMHO, because even the usage examples seem to be incomplete, 
and too basic/simple and glosses over things.


Mark

 [1]: 
https://firebirdsql.org/file/documentation/html/en/firebirddocs/wireprotocol/firebird-wire-protocol.html



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Fwd: [FirebirdSQL/firebird] Connection hangs after delivery of 2**32 - 1 packets (Issue #7065)

2021-12-12 Thread Mark Rotteveel
I strongly disagree with the chosen fix to make the counter size 
configurable. ChaCha20 is standardized in RFC-7539 with a 32-bit counter 
size[1]. Making the counter size configurable has two problems:


1) It is harder to support (as non-standard forms of ChaCha are not 
always available)
2) The client has no way to know which counter variant the server 
expects, and this needs to be explicitly configured both by the client 
and the server, which is really not ideal, and will lead to hard to 
diagnose connection problems.


The proper way to fix this is to define a separate encryption plugin 
name for the variant with a 64-bit counter, so that client and server 
can negotiate the appropriate plugin that is supported.


Alternatively, re-keying could be supported, so that client and server 
can change keys during a connection, but this comes with additional 
challenges.


Mark

 [1]: https://datatracker.ietf.org/doc/html/rfc7539#section-2.4

 Original Message 
Subject: [FirebirdSQL/firebird] Connection hangs after delivery of 2**32 
- 1 packets (Issue #7065)

Date: 2021-12-12 18:26
From: Alexander Peshkov 
To: FirebirdSQL/firebird 
Cc: Subscribed 
Reply-To: FirebirdSQL/firebird 



ChaCha wire encryption, used by default since FB4, is using 32-bit
counter. When counter overflows secure packets delivery becomes
impossible without reconnect.

--
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub [1], or unsubscribe [2].
Triage notifications on the go with GitHub Mobile for iOS [3] or Android
[4].

Links:
--
[1] https://github.com/FirebirdSQL/firebird/issues/7065
[2] 
https://github.com/notifications/unsubscribe-auth/ABI2Z4J2M42KMEBDP2KTWQDUQTLNJANCNFSM5J4PIROA
[3] 
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-emailmt=8pt=524675
[4] 
https://play.google.com/store/apps/details?id=com.github.androidreferrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors

2021-12-08 Thread Mark Rotteveel

On 2021-12-08 10:13, Dmitry Yemanov wrote:

28.11.2021 14:45, Mark Rotteveel wrote:


We don't have anything like this. Theoretically, we could extend 
IRecordSet with something similar (although it would also require a 
protocol change), but the question is whether it's really needed. 
Personally, I don't see it useful per se. If users want to know a 
number of rows, then perhaps an explicit getRowCount() method would 
be more useful (*). But AFAIU the Java API does not mention it.


(*) not applicable to uni-directional cursors?


The intent of the method is to report the position of the current row. 
The example it is abused by some to get the total size was just an 
illustration of why people would expect a value after requesting the 
last row. But the same would be a problem if people went to last, 
scrolled around and then want to know the current row position.


I was thinking about providing both "current position" and "row count"
information values for a cursor. Both are doable, however the former
has some issues:

1) Adjustments due to prefetch should be taken into account at both
client and server sides, thus complicating the code (information
response buffer must be parsed, value received from the engine/server
must be replaced with the adjusted one).

2) If used "with purpose", it may be requested by the client after
every fetch, thus multiplying the round-trips and killing the prefetch
benefits. So it could make sense to request the position (send
op_info_cursor) internally after every op_fetch_scroll packet. As
every fetch request may return multiple rows, sending many positions
is unnecessary and only position of the first row could be returned.
This puts additional requirements to the client-side protocol
implementations.

3) "row count" makes it possible to know the position after
fetchLast() and everything else could be calculated locally by the
client library, thus making the server-supported "current position"
totally unnecessary.

Do I miss anything? Could we agree on having only "row count" returned
via op_info_cursor and leaving "cursor position" (getRow() in Java
API) up the connectivity library developers?


That sounds doable, but I won't have time to really look and think about 
this until the weekend. I'll reply then if I have more concerns.


Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Protocol 15

2021-12-07 Thread Mark Rotteveel

On 2021-12-06 19:10, Jiří Činčura wrote:

IIRC, the only real change in protocol 14 was


Yeah. Not really worth the effort I meant, it will be "solved" by 15
implementation.


Yeah, I did the same in Jaybird: skipped implementation of protocol 14.

Mark


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Protocol 15

2021-12-06 Thread Mark Rotteveel

On 06-12-2021 15:12, Jiří Činčura wrote:

In a nutshell, what changed in protocol 15 compared to 13? Is it only the early 
crypt key callback?

And I suppose there's not much into protocol 14, right?


IIRC, the only real change in protocol 14 was the inclusion of the int32 
p_cc_reply in the op_crypt_key_callback.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors

2021-12-01 Thread Mark Rotteveel

On 01-12-2021 16:06, Dmitry Yemanov wrote:

01.12.2021 16:07, Dimitry Sibiryakov wrote:


Also if IResultSet is returned from IAttachment::openCursor() there is 
no (visible) IStatement at all.


And I see the same problem for setCursorName() which is available only 
through IStatement. An oversight?


You need to set the cursor name before you open it, IIRC. So - assuming 
I'm not mistaken in this - adding it to IResultSet doesn't make sense, 
as that will only return once the cursor has been opened.


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] WNET future

2021-11-30 Thread Mark Rotteveel

On 30-11-2021 15:26, Adriano dos Santos Fernandes wrote:

On 30/11/2021 10:24, Jiří Činčura wrote:

My vote goes to dropping it.



Agreed.


I have no opinion on this.

Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Firebird and DSQL Scrollable Cursors

2021-11-28 Thread Mark Rotteveel

On 28-11-2021 12:55, Dimitry Sibiryakov wrote:

Dmitry Yemanov wrote 28.11.2021 12:51:

28.11.2021 14:47, Dimitry Sibiryakov wrote:


RDB$DB_KEY for the current record as well.


It's not a property of the cursor. Consider joins, unions, procedures, 
views, etc.


   It is a property of a current record the same as position. It is ok 
to return wide concatenated DB_KEY (up to 255 bytes due limitation of 
info buffer format).


The statement info uses 16-bit size integers, so generally it would 
65535, but AFAIK, it's contextually sized (e.g. isc_info_sql_select and 
isc_info_sql_bind don't have their own size).


Mark
--
Mark Rotteveel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


  1   2   3   4   5   6   7   8   9   10   >