Re: [Firebird-devel] ICU in HEAD
On 2/8/19 10:56 AM, Vlad Khorsun wrote: 08.02.2019 9:31, Alex Peshkoff via Firebird-devel wrote: On 2/7/19 6:14 PM, Dimitry Sibiryakov wrote: Hello, All. I somehow understand why build of current HEAD is failing with message "Could not find acceptable ICU library", but I wonder why it takes so much time to get this error? More than 15 second between running of isql and this failure on my notebook. Probably, you have too much entries in PATH I can't reproduce performance problem: > > # time (echo 'exit;' | ./isql employee) > Statement failed, SQLSTATE = XX000 > Could not find acceptable ICU library > -Missing entrypoint u_setDataDirectory in ICU library > Use CONNECT or CREATE DATABASE to specify a database > > real 0m0.204s This is too much, imo. Hope strace could show the huge number of calls made. Yes. I can show such trace on Windows, if you wish. Is it time to re-think how UnicodeUtil::getConversionICU() search for ICU library ? Suggestions? The fastest way sooner of all will be storing favorite ICU verson (once detected using long process) somewhere out of databases. But may be some better idea? Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
08.02.2019 10:48, Alex Peshkoff via Firebird-devel wrote: On 2/8/19 10:56 AM, Vlad Khorsun wrote: Is it time to re-think how UnicodeUtil::getConversionICU() search for ICU library ? Suggestions? The fastest way sooner of all will be storing favorite ICU verson (once detected using long process) somewhere out of databases. But may be some better idea? I see "icu_versions" parameter at fbintl.conf but it seems not used by getConversionICU(). Probably it could be used here ? Also, we could limit number of ICU versions to check: - exclude really old versions (at least with old X.Y versioning scheme, i.e. 4.8 and older) - adapt search knowing existing versions numbers (see http://site.icu-project.org/download) - guess number of future versions Regards, Vlad Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Changing generator's initial value without restarting
Hi, is there a way to change the intial value of generator/sequence *without* restarting it? -- Mgr. Jiří Činčura https://www.tabsoverspaces.com/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
08.02.2019 8:31, Alex Peshkoff via Firebird-devel wrote: I can't reproduce performance problem: # time (echo 'exit;' | ./isql employee) Statement failed, SQLSTATE = XX000 Could not find acceptable ICU library -Missing entrypoint u_setDataDirectory in ICU library The error you are getting is different, so I would guess that search loop is aborted faster in this case. Also on linux other providers, which are called after Engine13 failure, may work differently and don't waste time waiting for connect to host "C", for example. Unfortunately, MSVC 2013 Community Edition does not include a profiler, but I'll try to see what is happening with ProcessMonitor. -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Changing generator's initial value without restarting
On 2/8/19 1:24 PM, Jiří Činčura wrote: Hi, is there a way to change the intial value of generator/sequence *without* restarting it? ALTER SEQUENCE? But please explain what do you mean under 'restart'. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
08.02.2019 10:15, Vlad Khorsun wrote: Also, we could limit number of ICU versions to check: - exclude really old versions (at least with old X.Y versioning scheme, i.e. 4.8 and older) - adapt search knowing existing versions numbers (see http://site.icu-project.org/download) - guess number of future versions I would also suggest to search from newest versions to oldest. But one question left: why the ICU which is in sources is considered "unacceptable"? Does it miss any functionality _required_ for build process? -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
On 2/8/19 2:01 PM, Dimitry Sibiryakov wrote: 08.02.2019 8:31, Alex Peshkoff via Firebird-devel wrote: I can't reproduce performance problem: # time (echo 'exit;' | ./isql employee) Statement failed, SQLSTATE = XX000 Could not find acceptable ICU library -Missing entrypoint u_setDataDirectory in ICU library The error you are getting is different, so I would guess that search loop is aborted faster in this case. Is it good idea to provide full error message? (like I did - not to make other guess) Also on linux other providers, which are called after Engine13 failure, may work differently and don't waste time waiting for connect to host "C", for example. I remember there was a code avoiding host names contatning single drive letter. But may be I'm missing something with windows build... Anyway - iif you suppose problems with other providers you can turn them off to avoid side effects. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
08.02.2019 12:21, Alex Peshkoff via Firebird-devel wrote: The error you are getting is different, so I would guess that search loop is aborted faster in this case. Is it good idea to provide full error message? (like I did - not to make other guess) Not in this case, but here it is: creating msg.fdb SQL> can't format message 17:0 -- message file C:\Users\SD\MYDOCU~1\firebird\temp\x64\release\firebird\firebird.msg not found Could not find acceptable ICU library SQL> loading msg.sql can't format message 17:0 -- message file C:\Users\SD\MYDOCU~1\firebird\temp\x64\release\firebird\firebird.msg not found Could not find acceptable ICU library can't format message 17:120 -- message file C:\Users\SD\MYDOCU~1\firebird\temp\x64\release\firebird\firebird.msg not found -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
On 2/8/19 2:24 PM, Dimitry Sibiryakov wrote: 08.02.2019 12:21, Alex Peshkoff via Firebird-devel wrote: The error you are getting is different, so I would guess that search loop is aborted faster in this case. Is it good idea to provide full error message? (like I did - not to make other guess) Not in this case, but here it is: creating msg.fdb SQL> can't format message 17:0 -- message file C:\Users\SD\MYDOCU~1\firebird\temp\x64\release\firebird\firebird.msg not found Could not find acceptable ICU library SQL> loading msg.sql can't format message 17:0 -- message file C:\Users\SD\MYDOCU~1\firebird\temp\x64\release\firebird\firebird.msg not found Could not find acceptable ICU library can't format message 17:120 -- message file C:\Users\SD\MYDOCU~1\firebird\temp\x64\release\firebird\firebird.msg not found Well - as for me I can with some efforts read that . As a minimum it's clear that it differs from what I get. When fully reproduced your case must say that result does not differ much: # time (echo 'exit;' | ./isql employee) Statement failed, SQLSTATE = XX000 Could not find acceptable ICU library Use CONNECT or CREATE DATABASE to specify a database real 0m0.199s user 0m0.048s sys 0m0.086s Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
On 2/8/19 12:15 PM, Vlad Khorsun wrote: 08.02.2019 10:48, Alex Peshkoff via Firebird-devel wrote: On 2/8/19 10:56 AM, Vlad Khorsun wrote: Is it time to re-think how UnicodeUtil::getConversionICU() search for ICU library ? Suggestions? The fastest way sooner of all will be storing favorite ICU verson (once detected using long process) somewhere out of databases. But may be some better idea? I see "icu_versions" parameter at fbintl.conf but it seems not used by getConversionICU(). Probably it could be used here ? Not sure is it needed or not currently (Adriano should know better) - but I dislike an idea of making .conf files firebird-writable for security reasons. Also, we could limit number of ICU versions to check: - exclude really old versions (at least with old X.Y versioning scheme, i.e. 4.8 and older) Suppose we may exclude versions missing TZ support needed for FB4. What about versioning scheme - now it depends upon build parameters and therefore in linux is distro-depended. Not good criteria IMHO :) - adapt search knowing existing versions numbers (see http://site.icu-project.org/download) Yes, this should be efficient. - guess number of future versions Ask the spirit of Vanga? ;) Getting serious we may just decrease number of future versions to analyze - anyway they will invent something making that versions not loadable with current code. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] CORE-5997
On 2/8/19 10:04 AM, liviuslivius wrote: Hi Can we discusse about CORE-5997? Why it was so fast closed as its iplementation is so important from performance POV? In mentioned tracker ticket I see: > This is e.g. slow compared to udf equivalent I.e. I assume that this problem is raised due to UDR deprecation? Why not use UDR (written using any language you like) for complex strings manipulatons? Adriano say that strings are immutable - but this is only some implementation detail - all strings in C++ are mutable. C++ is not good example here. First of all there are no strings in C/C++ at all. Just arrays of characters or classes with redefined [] operator. And std::string is also not string you want. This typedef for basic_string does not understand UTF-8 like you probably wish writing char_length (not octet_length) in your sample. Also being generic array, not real string, std::string does not understand upper/lower case operations, trimming strings, may be something else. I do not want to say that I hate your idea in general. But IMO in curretn state it's not ready for implementation. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
08.02.2019 13:29, Alex Peshkoff via Firebird-devel wrote: On 2/8/19 12:15 PM, Vlad Khorsun wrote: 08.02.2019 10:48, Alex Peshkoff via Firebird-devel wrote: On 2/8/19 10:56 AM, Vlad Khorsun wrote: Is it time to re-think how UnicodeUtil::getConversionICU() search for ICU library ? Suggestions? The fastest way sooner of all will be storing favorite ICU verson (once detected using long process) somewhere out of databases. But may be some better idea? I see "icu_versions" parameter at fbintl.conf but it seems not used by getConversionICU(). Probably it could be used here ? Not sure is it needed or not currently (Adriano should know better) - but I dislike an idea of making .conf files firebird-writable for security reasons. I doesn't offer to write anything into .conf. I ask if we can use already present parameter. It is used by loadICU(), but not by getConversionICU() (which is called earlier, as i see) Regards, Vlad Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
On 2/8/19 3:00 PM, Vlad Khorsun wrote: 08.02.2019 13:29, Alex Peshkoff via Firebird-devel wrote: On 2/8/19 12:15 PM, Vlad Khorsun wrote: 08.02.2019 10:48, Alex Peshkoff via Firebird-devel wrote: On 2/8/19 10:56 AM, Vlad Khorsun wrote: Is it time to re-think how UnicodeUtil::getConversionICU() search for ICU library ? Suggestions? The fastest way sooner of all will be storing favorite ICU verson (once detected using long process) somewhere out of databases. But may be some better idea? I see "icu_versions" parameter at fbintl.conf but it seems not used by getConversionICU(). Probably it could be used here ? Not sure is it needed or not currently (Adriano should know better) - but I dislike an idea of making .conf files firebird-writable for security reasons. I doesn't offer to write anything into .conf. I ask if we can use already present parameter. It is used by loadICU(), but not by getConversionICU() (which is called earlier, as i see) May be we can. I've thought about writing to it cause thread has my words ''storing favorite ICU verson (once detected using long process)". As solution for windows it's probably OK. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Changing generator's initial value without restarting
> ALTER SEQUENCE? > But please explain what do you mean under 'restart'. Without changing the current value. -- Mgr. Jiří Činčura https://www.tabsoverspaces.com/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Changing generator's initial value without restarting
08.02.2019 14:01, Jiří Činčura wrote: ALTER SEQUENCE? But please explain what do you mean under 'restart'. Without changing the current value. Try CREATE OR ALTER SEQUENCE. I cannot remember if I ported this improvement from Avalerion to main tree. -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] What type of UUID we use in Firebird ? i guess Version 1
What type of UUID we use in Firebird ? I guess Version 1 UUIDs in MySQL are really not random https://news.ycombinator.com/item?id=19085189 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] What type of UUID we use in Firebird ? i guess Version 1
08.02.2019 16:44, marius adrian popa wrote: What type of UUID we use in Firebird ? I guess Version 1 Look at text presentation. UUID version is fourth digit in second group IIRC. -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] What type of UUID we use in Firebird ? i guess Version 1
On Fri, 8 Feb 2019 17:44:16 +0200 marius adrian popa wrote: > What type of UUID we use in Firebird ? > I guess Version 1 > > UUIDs in MySQL are really not random > https://news.ycombinator.com/item?id=19085189 But are they meant to be random? I thought the only guarantee was that they are unique. Paul -- Paul Reeves http://www.ibphoenix.com Supporting users of Firebird Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] What type of UUID we use in Firebird ? i guess Version 1
On 08/02/2019 15:55, Paul Reeves wrote: What type of UUID we use in Firebird ? I guess Version 1 UUIDs in MySQL are really not random https://news.ycombinator.com/item?id=19085189 But are they meant to be random? I thought the only guarantee was that they are unique. I seem to remember a discussion on reordering the elements used to make indexing easier precisely because of the heavy grouping of values in some configurations? I seem to think it was at one of the conferences so must have been over ten years ago ... certainly the only requirement was that they were unique across all machines on the network. -- Lester Caine - G8HFL - Contact - https://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - https://lsces.co.uk EnquirySolve - https://enquirysolve.com/ Model Engineers Digital Workshop - https://medw.co.uk Rainbow Digital Media - https://rainbowdigitalmedia.co.uk Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Changing generator's initial value without restarting
On 8-2-2019 11:24, Jiří Činčura wrote: Hi, is there a way to change the intial value of generator/sequence *without* restarting it? Based on its name, I would guess this is the task for 'START WITH' clause in: CREATE OR ALTER SEQUENCE START WITH Unfortunately it behaves as ALTER SEQUENCE RESTART WITH So it doesn't seem that there is any option. Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] What type of UUID we use in Firebird ? i guess Version 1
On 8-2-2019 16:55, Paul Reeves wrote: On Fri, 8 Feb 2019 17:44:16 +0200 marius adrian popa wrote: What type of UUID we use in Firebird ? I guess Version 1 UUIDs in MySQL are really not random https://news.ycombinator.com/item?id=19085189 But are they meant to be random? I thought the only guarantee was that they are unique. I don't think there is such guarantee possible, it is just unlikely (assuming good enough randomness). Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] What type of UUID we use in Firebird ? i guess Version 1
On 8-2-2019 16:46, Dimitry Sibiryakov wrote: 08.02.2019 16:44, marius adrian popa wrote: What type of UUID we use in Firebird ? I guess Version 1 Look at text presentation. UUID version is fourth digit in second group IIRC. The first digit of the third group indicates the version, which is 4. On Windows Firebird calls CoCreateGuid which generates type 4 (although technically, the Windows API doesn't guarantee this). But in any case, I'll also point you to RFC-4122, section 6 (https://tools.ietf.org/html/rfc4122#section-6) which says: """ 6. Security Considerations Do not assume that UUIDs are hard to guess; they should not be used as security capabilities (identifiers whose mere possession grants access), for example. A predictable random number source will exacerbate the situation. Do not assume that it is easy to determine if a UUID has been slightly transposed in order to redirect a reference to another object. Humans do not have the ability to easily check the integrity of a UUID by simply glancing at it. Distributed applications generating UUIDs at a variety of hosts must be willing to rely on the random number source at all hosts. If this is not feasible, the namespace variant should be used. """ Mark -- Mark Rotteveel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
On 08/02/2019 09:29, Alex Peshkoff via Firebird-devel wrote: > On 2/8/19 12:15 PM, Vlad Khorsun wrote: >> 08.02.2019 10:48, Alex Peshkoff via Firebird-devel wrote: >>> On 2/8/19 10:56 AM, Vlad Khorsun wrote: >> Is it time to re-think how UnicodeUtil::getConversionICU() search for ICU library ? >>> >>> Suggestions? >>> The fastest way sooner of all will be storing favorite ICU verson >>> (once detected using long process) somewhere out of databases. But >>> may be some better idea? >> >> I see "icu_versions" parameter at fbintl.conf but it seems not used >> by getConversionICU(). >> Probably it could be used here ? > Chicken and egg problem, fbintl.conf uses default to mean the version used by the conversion. The conversion (not collates) used to be "statically-linked" (to the DLL/SO). It has changed due to different ICU names/versions in Linux, AFAIK. So I think if a settings should be used, it should not be the fbintl.conf, but a new one in firebird.conf. Adriano Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] ODP: CORE-5997
>>I.e. I assume that this problem is raised due to UDR deprecation? >>Why not use UDR (written using any language you like) for complex >>strings manipulatons? Not only. Previously there was no possibility to have PSQL functions and udf was the only option. The main reason to change udf is that, psql function is defined in the database itself and go with it to other servers. Udf/udr must be copied to destination server which most of the time require server stop/start. It also require in some situation to have some function which provide version of udf lib. And this version must be checked if it is compatibile with current version of the database (structure). >>I do not want to say that I hate your idea in general. But IMO in >>curretn state it's not ready for implementation. I understand that this require implementation time and team resources. And as any feature it require future maitenance. But without this, speed of string manipulations is not efficient and database require to have optimized speed in every possible aspect. Regards, Karol Bieniaszewski Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ODP: CORE-5997
On 08/02/2019 16:10, Karol Bieniaszewski wrote: > > > I understand that this require implementation time and team resources. > > And as any feature it require future maitenance. > > But without this, speed of string manipulations is not efficient and > database require to have optimized speed in every possible aspect. > > > > You ask for wrong thing. Instead of ask for a non standard feature that no DBMS made, and that completely changes SQL, you'd better ask for optimized PSQL execution. PSQL engine has none optimization currently. Adriano Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] ODP: ODP: CORE-5997
>>You ask for wrong thing. Instead of ask for a non standard feature that >>no DBMS made, and that completely changes SQL, you'd better ask for >>optimized PSQL execution. >>PSQL engine has none optimization currently. >>Adriano If this change something in string allocation on every change, than maybe it is the way. But this is only one point of this request. Other point is simple string manipulation. Compare: -- VAR_S = VAR_A[5] || VAR_A[3] || VAR_A[1]; Vs VAR_S = SUBSTRING(VAR_A FROM 5 FOR 1) || SUBSTRING(VAR_A FROM 3 FOR 1) || SUBSTRING(VAR_A FROM 1 FOR 1); -- Or better sample VAR_S[5] = VAR_A[3]; VAR_S[3] = VAR_A[5]; Vs VAR_S = SUBSTRING(VAR_S FROM 1 FOR 2) || SUBSTRING(VAR_A FROM 5 FOR 1) || SUBSTRING(VAR_S FROM 4 FOR 1) || SUBSTRING(VAR_A FROM 3 FOR 1) || SUBSTRING(VAR_S FROM 6); -- And i only show some simple sample. How many times you can put wrong calculation in above sample? Especially in the second one? Regards, Karol Bieniaszewski Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Changing generator's initial value without restarting
> So it doesn't seem that there is any option. Seems so. I just wanted confirmation from CORE guys, I didn't missed something. -- Mgr. Jiří Činčura https://www.tabsoverspaces.com/ Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] ICU in HEAD
08.02.2019 12:29, Alex Peshkoff via Firebird-devel wrote: - adapt search knowing existing versions numbers (see http://site.icu-project.org/download) Yes, this should be efficient. I would limit tried versions to only those which are known to work with Firebird. For example, cut off those versions that cannot be loaded from non-ANSI paths or those that contain outdated TZ information. -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
