Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-29 Thread Luis Forra
nonomura escreveu no dia segunda, 29/10/2018 à(s) 05:47: > > Luis, > > Thank you very much! > I'll examine it further and let yall know if found something. > > Hiro You're Welcome I believe the problem is in code that decides if the index is a full match or a partial match, somehow in the case

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-28 Thread nonomura
Luis, Thank you very much! I'll examine it further and let yall know if found something. Hiro (2018/10/27 21:34), Luis Forra wrote: I made the changes that you suggested: and tested in Linux x64, didn't solve the problem Firebird-Devel mailing list, web interface at

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-27 Thread Luis Forra
nonomura escreveu no dia sexta, 26/10/2018 à(s) 03:00: > > > > > > > > > >> https://github.com/Alexpux/firebird-git-svn/blob/master/src/common/unicode_util.cpp#L1334 > > > > > > > > > Just a heads up: that is an outdated clone of an old repository that > > > was synced from subversion and

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-25 Thread nonomura
https://github.com/Alexpux/firebird-git-svn/blob/master/src/common/unicode_util.cpp#L1334 Just a heads up: that is an outdated clone of an old repository that was synced from subversion and hasn't been updated since 2015. The Firebird repository is on

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-25 Thread nonomura
Adriano You don't need to be on the development team. You know, you fix, you send pull request. Please put aside those nonsense and pointless arguments. Shall we discuss in a constructive way? Hiro Firebird-Devel mailing list, web interface at

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-25 Thread Mark Rotteveel
On 24-10-2018 08:20, nonomura wrote: The source code cited below clearly tells the root of the problem that I reported. [..] https://github.com/Alexpux/firebird-git-svn/blob/master/src/common/unicode_util.cpp#L1334 Just a heads up: that is an outdated clone of an old repository that was

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-25 Thread Adriano dos Santos Fernandes
On 25/10/2018 00:01, nonomura wrote: > > (2018/10/24 17:51), Adriano dos Santos Fernandes wrote: >> As I said for you *many* times: if UNICODE (sensitive) has problems, as >> it has, > Nobody seems supporting your assumption. > Ok then. >> how can you changing code under the insensitive flags

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-24 Thread nonomura
(2018/10/24 17:51), Adriano dos Santos Fernandes wrote: As I said for you *many* times: if UNICODE (sensitive) has problems, as it has, Nobody seems supporting your assumption. how can you changing code under the insensitive flags condition will fix the problem? If I were one of the

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-24 Thread Luis Forra
> >Adriano dos Santos Fernandes escreveu no dia quarta, 24/10/2018 à(s) 11:52: > > > As I said for you *many* times: if UNICODE (sensitive) has problems, as > > it has, how can you changing code under the insensitive flags condition > > will fix the problem? > UNICODE collation doesn't

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-24 Thread Dimitry Sibiryakov
24.10.2018 12:51, Adriano dos Santos Fernandes wrote: if UNICODE (sensitive) has problems, as it has I cannot remember that in this thread anyone said that it has problems. Subject is "Case (and accent) *insensitive* ICU collations". -- WBR, SD. Firebird-Devel mailing list, web

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-24 Thread Adriano dos Santos Fernandes
As I said for you *many* times: if UNICODE (sensitive) has problems, as it has, how can you changing code under the insensitive flags condition will fix the problem? Adriano On 24/10/2018 03:20, nonomura wrote: > The source code cited below clearly tells the root of the problem that > I

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-24 Thread nonomura
The source code cited below clearly tells the root of the problem that I reported. The problem should be resolved by setting sortCollator's UCOL_STRENGTH property as well as compareCollator. There are two collater objects relating to this problem; Those are sortCollator(*A) and

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
(2018/10/19 20:18), Dimitry Sibiryakov wrote: AFAIK, sorting key is just an array of bytes and sorting comparison routine treat it as an array of bytes. Key generation routine transform any data into an array of bytes and after that operation source data type is irrelevant. I understood

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread Dimitry Sibiryakov
19.10.2018 14:03, Adriano dos Santos Fernandes wrote: ICU generates sort key for a single string. For multi segment (columns) index/sort, Firebird call ICU (or any collation) for each column and join the generated fully (with all levels/strengths) sort keys. AFAIK, sorting key is just an

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
(2018/10/19 19:50), Adriano dos Santos Fernandes wrote: I already showed to you that UNICODE (sensitive) has the problem, and it does not nor can't use this condition. If you wrote that way, I have no words other than that I could not see the problem in UNICODE (sensitive) and nobody

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread Adriano dos Santos Fernandes
On 19/10/2018 09:42, nonomura wrote: > (2018/10/19 19:03), Adriano dos Santos Fernandes wrote: >> ICU generates sort key for a single string. For multi segment >> (columns) index/sort, Firebird call ICU (or any collation) for each >> column and join the generated fully (with all levels/strengths)

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
(2018/10/19 19:03), Adriano dos Santos Fernandes wrote: ICU generates sort key for a single string. For multi segment (columns) index/sort, Firebird call ICU (or any collation) for each column and join the generated fully (with all levels/strengths) sort keys. To generate interleaved keys and

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread Adriano dos Santos Fernandes
On 19/10/2018 06:24, nonomura wrote: > Adriano, > > I've read your comment again, carefully, and understood what you were > thinking as > "the root of the problem". > > But I could not understand how it is relating to the case that I > reported. > > ICU library generates sort keys for those three

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
Adriano, I've read your comment again, carefully, and understood what you were thinking as "the root of the problem". But I could not understand how it is relating to the case that I reported. ICU library generates sort keys for those three collations and Firebird uses it. Firebird

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
>>I'm really trying to communicate with you here and in the tracker I hate to tell you this but I don't think so. You always wrote without reading my comment carefully with your prejudice, just like my first reply to you here :) >>you seem to answer every point of mine as rewriting what I

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread Adriano dos Santos Fernandes
I'm really trying to communicate with you here and in the tracker but you seem to answer every point of mine as rewriting what I wrote giving another meaning. Or it seems you're so expert on the subject, so please send pull request. It's very easy. Adriano On 18/10/2018 17:06, nonomura wrote:

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
>the root of the problem of multi-level and multi-segment with it. Sorry, it's like I missed the very point you meant. Ok, you will see that it is not the problem of multi-level and multi-segment if you examine the exact sort key in that site. The root of the problem should be much simpler as

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
Hi, >then When? :) Sort keys seem to be stored fully(body,case,accent) for utf8 fields. And compared fully where case or case and accent should not be compared. That's what I wrote. Regards, Hiro Please visit this site: (http://demo.icu-project.org/icu-bin/collation.html ) and try to see

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
>No. Base letter is most important. Sorry, I just meant that base letter's importance does not change anytime. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
>I really don't understand why you emphasize so much on case-/accent-insensitivity Because it is a bug. that's all. >Case is less important than base letter. No. Base letter is most important. a1 A1 a2 A2 ... This is correct result as they are ordered by CSAS in icu collation rule. If case

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread liviuslivius
Hi, maybe naive. But why keys ar not stored in uppercase then? Regards,Karol Bieniaszewski nullFirebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread Adriano dos Santos Fernandes
I really don't understand why you emphasize so much on case-/accent-insensitivity, and say UNICODE is ok, as we can easily demonstrate the root of the problem of multi-level and multi-segment with it. Yes, there is inconsistencies with insensitive with/without unique, but root of the problem

[Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
Hi there, I would like to ask someone who can check and correct if bug was found to see the relating part of source code regarding sorting on UTF8 + ICU collations. The symptom and what I tested and confirmed have commented in CORE-5940. The following is the summary: 1. UNICODE_CI and