Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-28 Thread nonomura
Luis, Thank you very much! I'll examine it further and let yall know if found something. Hiro (2018/10/27 21:34), Luis Forra wrote: I made the changes that you suggested: and tested in Linux x64, didn't solve the problem Firebird-Devel mailing list, web interface at

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-25 Thread nonomura
https://github.com/Alexpux/firebird-git-svn/blob/master/src/common/unicode_util.cpp#L1334 Just a heads up: that is an outdated clone of an old repository that was synced from subversion and hasn't been updated since 2015. The Firebird repository is on

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-25 Thread nonomura
Adriano You don't need to be on the development team. You know, you fix, you send pull request. Please put aside those nonsense and pointless arguments. Shall we discuss in a constructive way? Hiro Firebird-Devel mailing list, web interface at

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-24 Thread nonomura
(2018/10/24 17:51), Adriano dos Santos Fernandes wrote: As I said for you *many* times: if UNICODE (sensitive) has problems, as it has, Nobody seems supporting your assumption. how can you changing code under the insensitive flags condition will fix the problem? If I were one of the

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-24 Thread nonomura
The source code cited below clearly tells the root of the problem that I reported. The problem should be resolved by setting sortCollator's UCOL_STRENGTH property as well as compareCollator. There are two collater objects relating to this problem; Those are sortCollator(*A) and

Re: [Firebird-devel] [FB-Tracker] Created: (CORE-5947) UTF8 Collation UNICODE_CI_AI not working as expected causing poor performance in database

2018-10-19 Thread nonomura
(2018/10/19 18:32), Luis Forra (JIRA) wrote: The biggest problem is with foreign keys I didn't noticed this. I think the root of this problem is relating to CORE-5940. Both of them are causes of headache for me... Nowadays the importance of UTF8 is escalating. But Firebird has problem

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
(2018/10/19 20:18), Dimitry Sibiryakov wrote: AFAIK, sorting key is just an array of bytes and sorting comparison routine treat it as an array of bytes. Key generation routine transform any data into an array of bytes and after that operation source data type is irrelevant. I understood

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
(2018/10/19 19:50), Adriano dos Santos Fernandes wrote: I already showed to you that UNICODE (sensitive) has the problem, and it does not nor can't use this condition. If you wrote that way, I have no words other than that I could not see the problem in UNICODE (sensitive) and nobody

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
(2018/10/19 19:03), Adriano dos Santos Fernandes wrote: ICU generates sort key for a single string. For multi segment (columns) index/sort, Firebird call ICU (or any collation) for each column and join the generated fully (with all levels/strengths) sort keys. To generate interleaved keys and

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
1 A 2 Adriano On 18/10/2018 14:37, nonomura wrote: Hi there, I would like to ask someone who can check and correct if bug was found to see the relating part of source code regarding sorting on UTF8 + ICU collations. The symptom and what I tested and confirmed ha

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-19 Thread nonomura
ith you here and in the tracker but you seem to answer every point of mine as rewriting what I wrote giving another meaning. Or it seems you're so expert on the subject, so please send pull request. It's very easy. Adriano On 18/10/2018 17:06, nonomura wrote: I really don't understand why you empha

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
>the root of the problem of multi-level and multi-segment with it. Sorry, it's like I missed the very point you meant. Ok, you will see that it is not the problem of multi-level and multi-segment if you examine the exact sort key in that site. The root of the problem should be much simpler as

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
Hi, >then When? :) Sort keys seem to be stored fully(body,case,accent) for utf8 fields. And compared fully where case or case and accent should not be compared. That's what I wrote. Regards, Hiro Please visit this site: (http://demo.icu-project.org/icu-bin/collation.html ) and try to see

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
>No. Base letter is most important. Sorry, I just meant that base letter's importance does not change anytime. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
>I really don't understand why you emphasize so much on case-/accent-insensitivity Because it is a bug. that's all. >Case is less important than base letter. No. Base letter is most important. a1 A1 a2 A2 ... This is correct result as they are ordered by CSAS in icu collation rule. If case

[Firebird-devel] Case (and accent) insensitive ICU collations in multiple columns

2018-10-18 Thread nonomura
Hi there, I would like to ask someone who can check and correct if bug was found to see the relating part of source code regarding sorting on UTF8 + ICU collations. The symptom and what I tested and confirmed have commented in CORE-5940. The following is the summary: 1. UNICODE_CI and

[Firebird-devel] [FB-Tracker] Created: (CORE-5940) Sort order is wrong when ordering by multiple columns starting with collate UNICODE_CI or UNICODE_CI_AI

2018-10-12 Thread Hiro Nonomura (JIRA)
/browse/CORE-5940 Project: Firebird Core Issue Type: Bug Components: Charsets/Collation Affects Versions: 3.0.4 Environment: Firebird 2.5 embedded(Win32) and 3.0(Win32), Windows 7 Reporter: Hiro Nonomura If a sorting was ordered by a single