costin commented on code in PR #16355:
URL: https://github.com/apache/lucene/pull/16355#discussion_r3529236368
##########
lucene/core/src/java/org/apache/lucene/analysis/CharacterUtils.java:
##########
@@ -53,10 +53,19 @@ public static CharacterBuffer newCharacterBuffer(final int
bufferSize) {
public static void toLowerCase(final char[] buffer, final int offset, final
int limit) {
assert buffer.length >= limit;
assert 0 <= offset && offset <= buffer.length;
- for (int i = offset; i < limit; ) {
- i +=
- Character.toChars(
- Character.toLowerCase(Character.codePointAt(buffer, i, limit)),
buffer, i);
+ for (int i = offset; i < limit; i++) {
+ char c = buffer[i];
+ if (c > 127) { // non-ASCII: switch to full Unicode path from here onward
+ for (int j = i; j < limit; ) {
+ j +=
+ Character.toChars(
+ Character.toLowerCase(Character.codePointAt(buffer, j,
limit)), buffer, j);
+ }
+ return;
+ }
+ if (c >= 'A' && c <= 'Z') {
+ buffer[i] = (char) (c | 32); // set bit 5 to lowercase ASCII
Review Comment:
I had it as an addition initially however ended for the bit operation since
I found it easier to reason about being safe (no overflow) regardless at the
value of c.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
