dweiss commented on code in PR #16355:
URL: https://github.com/apache/lucene/pull/16355#discussion_r3527941960
##########
lucene/core/src/java/org/apache/lucene/analysis/CharacterUtils.java:
##########
@@ -53,10 +53,19 @@ public static CharacterBuffer newCharacterBuffer(final int
bufferSize) {
public static void toLowerCase(final char[] buffer, final int offset, final
int limit) {
assert buffer.length >= limit;
assert 0 <= offset && offset <= buffer.length;
- for (int i = offset; i < limit; ) {
- i +=
- Character.toChars(
- Character.toLowerCase(Character.codePointAt(buffer, i, limit)),
buffer, i);
+ for (int i = offset; i < limit; i++) {
+ char c = buffer[i];
+ if (c > 127) { // non-ASCII: switch to full Unicode path from here onward
+ for (int j = i; j < limit; ) {
+ j +=
+ Character.toChars(
+ Character.toLowerCase(Character.codePointAt(buffer, j,
limit)), buffer, j);
+ }
+ return;
+ }
+ if (c >= 'A' && c <= 'Z') {
+ buffer[i] = (char) (c | 32); // set bit 5 to lowercase ASCII
Review Comment:
Clever. I wonder if `c + ('a' - 'A')` wouldn't be more intuitive though.
It's fine with the comment.
##########
lucene/core/src/java/org/apache/lucene/analysis/CharacterUtils.java:
##########
@@ -53,10 +53,19 @@ public static CharacterBuffer newCharacterBuffer(final int
bufferSize) {
public static void toLowerCase(final char[] buffer, final int offset, final
int limit) {
assert buffer.length >= limit;
assert 0 <= offset && offset <= buffer.length;
- for (int i = offset; i < limit; ) {
- i +=
- Character.toChars(
- Character.toLowerCase(Character.codePointAt(buffer, i, limit)),
buffer, i);
+ for (int i = offset; i < limit; i++) {
+ char c = buffer[i];
+ if (c > 127) { // non-ASCII: switch to full Unicode path from here onward
+ for (int j = i; j < limit; ) {
Review Comment:
Can we just continue to reuse i, without declaring another variable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
