steffenvan commented on code in PR #1071:
URL: https://github.com/apache/jackrabbit-oak/pull/1071#discussion_r1318377448
##########
oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/LuceneDocumentMaker.java:
##########
@@ -315,6 +313,38 @@ protected boolean indexTypeOrderedFields(Document doc,
String pname, int tag, Pr
}
return fieldAdded;
}
+
+ protected static BytesRef checkTruncateLength(String prop, String value,
String path, int maxLength) {
Review Comment:
I think the function does quite a lot of things - and I generally like our
code to follow the SRP. Some minor refactoring can lead to this, which IMO is a
bit clearer. It is more code, but I think it's clearer - especially when we fix
the TODOs for new people to read it.
```java
// TODO: explain why we use 10
private static final int EMERGENCY_TRUNCATION_LENGTH = 10;
...
// assuming we rename checkTruncateLength() -> truncateToMaxLength()
protected static BytesRef truncateToMaxLength(String prop, String value,
String path, int maxLength) {
log.trace("Property {} at path:[{}] has value {}", prop, path, value);
BytesRef ref = new BytesRef(value);
if (ref.length <= maxLength) {
return ref;
}
ref = truncateBytesRef(ref, maxLength);
if (ref.length > maxLength) {
ref = emergencyTruncate(ref, maxLength);
}
return ref;
}
private static BytesRef truncateBytesRef(BytesRef ref, int maxLength) {
log.info("Truncating as length after encoding {} is > {} ", ref.length,
maxLength);
int end = calculateEndPosition(ref, maxLength);
byte[] truncatedBytes = Arrays.copyOf(ref.bytes, end + 1);
BytesRef truncatedRef = new BytesRef(new String(truncatedBytes,
StandardCharsets.UTF_8));
log.trace("Truncated to {}", truncatedRef.utf8ToString());
return truncatedRef;
}
// TODO: explain this more in detail, with examples and why it's necessary.
private static int calculateEndPosition(BytesRef ref, int maxLength) {
int end = maxLength - 1;
while ((ref.bytes[end] & 0b11000000) == 0b10000000) {
end--;
}
if ((ref.bytes[end] & 0b11000000) == 0b11000000) {
end--;
}
return end;
}
// TODO: explain why this is necessary and with examples.
private static BytesRef emergencyTruncate(BytesRef ref, int maxLength) {
log.error("Truncation did not work: still {} bytes", ref.length);
String truncated = ref.utf8ToString();
while (ref.length > maxLength) {
truncated = truncated.substring(0, truncated.length() -
EMERGENCY_TRUNCATION_LENGTH);
ref = new BytesRef(truncated);
}
return ref;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
