ankitsultana commented on code in PR #16608:
URL: https://github.com/apache/pinot/pull/16608#discussion_r2279711786
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/util/ValueReader.java:
##########
@@ -63,7 +64,7 @@ default byte[] getUnpaddedBytes(int index, int
numBytesPerValue, byte[] buffer)
*/
default String getUnpaddedString(int index, int numBytesPerValue, byte[]
buffer) {
int length = readUnpaddedBytes(index, numBytesPerValue, buffer);
- return new String(buffer, 0, length);
+ return new String(buffer, 0, length, StandardCharsets.UTF_8);
Review Comment:
(not blocking)
String in most JDKs support LATIN1 and UTF-16. My understanding is that most
Latin/English character strings end up using LATIN1 which is reasonably compact
(e.g. UUID is 36 characters).
Would passing UTF_8 switch the coder value to UTF-16 for some other strings
that could have been served with Latin-1? It could lead to a quite a big
increase in heap utilization for such cases, since UTF-16 is almost double the
size of Latin-1.
<img width="753" height="265" alt="image"
src="https://github.com/user-attachments/assets/03ab8977-a2c9-425f-b249-b4a5c4c33a20";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
