Jackie-Jiang commented on code in PR #19501:
URL: https://github.com/apache/pinot/pull/19501#discussion_r4262424676
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/readers/PinotSegmentRecordReader.java:
##########
@@ -275,8 +322,13 @@ public void getRecord(int docId, GenericRow buffer) {
for (Map.Entry<String, PinotSegmentColumnReader> entry :
_columnReaderMap.entrySet()) {
String column = entry.getKey();
PinotSegmentColumnReader columnReader = entry.getValue();
+ boolean isPrimaryKey = _primaryKeyColumns.contains(column);
if (!columnReader.isNull(docId)) {
- buffer.putValue(column, columnReader.getValue(docId));
+ Object value = columnReader.getValue(docId);
+ if (isPrimaryKey && value instanceof byte[]) {
+ value = new ByteArray((byte[]) value);
Review Comment:
[P1] Preserve raw BYTES in ordinary reader rows
`getRecord()` also feeds segment rebuilding through `FileFormat.PINOT`:
`SegmentIndexCreationDriverImpl` uses the schema-taking reader constructor, so
BYTES primary keys are now wrapped here. The normal `DataTypeTransformer` then
passes this `ByteArray` to `DataTypeTransformerUtils`, whose source-type
detection classifies it as `OBJECT`; conversion to BYTES throws
`UnsupportedOperationException: Cannot convert value from OBJECT to BYTES`.
With `continueOnError`, the transformer replaces the valid key with null
instead. A focused probe using these reviewed sources confirmed the conversion
succeeds for raw bytes and fails for the wrapped value.
Please retain `byte[]` in ordinary reader rows and normalize the row at
`MemoryOptimizedDimensionTable.getRow()`, alongside the existing
`getValue()`/`getValues()` normalization. This also fixes
`testPinotSegmentRecordReaderReadPrimaryKeysOnly()`: its `next()` call now
returns a `ByteArray`, so the `(byte[])` cast at test line 215 throws for the
same reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
