siddharthteotia commented on code in PR #9004:
URL: https://github.com/apache/pinot/pull/9004#discussion_r916343000
##########
pinot-core/src/main/java/org/apache/pinot/core/common/evaluators/DefaultJsonPathEvaluator.java:
##########
@@ -90,7 +90,7 @@ public <T extends ForwardIndexReaderContext> void
evaluateBlock(int[] docIds, in
}
}
} else {
- switch (reader.getValueType()) {
+ switch (reader.getStoredType()) {
Review Comment:
This is where I am having mixed feelings about this change. The case
statement goes over JSON as well which is not a storedType. JSON is logical
type and uses STRING as stored type.
https://github.com/apache/pinot/blob/master/pinot-spi/src/main/java/org/apache/pinot/spi/data/FieldSpec.java#L393
`JSON(STRING, false, false),`
Partly I think the change is right because when reading values out of
forward index reader, we should do it based on the `storedType` that dictates
how the bytes are stored. In that case, code like below should not reference
JSON or any other logical type. So, additional cleanup is also needed but want
to make sure no assumptions made in the code about logical type v/s storedType
are broken.
##########
pinot-core/src/main/java/org/apache/pinot/core/common/evaluators/DefaultJsonPathEvaluator.java:
##########
@@ -90,7 +90,7 @@ public <T extends ForwardIndexReaderContext> void
evaluateBlock(int[] docIds, in
}
}
} else {
- switch (reader.getValueType()) {
+ switch (reader.getStoredType()) {
Review Comment:
Secondly, if we make this change do we still need `getValueType` anywhere in
the code to get the actual logical datatype of the column ? I guess no because
ForwardIndexReader and it's users should only need to work with storedType
except for DefaultIndexReaderProvider code ?
Also, for columns like BOOLEAN / TIMESTAMP, this change should have ideally
failed some tests whose storedType is INT / LONG because now the return value
is different
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
