clintropolis commented on code in PR #13732:
URL: https://github.com/apache/druid/pull/13732#discussion_r1095437897
##########
processing/src/main/java/org/apache/druid/segment/virtual/NestedFieldVirtualColumn.java:
##########
@@ -258,18 +299,37 @@ public ColumnValueSelector<?> makeColumnValueSelector(
ReadableOffset offset
)
{
- final NestedDataComplexColumn column =
NestedDataComplexColumn.fromColumnSelector(columnSelector, this.columnName);
- if (column == null) {
+ ColumnHolder holder = columnSelector.getColumnHolder(this.columnName);
+ if (holder == null) {
return NilColumnValueSelector.instance();
}
+ BaseColumn theColumn = holder.getColumn();
- // processFromRaw is true, that means JSON_QUERY, which can return partial
results, otherwise this virtual column
- // is JSON_VALUE which only returns literals, so we can use the nested
columns value selector
- return processFromRaw
- ? new
RawFieldColumnSelector(column.makeColumnValueSelector(offset), parts)
- : hasNegativeArrayIndex
- ? new
RawFieldLiteralColumnValueSelector(column.makeColumnValueSelector(offset),
parts)
- : column.makeColumnValueSelector(parts, offset);
+ if (processFromRaw || hasNegativeArrayIndex) {
+ // if the path has negative array elements, or has set the flag to
process 'raw' values explicitly (JSON_QUERY),
+ // then we use the 'raw' processing of the
RawFieldColumnSelector/RawFieldLiteralColumnValueSelector created
+ // with the column selector factory instead of using the optimized
nested field column
+ return null;
+ }
Review Comment:
ah maybe I could explain why a bit better here. the reason is that jsonpath
expressions with a negative array element mean "from the end of the array", so
`$.x[-1]` returns the last element of the array x, and so on. We don't
currently have an easy way to know how many elements are in an array for each
row to be able to use the nested literal columns for those elements, so
`hasNegativeArrayIndex` forces us to fall back to the 'raw' data processing
which can get the whole array and find the appropriate element that way instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
