abstractdog commented on a change in pull request #2479:
URL: https://github.com/apache/hive/pull/2479#discussion_r690469208
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/VectorizedListColumnReader.java
##########
@@ -129,21 +131,16 @@ private boolean
fetchNextValue(PrimitiveObjectInspector.PrimitiveCategory catego
private void addElement(ListColumnVector lcv, List<Object> elements,
PrimitiveObjectInspector.PrimitiveCategory category, int index) throws
IOException {
lcv.offsets[index] = elements.size();
- // Return directly if last value is null
- if (definitionLevel < maxDefLevel) {
- lcv.isNull[index] = true;
- lcv.lengths[index] = 0;
- // fetch the data from parquet data page for next call
- fetchNextValue(category);
- return;
- }
-
do {
// add all data for an element in ListColumnVector, get out the loop if
there is no data or the data is for new element
+ if (definitionLevel < maxDefLevel) {
+ lcv.lengths[index] = 0;
+ lcv.isNull[index] = true;
Review comment:
fetch next value only returns a single primitive value into a list (e.g.
ListColumnVector.child.vector[i])
in this method, the elements of lists are only collected serially into
List<Object> elements, which will then be filled back into a ListColumnVector
in convertValueListToListColumnVector
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
