Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/10820#discussion_r50787370
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java
---
@@ -170,12 +204,13 @@ public final CalendarInterval getInterval(int
ordinal) {
@Override
public final InternalRow getStruct(int ordinal, int numFields) {
- throw new NotImplementedException();
+ return
ColumnVectorUtils.toRow(parent.column(ordinal).getStruct(rowId));
}
@Override
public final ArrayData getArray(int ordinal) {
- throw new NotImplementedException();
+ ColumnVector.Array array = parent.column(ordinal).getArray(rowId);
+ return ColumnVectorUtils.toGenericArray(array);
--- End diff --
I think for this and getStruct, it will be too slow. We are materializing
the collection which we should avoid if possible.
I think we need to clean up InternalRow's interface and then have getStruct
return that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
