Copilot commented on code in PR #12448:
URL: https://github.com/apache/gluten/pull/12448#discussion_r3558209944
##########
gluten-arrow/src/main/java/org/apache/gluten/vectorized/ArrowWritableColumnVector.java:
##########
@@ -265,8 +263,12 @@ private void createVectorAccessor(ValueVector vector,
ValueVector dictionary) {
accessor = new DecimalAccessor((DecimalVector) vector);
} else if (vector instanceof VarCharVector) {
accessor = new StringAccessor((VarCharVector) vector);
+ } else if (vector instanceof LargeVarCharVector) {
+ accessor = new LargeStringAccessor((LargeVarCharVector) vector);
} else if (vector instanceof VarBinaryVector) {
Review Comment:
`createVectorAccessor` now supports
`LargeVarCharVector`/`LargeVarBinaryVector` for non-dictionary columns, but the
dictionary-vector fast path above still only recognizes
`VarCharVector`/`VarBinaryVector`. If Arrow produces dictionary-encoded
`LargeUtf8`/`LargeBinary` arrays (which is valid when
`spark.sql.execution.arrow.useLargeVarTypes=true`), this will throw
`Unrecognized index value type` during load. Please add dictionary support for
`LargeVarCharVector`/`LargeVarBinaryVector` (e.g.,
`DictionaryEncodedLargeStringAccessor` using `NullableLargeVarCharHolder`, and
`DictionaryEncodedLargeBinaryAccessor` using `LargeVarBinaryVector#getObject`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
