sunchao commented on PR #36427:
URL: https://github.com/apache/spark/pull/36427#issuecomment-1115746980
Looks like `CodeGenerator.getValueFromVector` and `CodeGenerator.getValue`
needs to be updated since previously a `ColumnVector` won't be UDT type, but
now it can. If input `dataType` is UDT:
```scala
def getValueFromVector(vector: String, dataType: DataType, rowId: String):
String = {
if (dataType.isInstanceOf[StructType]) {
// `ColumnVector.getStruct` is different from `InternalRow.getStruct`,
it only takes an
// `ordinal` parameter.
s"$vector.getStruct($rowId)"
} else {
getValue(vector, dataType, rowId)
}
```
this will call `getValue` instead, and
```scala
def getValue(input: String, dataType: DataType, ordinal: String): String =
{
val jt = javaType(dataType)
dataType match {
case _ if isPrimitiveType(jt) =>
s"$input.get${primitiveTypeName(jt)}($ordinal)"
case t: DecimalType => s"$input.getDecimal($ordinal, ${t.precision},
${t.scale})"
case StringType => s"$input.getUTF8String($ordinal)"
case BinaryType => s"$input.getBinary($ordinal)"
case CalendarIntervalType => s"$input.getInterval($ordinal)"
case t: StructType => s"$input.getStruct($ordinal, ${t.size})"
case _: ArrayType => s"$input.getArray($ordinal)"
case _: MapType => s"$input.getMap($ordinal)"
case NullType => "null"
case udt: UserDefinedType[_] => getValue(input, udt.sqlType, ordinal)
case _ => s"($jt)$input.get($ordinal, null)"
}
}
```
and `getValue` will recursively call itself on `udt.sqlType` which is struct
type, and thus will call `$input.getStruct($ordinal, ${t.size})` which will
fail with the above error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
