Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21952#discussion_r207102304
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala ---
@@ -100,13 +100,14 @@ class AvroSerializer(rootCatalystType: DataType,
rootAvroType: Schema, nullable:
et, resolveNullableType(avroType.getElementType, containsNull))
(getter, ordinal) => {
val arrayData = getter.getArray(ordinal)
- val result = new java.util.ArrayList[Any]
+ val len = arrayData.numElements()
+ val result = new Array[Any](len)
--- End diff --
one more improvement: if the element is primitive type, we can call
`arrayData.toBoolean/Int/...Array` directly.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
