[
https://issues.apache.org/jira/browse/NIFI-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433075#comment-15433075
]
Matt Burgess commented on NIFI-2622:
------------------------------------
As it turns out, Hive returns complex types as strings but sets the JDBC type
"appropriately" (STRUCT, ARRAY, JAVA_OBJECT for Map, etc.). So at a minimum to
support complex types in SelectHiveQL, HiveJdbcCommon could add these types to
the String processing case in createSchema(). Please note that since Hive is
returning strings, the Avro fields will contain strings (not records, arrays,
etc.). In order to support that, we'd need to add a property for the user to
provide an Avro schema, then parse the string trying to deconstruct it into
elements according to the schema. That should be a separate improvement Jira if
desired.
> SelectHiveQL with Avro output doesn't handle complex types
> ----------------------------------------------------------
>
> Key: NIFI-2622
> URL: https://issues.apache.org/jira/browse/NIFI-2622
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> Hive supports table columns of complex type, such as Arrays, Structs, Maps,
> etc. However when using SelectHiveQL with Avro output, the result set is not
> being converted correctly and an error occurs.
> This is a fundamental issue with the way SelectHiveQL attempts to create an
> Avro schema without having seen any of the rows, since the sub-types of
> complex items cannot be determined from the SQL type alone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
