[
https://issues.apache.org/jira/browse/NIFI-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-4473:
-------------------------------
Description:
A number of enhancements were made to processors like QueryDatabaseTable to
allow for such things as:
- Splitting result sets into multiple flow files (i.e. Max Rows Per Flowfile
property)
- Max number of splits/rows returned (Max fragments)
- Normalizing names to be Avro-compatible
The RDBMS processors also now support Avro logical types, but the version of
Avro needed by the current version of Hive (1.2.1) is Avro 1.7.7, which does
not support logical types.
These enhancements were made to JdbcCommon, but not to HiveJdbcCommon (the Hive
version of the JDBC utils class). Since Hive queries can return even larger
result sets than traditional RDBMS, these properties/enhancements are at least
as valuable to have for SelectHiveQL.
was:
A number of enhancements were made to processors like QueryDatabaseTable to
allow for such things as:
- Splitting result sets into multiple flow files (i.e. Max Rows Per Flowfile
property)
- Max number of splits/rows returned (Max fragments)
- Support for Avro logical types
These enhancements were made to JdbcCommon, but not to HiveJdbcCommon (the Hive
version of the JDBC utils class). Since Hive queries can return even larger
result sets than traditional RDBMS, these properties/enhancements are at least
as valuable to have for SelectHiveQL.
> Add support for large result sets and normalizing Avro names to SelectHiveQL
> ----------------------------------------------------------------------------
>
> Key: NIFI-4473
> URL: https://issues.apache.org/jira/browse/NIFI-4473
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Matt Burgess
> Assignee: Matt Burgess
>
> A number of enhancements were made to processors like QueryDatabaseTable to
> allow for such things as:
> - Splitting result sets into multiple flow files (i.e. Max Rows Per Flowfile
> property)
> - Max number of splits/rows returned (Max fragments)
> - Normalizing names to be Avro-compatible
> The RDBMS processors also now support Avro logical types, but the version of
> Avro needed by the current version of Hive (1.2.1) is Avro 1.7.7, which does
> not support logical types.
> These enhancements were made to JdbcCommon, but not to HiveJdbcCommon (the
> Hive version of the JDBC utils class). Since Hive queries can return even
> larger result sets than traditional RDBMS, these properties/enhancements are
> at least as valuable to have for SelectHiveQL.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
