[
https://issues.apache.org/jira/browse/NIFI-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952663#comment-17952663
]
Daniel Stieglitz edited comment on NIFI-14579 at 5/19/25 3:56 PM:
------------------------------------------------------------------
[~zhtk] QueryRecord reads all the fields but it limits the results returned
based on the query. A schema is needed for QueryRecord to read all of the data.
Use Starting Row strategy should be used when its known ahead of time there is
little variation of the data between records thereby allowing the schema to be
generated from the first 10 rows without having to read the whole file. When
there is a lot of variation between the records, then the Infer Schema strategy
should be used.
was (Author: JIRAUSER294662):
[~zhtk] QueryRecord reads all the fields but it limits the results returned
based on the query. A schema is needed for QueryRecord to read all of the data.
Use Starting Row strategy should be used when its known ahead of time there is
little variation of the data between records thereby allowing the schema to be
generated from the first 10 rows without having to read the whole file. When
there is variation then the Infer Schema strategy should be used.
> Add parameter to configure number of rows used in schema inference with
> header in ExcelReader service
> -----------------------------------------------------------------------------------------------------
>
> Key: NIFI-14579
> URL: https://issues.apache.org/jira/browse/NIFI-14579
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Piotr Zalas
> Priority: Major
>
> Currently ExcelReader service allows to configure *Schema Access Strategy*
> parameter as {*}Use Starting Row{*}. With this parameter, only 10 first rows
> are used to infer schema of columns in the sheet, as opposed to *Infer
> Schema* strategy.
> My user requests that all rows in the sheet are used to infer schema.
> Moreover, the service is used in QueryRecord processor to limit number of
> rows read (as described in NIFI-14427). The user wants to infer schema only
> from rows they read, not all rows in the sheet.
> It would be great to add parameter to ExcelReader that allows to configure
> number of rows read during schema inference (i.e.
> {{ExcelHeaderSchemaStrategy#NUM_ROWS_TO_DETERMINE_TYPES}} variable). The
> parameter could probably show conditionally based on value of {*}Schema
> Access Strategy{*}. Value 0 could have special meaning that all rows in the
> sheet should be read. The default value could be 10 to preserve existing
> behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
