[
https://issues.apache.org/jira/browse/NIFI-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16130698#comment-16130698
]
ASF GitHub Bot commented on NIFI-4004:
--------------------------------------
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/1877
@ijokarumawak it does look good now. Unfortunately, though, it looks like
it needs to be rebased. It looks like this is a rather large PR, though, that
touches a lot of different points. And it looks like there are several new
processors, controller services, etc. that have been merged to master since the
PR was created. I wonder if it may make sense, instead of changing
RecordReaderFactory to take `Map<String, String>` variables instead of
`FlowFIle flowFile`, I wonder if it makes sense to add both methods to the
interface, and then have a default method in the interface, such as:
```
default RecordReader createRecordReader(FlowFile flowFile, InputStream in,
ComponentLog logger) throws MalformedRecordException, IOException,
SchemaNotFoundException {
return createRecordReader(flowFile == null ? Collections.emptyMap()
: flowFile.getAttributes(), in, logger);
}
RecordReader createRecordReader(Map<String, String> variables,
InputStream in, ComponentLog logger) throws MalformedRecordException,
IOException, SchemaNotFoundException;
```
Thoughts?
> Refactor RecordReaderFactory and SchemaAccessStrategy to be used without
> incoming FlowFile
> ------------------------------------------------------------------------------------------
>
> Key: NIFI-4004
> URL: https://issues.apache.org/jira/browse/NIFI-4004
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.2.0
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
>
> Current RecordReaderFactory and SchemaAccessStrategy implementation assumes
> there's always an incoming FlowFile available, and use it to resolve Record
> Schema.
> That is fine for components those convert or update incoming FlowFiles,
> however there are other components those does not have any incoming
> FlowFiles, for example, ConsumeKafkaRecord_0_10. Typically, ones fetches data
> from external system do not have incoming FlowFile. And current API doesn't
> fit well with these as it requires a FlowFile.
> In fact, [ConsumeKafkaRecord creates a temporal
> FlowFile|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L426]
> only to get RecordSchema. This should be avoided as we expect more
> components start using Record reader mechanism.
> This JIRA proposes refactoring current API to allow accessing RecordReaders
> without needing an incoming FlowFile.
> Additionally, since there's Schema Access Strategy that requires incoming
> FlowFile containing attribute values to access schema registry, it'd be
> useful if we could tell user when such RecordReader is specified that it
> can't be used.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
