Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/1891
As @joewitt suggested on JIRA ticket, even if a message contains multiple
records, the message probably should be treated as one. In order to do so, they
will define a schema that has a compound array object as the top level object.
I agree with that.
However, if an user is going to split it at downstream, they need another
schema to treat each items within the top-level array as individual record. If
ConsumeKafkaRecord can do that as it consumes messages, it may be better.
Ideally, I prefer ConsumeKafkaRecord to support both ways, by parsing a message
as a message, or split each records in a message.
I intended to support both mode in this PR. Please let me know if we have
strong reason not to do this.
By the way, I did try to setup a schema in AvroSchemaRegistry as follows:
```
{
"type": "array",
"items": {
"type": "record", "name": "temp",
"fields": [
{"name": "value", "type": "string"}
]
}
}
```
While this is a valid schema in Avro, current NiFi AvroSchemaRegistry
doesn't allow this as it requires the top level object to be a record. I got
"Not a record" validation error when [AvroTypeUtil tried to call
getFields](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java#L218):
I think this should be addressed by another JIRA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
