Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/2003
@Wesley-Lawrence I definitely agree that having to build up an entire Avro
schema can be a pain -- especially if you're not already familiar with Avro
schemas. And I absolutely love that you saw something that can be improved and
jumped in to improve the user experience! However, I am a bit concerned with
the approach taken here because while it does scratch an itch, it does so only
for CSV data and as a result isn't really consistent.
I would like to start a discussion on how we could perhaps handle this in a
more generic way. My first thought is to actually provide an alternative
implementation of the Schema Registry. Call it
ExplicitStringFieldSchemaRegistry for lack of a better name (i'm suggesting
calling it this for the sake of the discussion, not necessarily creating an
implementation with this name). The idea here, though, is that there would be a
new implementation of SchemaRegistry. In this new implementation you would add
properties just like AvroSchemaRegistry. The name of the property would be the
name of a schema. But the value, instead of an Avro Schema, would be a
comma-separated list of column names, and all would be assumed to be Strings.
This approach gives us a few different benefits.
First, it allows this same approach to be taken with other data formats
(flat JSON, for example). Also, it keeps things consistent in terms of how we
access the schema (we can still use the Schema Name property along with the
Schema Registry). It also makes the CSV Reader more re-usable because we can
use Expression Language to access the name of the schema, so a single CSV
Reader can be used by many different processors spread throughout the flow.
What I would love to see at some point is a more powerful SchemaRegistry
service that provides a Custom/Advanced UI for actually building schemas. I
think this would be extremely powerful and useful and far easier to use. But I
promise that I am the last person that you (or any user) wants building a UI :)
Any thoughts here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
