I'm in the process of creating an ingest workflow that will pull into Kafka topics a number of API endpoints on an hourly basis. I'd like convert them from JSON to AVRO when I bring them in. I have, however, run into a few problems that I haven't been able to figure out and haven't turned anything up through searches. This seems like it would be a fairly common use case of NiFi, so I figured I'd ask around to see what others are doing in these cases.
The first problem that I'm running into is that some of the endpoints have objects of the form: { "metricsPerAgent": { "6453": { "connectedEngagements": 3, "nonInteractiveTotalHandlingTime": 0 }, "6454": { "connectedEngagements": 1, "nonInteractiveTotalHandlingTime": 0 } } } I'm using an UpdateAttribute processor to add a schema that I get from running the object through the InferAvroSchema processor and running the flowfile into a ConvertJSONToAvro processor. There, unfortunately, I'm getting an error with the ConvertJSONToAvro processor not liking the field names being numbers. What do people normally do in cases like these? Thanks!