Hello, As the subject line states I am a new to NiFi. For my first flow I am attempting to replace my python code with a NiFi workflow that extracts data from a JSON endpoint and then writes the JSON to a Redshift database.schema.table.
I have set up my DBCP for redshift and I have assigned that to my PutDatabaseRecord processor; however, I am having (at least) two issues so far. 1. How do I tell InvokeHTTP (OR PutDatabaseRecord) to start reading the JSON from a specific point, most json responses are in the form of result.json()['data'] or result.json()['content']? 2. What is the best approach to having the PutDatabaseRecord go to a "Create Table" when the table and thus columns do not exist? Is there a way to have PutDatabaseRecord create the table if it doesn't exist? As a side note to #2, the PutDatabaseRecord is reporting "PutDatabaseRecord failed to process standardflowfilerecord.... due to None of the fields in the record map to the columns defined by the schema.table table." On failures the JSON contents are being written to a file so I am able to see evidence that InvokeHTTP is working and that PutDatabaseRecord has at least looked at / for the schema.table. Thank you!
