Hi Mike, Personally, I'm not familiar with DynamoDB either, but a quick look at the source & configuration do indeed show that it expects the user to fetch only a single 'attribute' from a Dynamo Item. Looking at the AWS javadocs, it appears that this is the typical usage, as they don't seem to provide a method for obtaining the entire item as JSON (unless I'm missing something).
One approach, though, would be to allow the 'Json Document attribute' property to be optional instead of required and if not specified, simply iterate over all of the keys in the Dynamo DB Item and create a JSON object ourselves that we write to the content of the FlowFile. Often what happens, though, is that a member of the NiFi community develops a processor to provide the functionality that they need and sometimes makes the processor a little more 'generic' to handle additional use cases, if it makes sense. As the community begins to use the processor, it begins to evolved over time to meet more and more use cases. This is a fairly new processor and so likely will evolve over time to meet more & more use cases. I think the use case of pulling back an entire DynamoDB item is very reasonable and likely will be needed by others. Is there any chance that you're interested in working on building out that capability and contributing it to the community? We would be more than happy to help shepherd you through the process of contributing it back to make NiFi's dynamo support better! Thanks -Mark > On Jul 5, 2016, at 8:51 AM, Mike Connolly <[email protected]> wrote: > > James, > Hmmm. I'm not too familiar with DynamoDB or NiFi, but having a processor > that can only return one attribute at a time doesn't seem very useful. I > would think the processor would retrieve all the attributes and package them > into one variable and send it back as one unit so when the user configures > the processor they could do something like set the json attribute value to > "ALL" or something like that. I tried modifying the processor source code to > call item.getJSON() to populate the return value with everything, but > something isn't liking { brace in the string value that gets sent back. Its > complaining about unacceptable characters in the string. I might need to > swich to MongoDB as my datasource and see if that works. > > Thanks, > > Mike > > On 07/01/2016 02:41 AM, James Wing wrote: >> Mike, >> >> I believe it was an intentional design choice to separate the hash/range >> keys from the flowfile content for validation and handling. There is >> currently no option in PutDynamoDB or GetDynamoDB to match the entire >> flowfile content to the entire DynamoDB item, but I believe that is >> certainly a possible extension. >> >> Thanks, >> >> James >> >> On Thu, Jun 30, 2016 at 9:57 AM, Mike Connolly <[email protected] >> <mailto:[email protected]>> wrote: >> I checked out the master branch of Nifi and noticed the GetDynamoDB >> processor which was something I really needed for a workflow. I tried using >> it to return the json representation of a DynamoDB item. It looks like it >> will only return a json representation of a a single attribute of a DynamoDB >> item. I'm wondering it this was the intention or if there is a specific >> value you need to set in the processor configuration that will return the >> whole item. Anybody ever use this? >> >> Thanks, >> >> Mike Connolly >> >
