Chakravarty, Isha, and Jim - thanks very much. I was able to run with your suggestions and build a nice and efficient solution using standard nifi processors. Nothing custom other than some domain-specific regex patterns in the processor configurations. My customers are very pleased. Thanks again for your help. - Jim
On Wed, Mar 9, 2022 at 6:46 PM Jim Williams <[email protected]> wrote: > Found the correct way to escape the “.” In the JOLT spec. The spec to get > exactly the output you want should be: > > > > [ > > { > > "operation": "shift", > > "spec": { > > "bigStructure": { > > "ID": "&", > > "lastUpdated": "&", > > "final_results": { > > "*": { > > "color": "&1\\.&0", > > "temp": "&1\\.&0" > > } > > } > > } > > } > > } > > ] > > > > > > Warm regards, > > > > <https://www.alertlogic.com/> > > *Jim Williams* | Manager, Site Reliability Engineering > > [email protected] | alertlogic.com <http://www.alertlogic.com/> > <https://twitter.com/alertlogic> > <https://www.linkedin.com/company/alert-logic> > > > > > > *From:* Jim Williams > *Sent:* Wednesday, March 9, 2022 5:15 PM > *To:* [email protected] > *Subject:* RE: Transforming json > > > > James – > > > > The JOLT processors can help with these sorts of JSON-JSON transformations. > > > > Given your input: > > > > { > > "bigStructure": { > > "final_results": { > > "Label1": { > > "color": "red", > > "temp": "0C" > > }, > > "AnotherLabel": { > > "color": "blue", > > "temp": "32F" > > }, > > "LastLabel": { > > "color": "black", > > "temp": "0K" > > } > > }, > > "ID": "12345", > > "lastUpdated": "20220309" > > } > > } > > > > Using the JOLT spec: > > > > [ > > { > > "operation": "shift", > > "spec": { > > "bigStructure": { > > "ID": "&", > > "lastUpdated": "&", > > "final_results": { > > "*": { > > "color": "&1-&0", > > "temp": "&1-&0" > > } > > } > > } > > } > > } > > ] > > > > Gives the result: > > > > { > > "ID" : "12345", > > "lastUpdated" : "20220309", > > "Label1-color" : "red", > > "Label1-temp" : "0C", > > "AnotherLabel-color" : "blue", > > "AnotherLabel-temp" : "32F", > > "LastLabel-color" : "black", > > "LastLabel-temp" : "0K" > > } > > > > Note the “-“ in the key names for the results versus “.”. The “.” has > meaning in the spec which I did not take time to noodle over and remove. > > > > There is a demo page where you can try this out yourself [1]. > Documentation for JOLT is available in github [2]. > > > > [1] https://jolt-demo.appspot.com/#inception > > [2] https://github.com/bazaarvoice/jolt > > > > > > Warm regards, > > > > <https://www.alertlogic.com/> > > *Jim Williams* | Manager, Site Reliability Engineering > > [email protected] | alertlogic.com <http://www.alertlogic.com/> > <https://twitter.com/alertlogic> > <https://www.linkedin.com/company/alert-logic> > > > > > > *From:* James McMahon <[email protected]> > *Sent:* Wednesday, March 9, 2022 1:47 PM > *To:* [email protected] > *Subject:* Transforming json > > > > *Caution:* This email originated from outside of the organization. Do not > click links or open attachments unless you recognize the sender and know > the content is safe > > > > Hello. I am not that familiar with NiFi JSON processors, and am hoping > someone can help me with a requirement to transform and parse incoming > json. The biggest challenges for me here are that the keys in my incoming > final_results can vary in number, and I need to use actual key names to > form my json output results. > > > > Here is a dummied up representation of an incoming object: > > { > > "bigStructure":{ > > . > > . > > . > > "final_results": { > > "Label1": { > > "color": "red, > > "temp": "0C" > > }, > > "AnotherLabel": { > > "color": "blue, > > "temp": "32F" > > }, > > . > > . > > . > > "LastLabel": { > > "color": "black, > > "temp": "0K" > > } > > }, > > "ID": "12345", > > "lastUpdated": "20220309" > > } > > } > > > > Here is what I need to generate as my output object: > > { > > "Label1.color": "red", > > "Label1.temp": "0C", > > "AnotherLabel.color": "blue", > > "AnotherLabel.temp": "32F", > > "LastLabel.color": "black", > > "LastLabel.temp": "0K", > > "ID": "12345", > > "lastUpdated": "20220309" > > } > > > > I gather that this transformation needs to happen because they feed that > output structure into a database that expects the data in this format. > > > > Can I do this with "out of the box" nifi processors? Any help is much > appreciated. > > Thank you in advance! > Confidentiality Notice | This email and any included attachments may be > privileged, confidential and/or otherwise protected from disclosure. Access > to this email by anyone other than the intended recipient is unauthorized. > If you believe you have received this email in error, please contact the > sender immediately and delete all copies. If you are not the intended > recipient, you are notified that disclosing, copying, distributing or > taking any action in reliance on the contents of this information is > strictly prohibited. > > *Disclaimer* > > The information contained in this communication from the sender is > confidential. It is intended solely for use by the recipient and others > authorized to receive it. If you are not the recipient, you are hereby > notified that any disclosure, copying, distribution or taking action in > relation of the contents of this information is strictly prohibited and may > be unlawful. > > This email has been scanned for viruses and malware, and may have been > automatically archived by Mimecast, a leader in email security and cyber > resilience. Mimecast integrates email defenses with brand protection, > security awareness training, web security, compliance and other essential > capabilities. Mimecast helps protect large and small organizations from > malicious activity, human error and technology failure; and to lead the > movement toward building a more resilient world. To find out more, visit > our website. >
