Nick,
You could use ExecuteScript to manipulate the JSON. The sample ECMAScript
below assumes that you already have the transformed timestamp as an
attribute "timestamp":
var flowFile = session.get();
if (flowFile !== null) {
var StreamCallback =
Java.type("org.apache.nifi.processor.io.StreamCallback");
var IOUtils = Java.type("org.apache.commons.io.IOUtils");
var StandardCharsets = Java.type("java.nio.charset.StandardCharsets");
flowFile = session.write(flowFile, new
StreamCallback(function(inputStream, outputStream) {
var inputJSON = IOUtils.toString(inputStream,
StandardCharsets.UTF_8);
var contentObj = JSON.parse(inputJSON);
contentObj.time = flowFile.getAttribute("timestamp");
outputStream.write(JSON.stringify(contentObj).getBytes(StandardCharsets.UTF_8));
}));
session.transfer(flowFile, REL_SUCCESS);
}
You could also do the date conversion directly in Javascript with something
like:
contentObj.time = new Date(contentObj.time).valueOf() / 1000;
Thanks,
James
On Mon, Jan 9, 2017 at 1:52 PM, Nick Carenza <
[email protected]> wrote:
> Hey folks,
>
> I am having a hard time figuring out how to work with date values in json
> documents using the standard processors available in Nifi.
>
> example flowfile:
>
> {
> "time": "2017-01-01T01:14:55+00:00",
> "any": {
> "nested": "data"
> }
> }
>
> what i want:
>
> {
> "time": 1483233295,
> "any": {
> "nested": "data"
> }
> }
>
> I can extract the time field into an attribute with EvaluateJsonPath and
> then using nifi expression language I can get a timestamp in seconds:
>
> ${time:toDate("yyyy-MM-dd'T'hh:mm:ss'+00:00'"):toNumber():divide(1000)}
>
> I can't find a way to get that value back into my document without losing
> nested data structures (AttributesToJson).
>
> I can't find anything in jolt-core that enables working with dates.
>
> Has anyone else been able to solve a transformation like this or know of a
> way to do it with standard nifi processors?
>
> If not? What options for enhancement seem best?
>
> - Enhance EvaluateJsonPath to be able to write to individual properties in
> a json document. Currently it can only overwrite the flowfile with the
> value of a single jsonpath expression.
>
> example properties:
> $.time : ${time:toDate("yyyy-MM-dd'T'hh:mm:ss'+00:00'"):toNumber():
> divide(1000)}
>
> - Extend Jolt with a customer transformer class
>
> - Enhance AttributesToJson to allow writing nested data structures. Behind
> a flag, each attribute value would be tested to see if it is valid json.
>
> Thanks,
> Nick
>
