Github user joewitt commented on the issue:
https://github.com/apache/nifi/pull/1695
one more observation and cannot recall if I mentioned this yet. But here
is some original JSON we'd get from provenance. This is before turning into
avro and then turning back into json.
{code}
{
"eventId":"3ddbed75-c436-4112-a2bd-bbdd05320b23",
"eventOrdinal":13720400,
"eventType":"CREATE",
"timestampMillis":1493439571667,
"timestamp":"2017-04-29T04:19:31.667Z",
"durationMillis":-1,
"lineageStart":1493439571667,
"componentId":"b259f36c-015b-1000-3ca6-c664e156cdf7",
"componentType":"GenerateFlowFile",
"componentName":"GenerateFlowFile",
"entityId":"2ab3c2d8-cb45-44ae-bd45-72080aff3e78",
"entityType":"org.apache.nifi.flowfile.FlowFile",
"entitySize":0,
"updatedAttributes":
{
"path":"./",
"uuid":"2ab3c2d8-cb45-44ae-bd45-72080aff3e78",
"filename":"464576540850588"
},
"previousAttributes":{},
"actorHostname":"10.0.0.15",
"contentURI":"http://10.0.0.15:8080/nifi-api/provenance-events/13720400/content/output";,
"previousContentURI":"http://10.0.0.15:8080/nifi-api/provenance-events/13720400/content/input";,
"parentIds":[],
"childIds":[],
"platform":"nifi",
"application":"NiFi Flow"
}
{code}
Here is the after avro and then back to json view
{code}
{
"eventId" : "edf16cc9-b50d-4cbb-9356-58eb4cf8fd3e",
"eventOrdinal" : 13585491,
"eventType" : "CREATE",
"timestampMillis" : "2017-04-29 04:18:50",
"durationMillis" : -1,
"lineageStart" : "2017-04-29 04:18:50",
"details" : null,
"componentId" : "b259f36c-015b-1000-3ca6-c664e156cdf7",
"componentType" : "GenerateFlowFile",
"componentName" : "GenerateFlowFile",
"entityId" : "ef12f52c-4ae4-46be-bc45-698200f08e54",
"entityType" : "org.apache.nifi.flowfile.FlowFile",
"entitySize" : 0,
"previousEntitySize" : null,
"updatedAttributes" : {
"path" : "./",
"filename" : "464535310815392",
"uuid" : "ef12f52c-4ae4-46be-bc45-698200f08e54"
},
"previousAttributes" : { },
"actorHostname" : "10.0.0.15",
"contentURI" :
"http://10.0.0.15:8080/nifi-api/provenance-events/13585491/content/output";,
"previousContentURI" :
"http://10.0.0.15:8080/nifi-api/provenance-events/13585491/content/input";,
"parentIds" : [ ],
"childIds" : [ ],
"platform" : "nifi",
"application" : "NiFi Flow",
"transitUri" : null
}
{code}
Notice timestamp millis. Seems like we should let json and csv writing use
the timestamp like it would be in an avro 'timestamp-millis' sort of way rather
than a formatted date/time string. We'd want to do this consistently on
reading and writing. The current default changes its representation i think
and makes it a bit tougher to do math with.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
