bqiu created MINIFICPP-403: ------------------------------ Summary: Enable tagging of flowfiles with flow metadata information in C++ Key: MINIFICPP-403 URL: https://issues.apache.org/jira/browse/MINIFICPP-403 Project: NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 1.0.0 Reporter: bqiu Assignee: bqiu Fix For: 1.0.0
Provide framework level support to tag flowfiles with metadata about the flow that created them. Design proposal Right now, MiNiFi support core attributes likeĀ // FlowFile Attribute enum FlowAttribute { // The flowfile's path indicates the relative directory to which a FlowFile belongs and does not contain the filename PATH = 0, // The flowfile's absolute path indicates the absolute directory to which a FlowFile belongs and does not contain the filename ABSOLUTE_PATH, // The filename of the FlowFile. The filename should not contain any directory structure. FILENAME, // A unique UUID assigned to this FlowFile. UUID, // A numeric value indicating the FlowFile priority priority, // The MIME Type of this FlowFile MIME_TYPE, // Specifies the reason that a FlowFile is being discarded DISCARD_REASON, // Indicates an identifier other than the FlowFile's UUID that is known to refer to this FlowFile. ALTERNATE_IDENTIFIER, MAX_FLOW_ATTRIBUTES }; So one approach is in the flow YAML file, specific the list of core flow attributes along with the processors that inject/import/create the flow files. When flow was created/imported/injected by this processor, we can apply these core attributes to the new flow. Also user can define their own core attributes template and EL for populate value for these core attributes, for example protocol, TTL, record route (expected route), key, version, etc. In current implementation, FILENAME, PATH and UUID are required attributes when flow was created, others are optional // Populate the default attributes addKeyedAttribute(FILENAME, std::to_string(getTimeNano())); addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH); addKeyedAttribute(UUID, getUUIDStr()) So if user specify the optional meta flow info section for the processor with the key/value pairs as above, MiNiFI will add these key attributes to the flow when flow was created by this processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)