bqiu created MINIFICPP-403:
------------------------------
Summary: Enable tagging of flowfiles with flow metadata
information in C++
Key: MINIFICPP-403
URL: https://issues.apache.org/jira/browse/MINIFICPP-403
Project: NiFi MiNiFi C++
Issue Type: Improvement
Affects Versions: 1.0.0
Reporter: bqiu
Assignee: bqiu
Fix For: 1.0.0
Provide framework level support to tag flowfiles with metadata about the flow
that created them.
Design proposal
Right now, MiNiFi support core attributes like
// FlowFile Attribute
enum FlowAttribute {
// The flowfile's path indicates the relative directory to which a FlowFile
belongs and does not contain the filename
PATH = 0,
// The flowfile's absolute path indicates the absolute directory to which a
FlowFile belongs and does not contain the filename
ABSOLUTE_PATH,
// The filename of the FlowFile. The filename should not contain any directory
structure.
FILENAME,
// A unique UUID assigned to this FlowFile.
UUID,
// A numeric value indicating the FlowFile priority
priority,
// The MIME Type of this FlowFile
MIME_TYPE,
// Specifies the reason that a FlowFile is being discarded
DISCARD_REASON,
// Indicates an identifier other than the FlowFile's UUID that is known to
refer to this FlowFile.
ALTERNATE_IDENTIFIER,
MAX_FLOW_ATTRIBUTES
};
So one approach is in the flow YAML file, specific the list of core flow
attributes along with the processors that inject/import/create the flow files.
When flow was created/imported/injected by this processor, we can apply these
core attributes to the new flow.
Also user can define their own core attributes template and EL for populate
value for these core attributes, for example protocol, TTL, record route
(expected route), key, version, etc.
In current implementation, FILENAME, PATH and UUID are required attributes when
flow was created, others are optional
// Populate the default attributes
addKeyedAttribute(FILENAME,
std::to_string(getTimeNano()));
addKeyedAttribute(PATH, DEFAULT_FLOWFILE_PATH);
addKeyedAttribute(UUID,
getUUIDStr())
So if user specify the optional meta flow info section for the processor with
the key/value pairs as above, MiNiFI will add these key attributes to the flow
when flow was created by this processor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
