Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractEmailAttachments/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractEmailAttachments/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractEmailAttachments/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractEmailAttachments/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ExtractEmailAttachments</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ExtractEmailAttachments</h1><h2>Description: </h2><p>Extract attachments from a mime formatted email file, splitting them into individual flowfiles.</p><h3>Tags: </h3><p>split, email</p><h3>Properties: </h3><p>This component has no required or optional properties.</p><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>failure</td><td>Flowfiles that could not be parsed</td></tr><tr><td>original</td><td>The original file</td></tr><tr><td>attachments</td><td>Each individual attachment will be routed to the attachments relationship</td></tr></t able><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename </td><td>The filename of the attachment</td></tr><tr><td>email.attachment.parent.filename </td><td>The filename of the parent FlowFile</td></tr><tr><td>email.attachment.parent.uuid</td><td>The UUID of the original FlowFile.</td></tr><tr><td>mime.type</td><td>The mime type of the attachment.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractEmailHeaders/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractEmailHeaders/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractEmailHeaders/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractEmailHeaders/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ExtractEmailHeaders</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ExtractEmailHeaders</h1><h2>Description: </h2><p>Using the flowfile content as source of data, extract header from an RFC compliant email file adding the relevant attributes to the flowfile. This processor does not perform extensive RFC validation but still requires a bare minimum compliance with RFC 2822</p><h3>Tags: </h3><p>split, email</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name< /th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Additional Header List</td><td id="default-value">x-mailer</td><td id="allowable-values"></td><td id="description">COLON separated list of additional headers to be extracted from the flowfile content.NOTE the header key is case insensitive and will be matched as lower-case. Values will respect email contents.</td></tr><tr><td id="name">Email Address Parsing</td><td id="default-value">true</td><td id="allowable-values"><ul><li>Strict Address Parsing <img src="../../../../../html/images/iconInfo.png" alt="Strict email address format will be enforced. FlowFiles will be transfered to the failure relationship if the email address is invalid." title="Strict email address format will be enforced. FlowFiles will be transfered to the failure relationship if the email address is invalid."></img></li><li>Non-Strict Address Parsing <img src="../../../../../html/images/iconInfo.png" alt="Accept emails, even if the address is poorly formed and doesn't strictly comply with RFC Validation." title="Accept emails, even if the address is poorly formed and doesn't strictly comply with RFC Validation."></img></li></ul></td><td id="description">If "strict", strict address format parsing rules are applied to mailbox and mailbox list fields, such as "to" and "from" headers, and FlowFiles with poorly formed addresses will be routed to the failure relationship, similar to messages that fail RFC compliant format validation. If "non-strict", the processor will extract the contents of mailbox list headers as comma-separated values without attempting to parse each value as well-formed Internet mailbox addresses. This is optional and defaults to Strict Address Parsing</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Extraction was successful</td></tr><tr><td>failure</td><td>Flowfiles that could not be parsed as a RF C-2822 compliant message</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>email.headers.bcc.*</td><td>Each individual BCC recipient (if available)</td></tr><tr><td>email.headers.cc.*</td><td>Each individual CC recipient (if available)</td></tr><tr><td>email.headers.from.*</td><td>Each individual mailbox contained in the From of the Email (array as per RFC-2822)</td></tr><tr><td>email.headers.message-id</td><td>The value of the Message-ID header (if available)</td></tr><tr><td>email.headers.received_date</td><td>The Received-Date of the message (if available)</td></tr><tr><td>email.headers.sent_date</td><td>Date the message was sent</td></tr><tr><td>email.headers.subject</td><td>Subject of the message (if available)</td></tr><tr><td>email.headers.to.*</td><td>Each individual TO recipient (if available)</td></tr><tr><td>email.attachment_count</td><td>Number of attachm ents of the message</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractTNEFAttachments/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractTNEFAttachments/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractTNEFAttachments/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ExtractTNEFAttachments/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ExtractTNEFAttachments</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ExtractTNEFAttachments</h1><h2>Description: </h2><p>Extract attachments from a mime formatted email file, splitting them into individual flowfiles.</p><h3>Tags: </h3><p>split, email</p><h3>Properties: </h3><p>This component has no required or optional properties.</p><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>failure</td><td>Each individual flowfile that could not be parsed will be routed to the failure relationship</td></tr><tr><td>original</td><td>Each original flowfile (i.e. before extraction) will be routed to the original relati onship</td></tr><tr><td>attachments</td><td>Each individual attachment will be routed to the attachments relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename </td><td>The filename of the attachment</td></tr><tr><td>email.tnef.attachment.parent.filename </td><td>The filename of the parent FlowFile</td></tr><tr><td>email.tnef.attachment.parent.uuid</td><td>The UUID of the original FlowFile.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ListenSMTP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ListenSMTP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ListenSMTP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-email-nar/1.4.0/org.apache.nifi.processors.email.ListenSMTP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ListenSMTP</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ListenSMTP</h1><h2>Description: </h2><p>This processor implements a lightweight SMTP server to an arbitrary port, allowing nifi to listen for incoming email. Note this server does not perform any email validation. If direct exposure to the internet is sought, it may be a better idea to use the combination of NiFi and an industrial scale MTA (e.g. Postfix). Threading for this processor is managed by the underlying smtp server used so the processor need not support more than one thread.</p><h3>Tags: </h3><p>listen, email, smtp</p><h3>Properties: </h3><p>In the list below, the names of required properti es appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Listening Port</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The TCP port the ListenSMTP processor will bind to.NOTE that on Unix derivative operating systems this port must be higher than 1024 unless NiFi is running as with root user permissions.</td></tr><tr><td id="name"><strong>Maximum number of SMTP connection</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">The maximum number of simultaneous SMTP connections.</td></tr><tr><td id="name"><strong>SMTP connection timeout</strong></td><td id="default-value">60 seconds</td><td id="allowable-values"></td><td id="description">The maximum time to wait for an action of SMTP client.</td></tr><tr><td id="name"><strong>SMTP Maximum Message Size</strong></td><td id="default-value">20 MB</td><td id="allowable-values"></td><td id="description">The maximum number of bytes the server will accept.</td></tr><tr><td id="name">SSL Context Service</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RestrictedSSLContextService<br/><strong>Implementation: </strong><a href="../../../nifi-ssl-context-service-nar/1.4.0/org.apache.nifi.ssl.StandardRestrictedSSLContextService/index.html">StandardRestrictedSSLContextService</a></td><td id="description">The Controller Service to use in order to obtain an SSL Context. If this property is set, messages will be received over a secure connection.</td></tr><tr><td id="name">Client Auth</td><td id="default-value"></td><td id="allowable-values"><ul><li>NONE</li><li>REQUIRED</li></ul></td><td id="description">The client authentication policy to use for the SSL Context. Only used if an SSL Context Service is provided.</td></tr><tr><td id="name">SMTP hostname</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The hostname to be embedded into the banner displayed when an SMTP client connects to the processor TCP port .</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>All new messages will be routed as FlowFiles to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>smtp.helo</td><td>The value used during HELO</td></tr><tr><td>smtp.certificates.*.serial</td><td>The serial numbers for each of the certificates used by an TLS peer</td></tr><tr><td>smtp.certificates.*.principal</td><td>The principal for each of the certificates used by an TLS peer</td></tr><tr><td>smtp.src</td><td>The source IP and port of the SMTP connection</td></tr><tr><td>smtp.from</td><td>The value used during MAIL FROM (i.e. envelope)</td></tr><tr><td>smtp.recipient.*</td><td>The values used during RCPT TO (i.e. envelope)</td></tr><tr><td>mime.type</td><td>Mime type of the message</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component does not allow an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.GeoEnrichIP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.GeoEnrichIP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.GeoEnrichIP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.GeoEnrichIP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>GeoEnrichIP</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">GeoEnrichIP</h1><h2>Description: </h2><p>Looks up geolocation information for an IP address and adds the geo information to FlowFile attributes. The geo data is provided as a MaxMind database. The attribute that contains the IP address to lookup is provided by the 'IP Address Attribute' property. If the name of the attribute provided is 'X', then the the attributes added by enrichment will take the form X.geo.<fieldName></p><h3>Tags: </h3><p>geo, enrich, ip, maxmind</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other propert ies (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>MaxMind Database File</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Path to Maxmind IP Enrichment Database File</td></tr><tr><td id="name"><strong>IP Address Attribute</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of an attribute whose value is a dotted decimal IP address for which enrichment should occur<br/><strong>Supports Expression Language: true</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>not found</td><td>Where to route flow files after unsucces sfully enriching attributes because no data was found</td></tr><tr><td>found</td><td>Where to route flow files after successfully enriching attributes with data provided by database</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>X.geo.lookup.micros</td><td>The number of microseconds that the geo lookup took</td></tr><tr><td>X.geo.city</td><td>The city identified for the IP address</td></tr><tr><td>X.geo.accuracy</td><td>The accuracy radius if provided by the database (in Kilometers)</td></tr><tr><td>X.geo.latitude</td><td>The latitude identified for this IP address</td></tr><tr><td>X.geo.longitude</td><td>The longitude identified for this IP address</td></tr><tr><td>X.geo.subdivision.N</td><td>Each subdivision that is identified for this IP address is added with a one-up number appended to the attribute name, starting with 0</td></tr><tr><td>X.geo.subdivision.isoco de.N</td><td>The ISO code for the subdivision that is identified by X.geo.subdivision.N</td></tr><tr><td>X.geo.country</td><td>The country identified for this IP address</td></tr><tr><td>X.geo.country.isocode</td><td>The ISO Code for the country identified</td></tr><tr><td>X.geo.postalcode</td><td>The postal code for the country identified</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.ISPEnrichIP/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.ISPEnrichIP/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.ISPEnrichIP/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.ISPEnrichIP/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ISPEnrichIP</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ISPEnrichIP</h1><h2>Description: </h2><p>Looks up ISP information for an IP address and adds the information to FlowFile attributes. The ISP data is provided as a MaxMind ISP database (Note that this is NOT the same as the GeoLite database utilizedby some geo enrichment tools). The attribute that contains the IP address to lookup is provided by the 'IP Address Attribute' property. If the name of the attribute provided is 'X', then the the attributes added by enrichment will take the form X.isp.<fieldName></p><h3>Tags: </h3><p>ISP, enrich, ip, maxmind</p><h3>Properties: </h3><p>In the list belo w, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>MaxMind Database File</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Path to Maxmind IP Enrichment Database File</td></tr><tr><td id="name"><strong>IP Address Attribute</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of an attribute whose value is a dotted decimal IP address for which enrichment should occur<br/><strong>Supports Expression Language: true</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>D escription</th></tr><tr><td>not found</td><td>Where to route flow files after unsuccessfully enriching attributes because no data was found</td></tr><tr><td>found</td><td>Where to route flow files after successfully enriching attributes with data provided by database</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>X.isp.lookup.micros</td><td>The number of microseconds that the geo lookup took</td></tr><tr><td>X.isp.asn</td><td>The Autonomous System Number (ASN) identified for the IP address</td></tr><tr><td>X.isp.asn.organization</td><td>The Organization Associated with the ASN identified</td></tr><tr><td>X.isp.name</td><td>The name of the ISP associated with the IP address provided</td></tr><tr><td>X.isp.organization</td><td>The Organization associated with the IP address provided</td></tr></table><h3>State management: </h3>This component does not store state.<h3>R estricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.enrich.QueryDNS/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.enrich.QueryDNS/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.enrich.QueryDNS/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.enrich.QueryDNS/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,2 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>QueryDNS</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">QueryDNS</h1><h2>Description: </h2><p>A powerful DNS query processor primary designed to enrich DataFlows with DNS based APIs (e.g. RBLs, ShadowServer's ASN lookup) but that can be also used to perform regular DNS lookups.</p><h3>Tags: </h3><p>dns, enrich, ip</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><t able id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Lookup value</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The value that should be used to populate the query<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Results Parser</strong></td><td id="default-value">None</td><td id="allowable-values"><ul><li>Split <img src="../../../../../html/images/iconInfo.png" alt="Use a delimiter character or RegEx to split the results into attributes" title="Use a delimiter character or RegEx to split the results into attributes"></img></li><li>RegEx <img src="../../../../../html/images/iconInfo.png" alt="Use a regular expression to split the results into attributes " title="Use a regular expression to split the results into attributes "></img></li><li>None <img src="../../../../../html/images/iconInfo.png" alt="Do not split results" title="Do not split results"></img></li></ul></td><td id="description">The method used to slice the results into attribute groups</td></tr><tr><td id="name">Parser RegEx</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Choice between a splitter and regex matcher used to parse the results of the query into attribute groups. +NOTE: This is a multiline regular expression, therefore, the DFM should decide how to handle trailing new line characters.</td></tr><tr><td id="name"><strong>DNS Query Retries</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">The number of attempts before giving up and moving on</td></tr><tr><td id="name"><strong>DNS Query Timeout</strong></td><td id="default-value">1500 ms</td><td id="allowable-values"></td><td id="description">The amount of time to wait until considering a query as failed</td></tr><tr><td id="name">DNS Servers</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A comma separated list of DNS servers to be used. (Defaults to system wide if none is used)</td></tr><tr><td id="name"><strong>DNS Query Type</strong></td><td id="default-value">TXT</td><td id="allowable-values"></td><td id="description">The DNS query type to be used by the processor (e.g. TXT, A)</td></tr></table><h3>Relationships : </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>not found</td><td>Where to route flow files if data enrichment query rendered no results</td></tr><tr><td>found</td><td>Where to route flow files after successfully enriching attributes with data</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>enrich.dns.record*.group*</td><td>The captured fields of the DNS query response for each of the records received</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.enrich.QueryWhois/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.enrich.QueryWhois/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.enrich.QueryWhois/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-enrich-nar/1.4.0/org.apache.nifi.processors.enrich.QueryWhois/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,2 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>QueryWhois</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">QueryWhois</h1><h2>Description: </h2><p>A powerful whois query processor primary designed to enrich DataFlows with whois based APIs (e.g. ShadowServer's ASN lookup) but that can be also used to perform regular whois lookups.</p><h3>Tags: </h3><p>whois, enrich, ip</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>. </p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Lookup value</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The value that should be used to populate the query<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name">Whois Query Type</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The Whois query type to be used by the processor (if used)</td></tr><tr><td id="name"><strong>Whois Server</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The Whois server to be used</td></tr><tr><td id="name"><strong>Whois Server Port</strong></td><td id="default-value">43</td><td id="allowable-values"></td><td id="description">The TCP port of the remote Whois server</td></tr><tr><td id="name"><strong>Whois Query Timeout</strong></td><td id="default-value">1500 m s</td><td id="allowable-values"></td><td id="description">The amount of time to wait until considering a query as failed</td></tr><tr><td id="name"><strong>Batch Size</strong></td><td id="default-value">25</td><td id="allowable-values"></td><td id="description">The number of incoming FlowFiles to process in a single execution of this processor. </td></tr><tr><td id="name"><strong>Bulk Protocol</strong></td><td id="default-value">None</td><td id="allowable-values"><ul><li>Begin/End <img src="../../../../../html/images/iconInfo.png" alt="The evaluated input of each flowfile is enclosed within begin and end tags. Each row contains a delimited set of fields" title="The evaluated input of each flowfile is enclosed within begin and end tags. Each row contains a delimited set of fields"></img></li><li>None <img src="../../../../../html/images/iconInfo.png" alt="Queries are made without any particular dialect" title="Queries are made without any particular dialect"></img></li></ul></td><td id="description">The protocol used to perform the bulk query. </td></tr><tr><td id="name"><strong>Results Parser</strong></td><td id="default-value">None</td><td id="allowable-values"><ul><li>Split <img src="../../../../../html/images/iconInfo.png" alt="Use a delimiter character or RegEx to split the results into attributes" title="Use a delimiter character or RegEx to split the results into attributes"></img></li><li>RegEx <img src="../../../../../html/images/iconInfo.png" alt="Use a regular expression to split the results into attributes " title="Use a regular expression to split the results into attributes "></img></li><li>None <img src="../../../../../html/images/iconInfo.png" alt="Do not split results" title="Do not split results"></img></li></ul></td><td id="description">The method used to slice the results into attribute groups</td></tr><tr><td id="name">Parser RegEx</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Choice between a splitte r and regex matcher used to parse the results of the query into attribute groups. +NOTE: This is a multiline regular expression, therefore, the DFM should decide how to handle trailing new line characters.</td></tr><tr><td id="name">Key lookup group (multiline / batch)</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">When performing a batched lookup, the following RegEx numbered capture group or Column number will be used to match the whois server response with the lookup field</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>not found</td><td>Where to route flow files if data enrichment query rendered no results</td></tr><tr><td>found</td><td>Where to route flow files after successfully enriching attributes with data</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>enrich.dns.record*.group*</td><td>The captured fields of the Whois query response for each of the records received</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.4.0/org.apache.nifi.processors.evtx.ParseEvtx/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.4.0/org.apache.nifi.processors.evtx.ParseEvtx/additionalDetails.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.4.0/org.apache.nifi.processors.evtx.ParseEvtx/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.4.0/org.apache.nifi.processors.evtx.ParseEvtx/additionalDetails.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,91 @@ +<!DOCTYPE html> +<html lang="en"> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <head> + <meta charset="utf-8" /> + <title>ParseEvtx</title> + + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" /> + </head> + + <body> + <!-- Processor Documentation ================================================== --> + <h2>Description:</h2> + <p>This processor is used to parse Windows event logs in the binary evtx format. The input flow files' content should be evtx files. The processor has 4 outputs: + <ul> + <li>The original unmodified FlowFile</li> + <li>The XML resulting from parsing at the configured granularity</li> + <li>Failed parsing with partial output</li> + <li>Malformed chunk in binary form</li> + </ul> + </p> + <h2>Output XML Example:</h2> + <p> + <pre> +<?xml version="1.0"?> +<Events> + <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> + <System> + <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" Ev +entSourceName="Service Control Manager"/> + <EventID Qualifiers="16384">7036</EventID> + <Version>0</Version> + <Level>4</Level> + <Task>0</Task> + <Opcode>0</Opcode> + <Keywords>0x8080000000000000</Keywords> + <TimeCreated SystemTime="2016-01-08 16:49:47.518"/> + <EventRecordID>780</EventRecordID> + <Correlation ActivityID="" RelatedActivityID=""/> + <Execution ProcessID="480" ThreadID="596"/> + <Channel>System</Channel> + <Computer>win7-pro-vm</Computer> + <Security UserID=""/> + </System> + <EventData> + <Data Name="param1">Workstation</Data> + <Data Name="param2">running</Data> + <Binary>TABhAG4AbQBhAG4AVwBvAHIAawBzAHQAYQB0AGkAbwBuAC8ANAAAAA==</Binary> + </EventData> + </Event> + <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> + <System> + <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager"/> + <EventID Qualifiers="16384">7036</EventID> + <Version>0</Version> + <Level>4</Level> + <Task>0</Task> + <Opcode>0</Opcode> + <Keywords>0x8080000000000000</Keywords> + <TimeCreated SystemTime="2016-01-08 16:49:47.535"/> + <EventRecordID>781</EventRecordID> + <Correlation ActivityID="" RelatedActivityID=""/> + <Execution ProcessID="480" ThreadID="576"/> + <Channel>System</Channel> + <Computer>win7-pro-vm</Computer> + <Security UserID=""/> + </System> + <EventData> + <Data Name="param1">Cryptographic Services</Data> + <Data Name="param2">running</Data> + <Binary>QwByAHkAcAB0AFMAdgBjAC8ANAAAAA==</Binary> + </EventData> + </Event> +</Events> + </pre> + </p> +</body> +</html> Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.4.0/org.apache.nifi.processors.evtx.ParseEvtx/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.4.0/org.apache.nifi.processors.evtx.ParseEvtx/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.4.0/org.apache.nifi.processors.evtx.ParseEvtx/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-evtx-nar/1.4.0/org.apache.nifi.processors.evtx.ParseEvtx/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ParseEvtx</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ParseEvtx</h1><h2>Description: </h2><p>Parses the contents of a Windows Event Log file (evtx) and writes the resulting XML to the FlowFile</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>logs, windows, event, evtx, message, file</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td i d="name"><strong>Granularity</strong></td><td id="default-value">Chunk</td><td id="allowable-values"><ul><li>Record</li><li>Chunk</li><li>File</li></ul></td><td id="description">Output flow file for each Record, Chunk, or File encountered in the event log</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Any FlowFile that was successfully converted from evtx to XML</td></tr><tr><td>failure</td><td>Any FlowFile that encountered an exception during conversion will be transferred to this relationship with as much parsing as possible done</td></tr><tr><td>original</td><td>The unmodified input FlowFile will be transferred to this relationship</td></tr><tr><td>bad chunk</td><td>Any bad chunks of records will be transferred to this relationship in their original binary form</td></tr></table><h3>Reads Attributes: </h3><table id="reads-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td> <td>The filename of the evtx file</td></tr></table><h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td><td>The output filename</td></tr><tr><td>mime.type</td><td>The output filetype (application/xml for success and failure relationships, original value for bad chunk and original relationships)</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/additionalDetails.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,155 @@ +<!DOCTYPE html> +<html lang="en"> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> +<head> + <meta charset="utf-8" /> + <title>ExecuteFlumeSink</title> + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" /> +</head> + +<body> +<h2>Data Model</h2> +<p> + This processor executes an Apache Flume sink. FlowFiles are wrapped in Flume's + Event interface. The content of the FlowFile becomes the body of the Event and + the attributes of the FlowFile become Event headers. The following special + headers are also set: +</p> +<table id="headers"> + <tr> + <th>Flume Event Header</th> + <th>FlowFile Attribute</th> + </tr> + <tr> + <td>nifi.entry.date</td> + <td>FlowFile#getEntryDate()</td> + </tr> + <tr> + <td>nifi.id</td> + <td>FlowFile#getId()</td> + </tr> + <tr> + <td>nifi.last.queue.date</td> + <td>FlowFile#getLastQueueDate()</td> + </tr> + <tr> + <td>nifi.lineage.start.date</td> + <td>FlowFile#getLineageStartDate()</td> + </tr> + <tr> + <td>nifi.size</td> + <td>FlowFile#getSize()</td> + </tr> +</table> +<h2>Warning</h2> +<p> + In NiFi, the contents of a FlowFile are accessed via a stream, but in Flume it is + stored in a byte array. This means the full content will be loaded into memory when + a FlowFile is processed by the ExecuteFlumeSink processor. You should consider the + typical size of the FlowFiles you'll process and the batch size, if any, your sink + is configured with when setting NiFi's heap size. +</p> +<h2>Configuration Details</h2> +<p> + This processor is designed to execute arbitrary Flume sinks. Most of the details + of configuring the sink is deferred to Flume's built-in configuration system. + For details on the available settings for each sink type, refer to the Flume + <a href="http://flume.apache.org/FlumeUserGuide.html#flume-sinks";>User Guide</a>. + Configuring the Flume sink is a four step process: +</p> +<ol> + <li>Set the Sink Type property to a valid Flume sink type.</li> + <li> + Set the Agent Name property to the name of the agent in your + Flume configuration. This is the prefix of the properties in the Flume + configuration file. Example: <code>tier1</code> + </li> + <li> + Set the Sink Name property to the name of the sink in your Flume + configuration. If Agent Name is <code>tier1</code>, then the Sink Name + is the value of the <code>tier1.sinks</code> property. Example: <code>sink-1</code> + </li> + <li> + Copy and paste the configuration for the sink from your Flume configuration + file into the Flume Configuration property. Assuming you're using + the same Agent Name and Sink Name as in the examples above, this will be all + of the properties that start with <code>tier1.sinks.sink-1</code>. + Do not copy the <code>tier1.sinks.sink-1.type</code> or + <code>tier1.sinks.sink-1.channel</code> properties. + </li> +</ol> +<h2>Usage Example</h2> +<p> + Assuming you had the following existing Flume configuration file: +</p> + <pre> +a1.sources = r1 +a1.sinks = k1 +a1.channels = c1 + +a1.sources.r1.type = netcat +a1.sources.r1.bind = localhost +a1.sources.r1.port = 44444 + +a1.sinks.k1.type = hdfs +a1.sinks.k1.channel = c1 +a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/%H%M/%S +a1.sinks.k1.hdfs.filePrefix = events- +a1.sinks.k1.hdfs.round = true +a1.sinks.k1.hdfs.roundValue = 10 +a1.sinks.k1.hdfs.roundUnit = minute + +a1.channels.c1.type = memory +a1.channels.c1.capacity = 1000 +a1.channels.c1.transactionCapacity = 100 + +a1.sources.r1.channels = c1 +a1.sinks.k1.channel = c1</pre> +<p> + Then you'd configure the ExecuteFlumeSink as follows: +</p> +<table id="example"> + <tr> + <th>Property</th> + <th>Value</th> + </tr> + <tr> + <td>Sink Type</td> + <td>hdfs</td> + </tr> + <tr> + <td>Agent Name</td> + <td>a1</td> + </tr> + <tr> + <td>Sink Name</td> + <td>k1</td> + </tr> + <tr> + <td>Flume Configuration</td> + <td> + <code> + a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/%H%M/%S<br> + a1.sinks.k1.hdfs.filePrefix = events-<br> + a1.sinks.k1.hdfs.round = true<br> + a1.sinks.k1.hdfs.roundValue = 10<br> + a1.sinks.k1.hdfs.roundUnit = minute + </code> + </td> + </tr> +</table> +</body> +</html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSink/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ExecuteFlumeSink</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ExecuteFlumeSink</h1><h2>Description: </h2><p>Execute a Flume sink. Each input FlowFile is converted into a Flume Event for processing by the sink.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>flume, hadoop, put, sink, restricted</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></t r><tr><td id="name"><strong>Sink Type</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The component type name for the sink. For some sinks, this is a short, symbolic name (e.g. hdfs). For others, it's the fully-qualified name of the Sink class. See the Flume User Guide for details.</td></tr><tr><td id="name"><strong>Agent Name</strong></td><td id="default-value">tier1</td><td id="allowable-values"></td><td id="description">The name of the agent used in the Flume sink configuration</td></tr><tr><td id="name"><strong>Sink Name</strong></td><td id="default-value">sink-1</td><td id="allowable-values"></td><td id="description">The name of the sink used in the Flume sink configuration</td></tr><tr><td id="name"><strong>Flume Configuration</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The Flume configuration for the sink copied from the flume.properties file</td></tr></table><h3>Relationships: </h3> <table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td></td></tr><tr><td>failure</td><td></td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>Provides operator the ability to execute arbitrary Flume configurations assuming all permissions that NiFi has.<h3>Input requirement: </h3>This component requires an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/additionalDetails.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/additionalDetails.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,114 @@ +<!DOCTYPE html> +<html lang="en"> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> +<head> + <meta charset="utf-8" /> + <title>ExecuteFlumeSource</title> + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" /> +</head> + +<body> +<h2>Data Model</h2> +<p> + This processor executes an Apache Flume source. Each Flume Event is turned into + a FlowFile. The content of the FlowFile is set to the body of the Event and + the Event headers become FlowFile attributes. +</p> +<h2>Configuration Details</h2> +<p> + This processor is designed to execute arbitrary Flume sources. Most of the details + of configuring the source is deferred to Flume's built-in configuration system. + For details on the available settings for each source type, refer to the Flume + <a href="http://flume.apache.org/FlumeUserGuide.html#flume-sources";>User Guide</a>. + Configuring the Flume source is a four step process: +</p> +<ol> + <li>Set the Source Type property to a valid Flume source type.</li> + <li> + Set the Agent Name property to the name of the agent in your + Flume configuration. This is the prefix of the properties in the Flume + configuration file. Example: <code>tier1</code> + </li> + <li> + Set the Source Name property to the name of the source in your Flume + configuration. If Agent Name is <code>tier1</code>, then the Source Name + is the value of the <code>tier1.sources</code> property. Example: <code>src-1</code> + </li> + <li> + Copy and paste the configuration for the source from your Flume configuration + file into the Flume Configuration property. Assuming you're using + the same Agent Name and Source Name as in the examples above, this will be all + of the properties that start with <code>tier1.sources.src-1</code>. + Do not copy the <code>tier1.sources.src-1.type</code> or + <code>tier1.sources.src-1.channel</code> properties. + </li> +</ol> +<h2>Usage Example</h2> +<p> + Assuming you had the following existing Flume configuration file: +</p> + <pre> +a1.sources = r1 +a1.sinks = k1 +a1.channels = c1 + +a1.sources.r1.type = multiport_syslogtcp +a1.sources.r1.channels = c1 +a1.sources.r1.host = 0.0.0.0 +a1.sources.r1.ports = 10001 10002 10003 +a1.sources.r1.portHeader = port + +a1.sinks.k1.type = logger + +a1.channels.c1.type = memory +a1.channels.c1.capacity = 1000 +a1.channels.c1.transactionCapacity = 100 + +a1.sources.r1.channels = c1 +a1.sinks.k1.channel = c1</pre> +<p> + Then you'd configure the ExecuteFlumeSource as follows: +</p> +<table id="example"> + <tr> + <th>Property</th> + <th>Value</th> + </tr> + <tr> + <td>Source Type</td> + <td>multiport_syslogtcp</td> + </tr> + <tr> + <td>Agent Name</td> + <td>a1</td> + </tr> + <tr> + <td>Source Name</td> + <td>r1</td> + </tr> + <tr> + <td>Flume Configuration</td> + <td> + <code> + a1.sources.r1.host = 0.0.0.0<br> + a1.sources.r1.ports = 10001 10002 10003<br> + a1.sources.r1.portHeader = port + </code> + </td> + </tr> +</table> +</body> +</html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-flume-nar/1.4.0/org.apache.nifi.processors.flume.ExecuteFlumeSource/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ExecuteFlumeSource</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ExecuteFlumeSource</h1><h2>Description: </h2><p>Execute a Flume source. Each Flume Event is sent to the success relationship as a FlowFile</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>flume, hadoop, get, source, restricted</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr ><td id="name"><strong>Source Type</strong></td><td >id="default-value"></td><td id="allowable-values"></td><td >id="description">The component type name for the source. For some sources, >this is a short, symbolic name (e.g. spooldir). For others, it's the >fully-qualified name of the Source class. See the Flume User Guide for >details.</td></tr><tr><td id="name"><strong>Agent Name</strong></td><td >id="default-value">tier1</td><td id="allowable-values"></td><td >id="description">The name of the agent used in the Flume source >configuration</td></tr><tr><td id="name"><strong>Source Name</strong></td><td >id="default-value">src-1</td><td id="allowable-values"></td><td >id="description">The name of the source used in the Flume source >configuration</td></tr><tr><td id="name"><strong>Flume >Configuration</strong></td><td id="default-value"></td><td >id="allowable-values"></td><td id="description">The Flume configuration for >the source copied from the flume.properties file</td></tr></table><h3>Rela tionships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td></td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>Provides operator the ability to execute arbitrary Flume configurations assuming all permissions that NiFi has.<h3>Input requirement: </h3>This component does not allow an incoming relationship.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>GCPCredentialsControllerService</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">GCPCredentialsControllerService</h1><h2>Description: </h2><p>Defines credentials for Google Cloud Platform processors. Uses Application Default credentials without configuration. Application Default credentials support environmental variable (GOOGLE_APPLICATION_CREDENTIALS) pointing to a credential file, the config generated by `gcloud auth application-default login`, AppEngine/Compute Engine service accounts, etc.</p><h3>Tags: </h3><p>gcp, credentials, provider</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any oth er properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">Use Application Default Credentials</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If true, uses Google Application Default Credentials, which checks the GOOGLE_APPLICATION_CREDENTIALS environment variable for a filepath to a service account JSON key, the config generated by the gcloud sdk, the A pp Engine service account, and the Compute Engine service account.</td></tr><tr><td id="name">Use Compute Engine Credentials</td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If true, uses Google Compute Engine Credentials of the Compute Engine VM Instance which NiFi is running on.</td></tr><tr><td id="name">Service Account JSON File</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Path to a file containing a Service Account key file in JSON format.</td></tr><tr><td id="name">Service Account JSON</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The raw JSON containing a Service Account keyfile.<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true</strong></td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>DeleteGCSObject</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">DeleteGCSObject</h1><h2>Description: </h2><p>Deletes objects from a Google Cloud Bucket. If attempting to delete a file that does not exist, FlowFile is routed to success.</p><h3>Tags: </h3><p>google cloud, gcs, google, storage, delete</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properti es"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name"><strong>Project ID</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID</td></tr><tr><td id="name"><strong>Number of retries</strong></td><td id="default-value">6</td><td id="allowable-values"></td><td id="description">How many retry attempts should be made before routing to the failure relationship.</td></tr><tr><td id="name"><strong>Buck et</strong></td><td id="default-value">${gcs.bucket}</td><td id="allowable-values"></td><td id="description">Bucket of the object.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Key</strong></td><td id="default-value">${filename}</td><td id="allowable-values"></td><td id="description">Name of the object.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name">Generation</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The generation of the object to be deleted. If null, will use latest version of the object.<br/><strong>Supports Expression Language: true</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles are routed to this relationship after a successful Google Cloud Storage operation.</td></tr><tr><td>failure</td><td>FlowFiles are routed to this relationship if the Google Cloud Storage operation fails.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.storage.PutGCSObject/index.html">PutGCSObject</a>, <a href="../org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html">FetchGCSObject</a>, <a href="../org.apache.nifi.processors.gcp.storage.ListGCSBucket/index.html">ListGCSBucket</a></p></body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>FetchGCSObject</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">FetchGCSObject</h1><h2>Description: </h2><p>Fetches a file from a Google Cloud Bucket. Designed to be used in tandem with ListGCSBucket.</p><h3>Tags: </h3><p>google cloud, google, storage, gcs, fetch</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name"><strong>Project ID</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID</td></tr><tr><td id="name"><strong>Number of retries</strong></td><td id="default-value">6</td><td id="allowable-values"></td><td id="description">How many retry attempts should be made before routing to the failure relationship.</td></tr><tr><td id="name"><strong>Bucket</strong></td><td id="default-value">${gcs.bucket}</td><td id="allowable-values"></td><td id="description">Bucket of the object.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name"><strong>Key</strong></td><td id="default-value">${filename}</td><td id="allowable-values"></td><td id="description">Name of the object.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name">Object Generation</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The generation of the Object to download. If null, will download latest generation.<br/><strong>Supports Expression Language: true</strong></td></tr><tr><td id="name">Server Side Encryption Key</td><td id="default-value"></td><td id="a llowable-values"></td><td id="description">An AES256 Key (encoded in base64) which the object has been encrypted in.<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles are routed to this relationship after a successful Google Cloud Storage operation.</td></tr><tr><td>failure</td><td>FlowFiles are routed to this relationship if the Google Cloud Storage operation fails.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td><td>The name of the file, parsed if possible from the Content-Disposition response header</td></tr><tr><td>gcs.bucket</td><td>Bucket of the object.</td></tr><tr><td>gcs.key</td><td>Name of the object.</td></tr><tr><td>gcs.size</td><td>Size of the objec t.</td></tr><tr><td>gcs.cache.control</td><td>Data cache control of the object.</td></tr><tr><td>gcs.component.count</td><td>The number of components which make up the object.</td></tr><tr><td>gcs.content.disposition</td><td>The data content disposition of the object.</td></tr><tr><td>gcs.content.encoding</td><td>The content encoding of the object.</td></tr><tr><td>gcs.content.language</td><td>The content language of the object.</td></tr><tr><td>mime.type</td><td>The MIME/Content-Type of the object</td></tr><tr><td>gcs.crc32c</td><td>The CRC32C checksum of object's data, encoded in base64 in big-endian order.</td></tr><tr><td>gcs.create.time</td><td>The creation time of the object (milliseconds)</td></tr><tr><td>gcs.update.time</td><td>The last modification time of the object (milliseconds)</td></tr><tr><td>gcs.encryption.algorithm</td><td>The algorithm used to encrypt the object.</td></tr><tr><td>gcs.encryption.sha256</td><td>The SHA256 hash of the key used to encrypt the object</t d></tr><tr><td>gcs.etag</td><td>The HTTP 1.1 Entity tag for the object.</td></tr><tr><td>gcs.generated.id</td><td>The service-generated for the object</td></tr><tr><td>gcs.generation</td><td>The data generation of the object.</td></tr><tr><td>gcs.md5</td><td>The MD5 hash of the object's data encoded in base64.</td></tr><tr><td>gcs.media.link</td><td>The media download link to the object.</td></tr><tr><td>gcs.metageneration</td><td>The metageneration of the object.</td></tr><tr><td>gcs.owner</td><td>The owner (uploader) of the object.</td></tr><tr><td>gcs.owner.type</td><td>The ACL entity type of the uploader of the object.</td></tr><tr><td>gcs.uri</td><td>The URI of the object as a string.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.storage.ListGCSBucket/i ndex.html">ListGCSBucket</a>, <a href="../org.apache.nifi.processors.gcp.storage.PutGCSObject/index.html">PutGCSObject</a>, <a href="../org.apache.nifi.processors.gcp.storage.DeleteGCSObject/index.html">DeleteGCSObject</a></p></body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.ListGCSBucket/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.ListGCSBucket/index.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.ListGCSBucket/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-gcp-nar/1.4.0/org.apache.nifi.processors.gcp.storage.ListGCSBucket/index.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ListGCSBucket</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ListGCSBucket</h1><h2>Description: </h2><p>Retrieves a listing of objects from an GCS bucket. For each object that is listed, creates a FlowFile that represents the object so that it can be fetched in conjunction with FetchGCSObject. This Processor is designed to run on Primary Node only in a cluster. If the primary node changes, the new Primary Node will pick up where the previous node left off without duplicating all of the data.</p><h3>Tags: </h3><p>google cloud, google, storage, gcs, list</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</str ong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>GCP Credentials Provider Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>GCPCredentialsService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.processors.gcp.credentials.service.GCPCredentialsControllerService/index.html">GCPCredentialsControllerService</a></td><td id="description">The Controller Service used to obtain Google Cloud Platform credentials.</td></tr><tr><td id="name"><strong>Project ID</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Google Cloud Project ID</td></tr><tr><td id="name"><strong>Number of retries</strong></td><td id="default-value">6</td><td id="allowable-values"></td><td id="de scription">How many retry attempts should be made before routing to the failure relationship.</td></tr><tr><td id="name"><strong>Bucket</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Bucket of the object.</td></tr><tr><td id="name">Prefix</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The prefix used to filter the object list. In most cases, it should end with a forward slash ('/').</td></tr><tr><td id="name"><strong>Use Generations</strong></td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Specifies whether to use GCS Generations, if applicable. If false, only the latest version of each object will be returned.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles are routed to this relationship after a successful Google Cloud Storage ope ration.</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>filename</td><td>The name of the file</td></tr><tr><td>gcs.bucket</td><td>Bucket of the object.</td></tr><tr><td>gcs.key</td><td>Name of the object.</td></tr><tr><td>gcs.size</td><td>Size of the object.</td></tr><tr><td>gcs.cache.control</td><td>Data cache control of the object.</td></tr><tr><td>gcs.component.count</td><td>The number of components which make up the object.</td></tr><tr><td>gcs.content.disposition</td><td>The data content disposition of the object.</td></tr><tr><td>gcs.content.encoding</td><td>The content encoding of the object.</td></tr><tr><td>gcs.content.language</td><td>The content language of the object.</td></tr><tr><td>mime.type</td><td>The MIME/Content-Type of the object</td></tr><tr><td>gcs.crc32c</td><td>The CRC32C checksum of object's data, encoded in base64 in big-endian order.</td>< /tr><tr><td>gcs.create.time</td><td>The creation time of the object (milliseconds)</td></tr><tr><td>gcs.update.time</td><td>The last modification time of the object (milliseconds)</td></tr><tr><td>gcs.encryption.algorithm</td><td>The algorithm used to encrypt the object.</td></tr><tr><td>gcs.encryption.sha256</td><td>The SHA256 hash of the key used to encrypt the object</td></tr><tr><td>gcs.etag</td><td>The HTTP 1.1 Entity tag for the object.</td></tr><tr><td>gcs.generated.id</td><td>The service-generated for the object</td></tr><tr><td>gcs.generation</td><td>The data generation of the object.</td></tr><tr><td>gcs.md5</td><td>The MD5 hash of the object's data encoded in base64.</td></tr><tr><td>gcs.media.link</td><td>The media download link to the object.</td></tr><tr><td>gcs.metageneration</td><td>The metageneration of the object.</td></tr><tr><td>gcs.owner</td><td>The owner (uploader) of the object.</td></tr><tr><td>gcs.owner.type</td><td>The ACL entity type of the uploader of the object.</td></tr><tr><td>gcs.uri</td><td>The URI of the object as a string.</td></tr></table><h3>State management: </h3><table id="stateful"><tr><th>Scope</th><th>Description</th></tr><tr><td>CLUSTER</td><td>After performing a listing of keys, the timestamp of the newest key is stored, along with the keys that share that same timestamp. This allows the Processor to list only keys that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node can pick up where the previous node left off, without duplicating the data.</td></tr></table><h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component does not allow an incoming relationship.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.gcp.storage.PutGCSObject/index.html">PutGCSObject</a>, <a href="../org.apache.nifi.processo rs.gcp.storage.DeleteGCSObject/index.html">DeleteGCSObject</a>, <a href="../org.apache.nifi.processors.gcp.storage.FetchGCSObject/index.html">FetchGCSObject</a></p></body></html> \ No newline at end of file
