Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.TransformXml/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.TransformXml/index.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.TransformXml/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.TransformXml/index.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>TransformXml</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">TransformXml</h1><h2>Description: </h2><p>Applies the provided XSLT file to the flowfile XML payload. A new FlowFile is created with transformed content and is routed to the 'success' relationship. If the XSL transform fails, the original FlowFile is routed to the 'failure' relationship</p><h3>Tags: </h3><p>xml, xslt, transform</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../. ./html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name">XSLT file name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Provides the name (including full path) of the XSLT file to apply to the flowfile XML content.One of the 'XSLT file name' and 'XSLT Lookup' properties must be defined.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">XSLT Lookup</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>StringLookupService<br/><strong>Implementations: </strong><a href="../../../nifi-lookup-services-nar/1.11.4/org.apache.nifi.lookup.db.SimpleDatabaseLookupService/index.html">SimpleDatabaseLookupService</a><br/><a href="../../../nifi-lookup-services-nar/1.11.4/ org.apache.nifi.lookup.SimpleCsvFileLookupService/index.html">SimpleCsvFileLookupService</a><br/><a href="../../../nifi-lookup-services-nar/1.11.4/org.apache.nifi.lookup.PropertiesFileLookupService/index.html">PropertiesFileLookupService</a><br/><a href="../../../nifi-scripting-nar/1.11.4/org.apache.nifi.lookup.script.SimpleScriptedLookupService/index.html">SimpleScriptedLookupService</a><br/><a href="../../../nifi-lookup-services-nar/1.11.4/org.apache.nifi.lookup.SimpleKeyValueLookupService/index.html">SimpleKeyValueLookupService</a><br/><a href="../../../nifi-lookup-services-nar/1.11.4/org.apache.nifi.lookup.XMLFileLookupService/index.html">XMLFileLookupService</a><br/><a href="../../../nifi-lookup-services-nar/1.11.4/org.apache.nifi.lookup.DistributedMapCacheLookupService/index.html">DistributedMapCacheLookupService</a><br/><a href="../../../nifi-elasticsearch-client-service-nar/1.11.4/org.apache.nifi.elasticsearch.ElasticSearchStringLookupService/index.html">ElasticSearchStringL ookupService</a><br/><a href="../../../nifi-couchbase-nar/1.11.4/org.apache.nifi.couchbase.CouchbaseKeyValueLookupService/index.html">CouchbaseKeyValueLookupService</a></td><td id="description">Controller lookup used to store XSLT definitions. One of the 'XSLT file name' and 'XSLT Lookup' properties must be defined. WARNING: note that the lookup controller service should not be used to store large XSLT files.</td></tr><tr><td id="name">XSLT Lookup key</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Key used to retrieve the XSLT definition from the XSLT lookup controller. This property must be set when using the XSLT controller property.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Indent</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Whether or not t o indent the output.</td></tr><tr><td id="name"><strong>Secure processing</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Whether or not to mitigate various XML-related attacks like XXE (XML External Entity) attacks.</td></tr><tr><td id="name"><strong>Cache size</strong></td><td id="default-value">10</td><td id="allowable-values"></td><td id="description">Maximum number of stylesheets to cache. Zero disables the cache.</td></tr><tr><td id="name"><strong>Cache TTL after last access</strong></td><td id="default-value">60 secs</td><td id="allowable-values"></td><td id="description">The cache TTL (time-to-live) or how long to keep stylesheets in the cache after last access.</td></tr></table><h3>Dynamic Properties: </h3><p>Dynamic Properties allow the user to specify both the name and value of a property.<table id="dynamic-properties"><tr><th>Name</th><th>Value</th><th>Description</th></tr><tr><td id="na me">An XSLT transform parameter name</td><td id="value">An XSLT transform parameter value</td><td>These XSLT parameters are passed to the transformer<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table></p><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>The FlowFile with transformed content will be routed to this relationship</td></tr><tr><td>failure</td><td>If a FlowFile fails processing for any reason (for example, the FlowFile is not valid XML), it will be routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specif ied.</body></html> \ No newline at end of file
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UnpackContent/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UnpackContent/index.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UnpackContent/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UnpackContent/index.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>UnpackContent</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">UnpackContent</h1><h2>Description: </h2><p>Unpacks the content of FlowFiles that have been packaged with one of several different Packaging Formats, emitting one to many FlowFiles for each input FlowFile</p><h3>Tags: </h3><p>Unpack, un-merge, tar, zip, archive, flowfile-stream, flowfile-stream-v3</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th ><th>Description</th></tr><tr><td id="name"><strong>Packaging >Format</strong></td><td id="default-value">use mime.type attribute</td><td >id="allowable-values"><ul><li>use mime.type >attribute</li><li>tar</li><li>zip</li><li>flowfile-stream-v3</li><li>flowfile-stream-v2</li><li>flowfile-tar-v1</li></ul></td><td > id="description">The Packaging Format used to create the >file</td></tr><tr><td id="name"><strong>File Filter</strong></td><td >id="default-value">.*</td><td id="allowable-values"></td><td >id="description">Only files contained in the archive whose names match the >given regular expression will be extracted (tar/zip >only)</td></tr></table><h3>Relationships: </h3><table >id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Unpacked > FlowFiles are sent to this relationship</td></tr><tr><td>failure</td><td>The >original FlowFile is sent to this relationship when it cannot be unpacked for >some reason</td></tr><tr><td>original</td><td>The original FlowFile i s sent to this relationship after it has been successfully unpacked</td></tr></table><h3>Reads Attributes: </h3><table id="reads-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>mime.type</td><td>If the <Packaging Format> property is set to use mime.type attribute, this attribute is used to determine the FlowFile's MIME Type. In this case, if the attribute is set to application/tar, the TAR Packaging Format will be used. If the attribute is set to application/zip, the ZIP Packaging Format will be used. If the attribute is set to application/flowfile-v3 or application/flowfile-v2 or application/flowfile-v1, the appropriate FlowFile Packaging Format will be used. If this attribute is missing, the FlowFile will be routed to 'failure'. Otherwise, if the attribute's value is not one of those mentioned above, the FlowFile will be routed to 'success' without being unpacked. Use the File Filter property only extract files matching a specific regular expression.</td></tr>< /table><h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>mime.type</td><td>If the FlowFile is successfully unpacked, its MIME Type is no longer known, so the mime.type attribute is set to application/octet-stream.</td></tr><tr><td>fragment.identifier</td><td>All unpacked FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attribute</td></tr><tr><td>fragment.index</td><td>A one-up number that indicates the ordering of the unpacked FlowFiles that were created from a single parent FlowFile</td></tr><tr><td>fragment.count</td><td>The number of unpacked FlowFiles generated from the parent FlowFile</td></tr><tr><td>segment.original.filename </td><td>The filename of the parent FlowFile. Extensions of .tar, .zip or .pkg are removed because the MergeContent processor automatically adds those extensions if it is used to rebuild the original FlowFile</td></tr></table><h3>State manag ement: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.standard.MergeContent/index.html">MergeContent</a></p></body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateCounter/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateCounter/index.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateCounter/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateCounter/index.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>UpdateCounter</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">UpdateCounter</h1><h2>Description: </h2><p>This processor allows users to set specific counters and key points in their flow. It is useful for debugging and basic counting functions.</p><h3>Tags: </h3><p>counter, debug, instrumentation</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties "><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Counter Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the counter you want to set the value off - supports expression language like ${counterName}<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Delta</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">Adjusts the counter by the specified delta for each flow file received. May be a positive or negative integer.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Counter was updated/retrieved</td></tr>< /table><h3>Reads Attributes: </h3><table id="reads-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>counterName</td><td>The name of the counter to update/get.</td></tr></table><h3>Writes Attributes: </h3>None specified.<h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1,390 @@ +<!DOCTYPE html> +<html lang="en"> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <head> + <meta charset="utf-8" /> + <title>UpdateRecord</title> + + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" /> + </head> + + <body> + <p> + UpdateRecord makes use of the NiFi <a href="../../../../../html/record-path-guide.html"> + RecordPath Domain-Specific Language (DSL)</a> to allow the user + to indicate which field(s) in the Record should be updated. Users do this by adding a User-defined + Property to the Processor's configuration. The name of the User-defined Property must be the + RecordPath text that should be evaluated against each Record. The value of the Property specifies + what value should go into that selected Record field. + </p> + + <p> + When specifying the replacement value (the value of the User-defined Property), the user is able + to specify a literal value such as the number <code>10</code>; an Expression Language Expression + to reference FlowFile attributes, such as <code>${filename}</code>; or another RecordPath path + from which to retrieve the desired value from the Record itself. Whether the value entered should + be interpreted as a literal or a RecordPath path is determined by the value of the <Replacement + Value Strategy> Property. + </p> + + <p> + If a RecordPath is given and does not match any field in an input Record, that Property will be skipped + and all other Properties will still be evaluated. If the RecordPath matches exactly one field, that field + will be updated with the corresponding value. If multiple fields match the RecordPath, then all fields that + match will be updated. If the replacement value is itself a RecordPath that does not match, then a + <code>null</code> value will be set for the field. For instances where this is not the desired behavior, + RecordPath predicates can be used to filter the fields that match so that no fields will be selected. + See <a href="../../../../../html/record-path-guide.html#predicates">RecordPath Predicates</a> for more information. + </p> + + <p> + Below, we lay out some examples in order to provide clarity about the Processor's behavior. For all of + the examples below, consider the example to operate on the following set of 2 (JSON) records: + </p> + +<code> +<pre> + [{ + "id": 17, + "name": "John", + "child": { + "id": "1" + }, + "siblingIds": [4, 8], + "siblings": [ + { "name": "Jeremy", "id": 4 }, + { "name": "Julia", "id": 8 } + ] + }, + { + "id": 98, + "name": "Jane", + "child": { + "id": 2 + }, + "gender": "F", + "siblingIds": [], + "siblings": [] + }] +</pre> +</code> + + <p> + For brevity, we will omit the corresponding schema and configuration of the RecordReader and + RecordWriter. Otherwise, consider the following set of Properties are configured for the Processor + and their associated outputs. + </p> + + <h3>Example 1 - Replace with Literal</h3> + + <p> + Here, we will replace the name of each Record with the name 'Jeremy' and set the gender to 'M': + </p> + + <table> + <tr> + <th>Property Name</th> + <th>Property Value</th> + </tr> + <tr> + <td>Replacement Value Strategy</td> + <td>Literal Value</td> + </tr> + <tr> + <td>/name</td> + <td>Jeremy</td> + </tr> + <tr> + <td>/gender</td> + <td>M</td> + </tr> + </table> + + <p> + This will yield the following output: + </p> + +<code> +<pre> + [{ + "id": 17, + "name": "Jeremy", + "child": { + "id": "1" + }, + "gender": "M", + "siblingIds": [4, 8], + "siblings": [ + { "name": "Jeremy", "id": 4 }, + { "name": "Julia", "id": 8 } + ] + }, + { + "id": 98, + "name": "Jeremy", + "child": { + "id": 2 + }, + "gender": "M", + "siblingIds": [], + "siblings": [] + }] +</pre> +</code> + + <p> + Note that even though the first record did not have a "gender" field in the input, one + will be added after the "child" field, as that's where the field is located in the schema. + </p> + + + <h3>Example 2 - Replace with RecordPath</h3> + + <p> + This example will replace the value in one field of the Record with the value from another field. + For this example, consider the following set of Properties: + </p> + + <table> + <tr> + <th>Property Name</th> + <th>Property Value</th> + </tr> + <tr> + <td>Replacement Value Strategy</td> + <td>Record Path Value</td> + </tr> + <tr> + <td>/name</td> + <td>/siblings[0]/name</td> + </tr> + </table> + + <p> + This will yield the following output: + </p> + +<code> +<pre> + [{ + "id": 17, + "name": "Jeremy", + "child": { + "id": "1" + }, + "siblingIds": [4, 8], + "siblings": [ + { "name": "Jeremy", "id": 4 }, + { "name": "Julia", "id": 8 } + ] + }, + { + "id": 98, + "name": null, + "child": { + "id": 2 + }, + "gender": "F", + "siblingIds": [], + "siblings": [] + }] +</pre> +</code> + + + + <h3>Example 3 - Replace with Relative RecordPath</h3> + + <p> + In the above example, we replaced the value of field based on another RecordPath. That RecordPath was an "absolute RecordPath," + meaning that it starts with a "slash" character (<code>/</code>) and therefore it specifies the path from the "root" or "outer most" element. + However, sometimes we want to reference a field in such a way that we defined the RecordPath relative to the field being updated. This example + does just that. For each of the siblings given in the "siblings" array, we will replace the sibling's name with their id's. To do so, we will + configure the processor with the following properties: + </p> + + <table> + <tr> + <th>Property Name</th> + <th>Property Value</th> + </tr> + <tr> + <td>Replacement Value Strategy</td> + <td>Record Path Value</td> + </tr> + <tr> + <td>/siblings[*]/name</td> + <td>../id</td> + </tr> + </table> + + <p> + Note that the RecordPath that was given for the value starts with <code>..</code>, which is a reference to the parent. We do this because the field + that we are going to update is the "name" field of the sibling. To get to the associated "id" field, we need to go to the "name" field's parent and then + to its "id" child field. The above example results in the following output: + </p> + +<code> +<pre> + [{ + "id": 17, + "name": "John", + "child": { + "id": "1" + }, + "siblingIds": [4, 8], + "siblings": [ + { "name": "4", "id": 4 }, + { "name": "8", "id": 8 } + ] + }, + { + "id": 98, + "name": "Jane", + "child": { + "id": 2 + }, + "gender": "F", + "siblingIds": [], + "siblings": [] + }] +</pre> +</code> + + + + + <h3>Example 4 - Replace Multiple Values</h3> + + <p> + This example will replace the value of all fields that have the name "id", regardless of + where in the Record hierarchy the field is found. The value that it uses references the Expression Language, + so for this example, let's assume that the incoming FlowFile has an attribute named "replacement.id" that + has a value of "91": + </p> + + <table> + <tr> + <th>Property Name</th> + <th>Property Value</th> + </tr> + <tr> + <td>Replacement Value Strategy</td> + <td>Literal Value</td> + </tr> + <tr> + <td>//id</td> + <td>${replacement.id}</td> + </tr> + </table> + + <p> + This will yield the following output: + </p> + +<code> +<pre> + [{ + "id": 91, + "name": "John", + "child": { + "id": "91" + }, + "siblingIds": [4, 8], + "siblings": [ + { "name": "Jeremy", "id": 91 }, + { "name": "Julia", "id": 91 } + ] + }, + { + "id": 91, + "name": "Jane", + "child": { + "id": 91 + }, + "gender": "F", + "siblingIds": [], + "siblings": [] + }] +</pre> +</code> + + <p> + It is also worth noting that in this example, some of the "id" fields were of type STRING, while + others were of type INT. This is okay because the RecordReaders and RecordWriters should handle + these simple type coercions for us. + </p> + + + + + <h3>Example 5 - Use Expression Language to Modify Value</h3> + + <p> + This example will capitalize the value of all 'name' fields, regardless of + where in the Record hierarchy the field is found. This is done by referencing the 'field.value' variable in the Expression Language. + We can also access the field.name variable and the field.type variable. + </p> + + <table> + <tr> + <th>Property Name</th> + <th>Property Value</th> + </tr> + <tr> + <td>Replacement Value Strategy</td> + <td>Literal Value</td> + </tr> + <tr> + <td>//name</td> + <td>${field.value:toUpper()}</td> + </tr> + </table> + + <p> + This will yield the following output: + </p> + +<code> +<pre> + [{ + "id": 17, + "name": "JOHN", + "child": { + "id": "1" + }, + "siblingIds": [4, 8], + "siblings": [ + { "name": "JEREMY", "id": 4 }, + { "name": "JULIA", "id": 8 } + ] + }, + { + "id": 98, + "name": "JANE", + "child": { + "id": 2 + }, + "gender": "F", + "siblingIds": [], + "siblings": [] + }] +</pre> +</code> + + + </body> +</html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateRecord/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateRecord/index.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateRecord/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.UpdateRecord/index.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>UpdateRecord</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">UpdateRecord</h1><h2>Description: </h2><p>Updates the contents of a FlowFile that contains Record-oriented data (i.e., data that can be read via a RecordReader and written by a RecordWriter). This Processor requires that at least one user-defined Property be added. The name of the Property should indicate a RecordPath that determines the field that should be updated. The value of the Property is either a replacement value (optionally making use of the Expression Language) or is itself a RecordPath that extracts a value from the Record. Whether the Property value is determined to be a RecordPath or a literal value depends on the configuration of the <Replacement Value Strategy> Property.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>update, record, generic, schema, json, csv, avro, log, logs, freeform, text</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Record Reader</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordReaderFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.syslog.Syslog5424Reader/index.html">Syslog5424Reader</a><br/><a href="../../../nifi-record-serialization-services-n ar/1.11.4/org.apache.nifi.grok.GrokReader/index.html">GrokReader</a><br/><a href="../../../nifi-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedReader/index.html">ScriptedReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.json.JsonTreeReader/index.html">JsonTreeReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.syslog.SyslogReader/index.html">SyslogReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.avro.AvroReader/index.html">AvroReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.xml.XMLReader/index.html">XMLReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.json.JsonPathReader/index.html">JsonPathReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.csv.CSVReader/index.html">CSVReader</a><br/><a href="../../../nifi -parquet-nar/1.11.4/org.apache.nifi.parquet.ParquetReader/index.html">ParquetReader</a></td><td id="description">Specifies the Controller Service to use for reading incoming data</td></tr><tr><td id="name"><strong>Record Writer</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSetWriterFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.xml.XMLRecordSetWriter/index.html">XMLRecordSetWriter</a><br/><a href="../../../nifi-parquet-nar/1.11.4/org.apache.nifi.parquet.ParquetRecordSetWriter/index.html">ParquetRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.csv.CSVRecordSetWriter/index.html">CSVRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.text.FreeFormTextRecordSetWriter/index.html">FreeFormTextRecordSetWriter</a><br/><a href="../../../ nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.json.JsonRecordSetWriter/index.html">JsonRecordSetWriter</a><br/><a href="../../../nifi-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html">ScriptedRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.avro.AvroRecordSetWriter/index.html">AvroRecordSetWriter</a></td><td id="description">Specifies the Controller Service to use for writing out the records</td></tr><tr><td id="name"><strong>Replacement Value Strategy</strong></td><td id="default-value">literal-value</td><td id="allowable-values"><ul><li>Literal Value <img src="../../../../../html/images/iconInfo.png" alt="The value entered for a Property (after Expression Language has been evaluated) is the desired value to update the Record Fields with. Expression Language may reference variables 'field.name', 'field.type', and 'field.value' to access information about the field and the val ue of the field being evaluated." title="The value entered for a Property (after Expression Language has been evaluated) is the desired value to update the Record Fields with. Expression Language may reference variables 'field.name', 'field.type', and 'field.value' to access information about the field and the value of the field being evaluated."></img></li><li>Record Path Value <img src="../../../../../html/images/iconInfo.png" alt="The value entered for a Property (after Expression Language has been evaluated) is not the literal value to use but rather is a Record Path that should be evaluated against the Record, and the result of the RecordPath will be used to update the Record. Note that if this option is selected, and the Record Path results in multiple values for a given Record, the input FlowFile will be routed to the 'failure' Relationship." title="The value entered for a Property (after Expression Language has been evaluated) is not the literal value to use but rather is a Record Path that should be evaluated against the Record, and the result of the RecordPath will be used to update the Record. Note that if this option is selected, and the Record Path results in multiple values for a given Record, the input FlowFile will be routed to the 'failure' Relationship."></img></li></ul></td><td id="description">Specifies how to interpret the configured replacement values</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>FlowFiles that are successfully transformed will be routed to this relationship</td></tr><tr><td>failure</td><td>If a FlowFile cannot be transformed from the configured input format to the configured output format, the unchanged FlowFile will be routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>record.index</td><td>Thi s attribute provides the current row index and is only available inside the literal value expression.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.standard.ConvertRecord/index.html">ConvertRecord</a></p></body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1,109 @@ +<!DOCTYPE html> +<html lang="en"> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> +<head> + <meta charset="utf-8"/> + <title>ValidateCsv</title> + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"/> +</head> + +<body> +<!-- Processor Documentation ================================================== --> +<h2>Usage Information</h2> + +<p> + The Validate CSV processor is based on the super-csv library and the concept of + <a href="http://super-csv.github.io/super-csv/cell_processors.html"; target="_blank">Cell Processors</a>. + The corresponding java documentation can be found + <a href="http://super-csv.github.io/super-csv/apidocs/org/supercsv/cellprocessor/ift/CellProcessor.html"; target="_blank">here</a>. +</p> + +<p> + The cell processors cannot be nested (except with Optional which gives the possibility to define a CellProcessor for values + that could be null) and must be defined in a comma-delimited string as the Schema property. +</p> + +<p> + The supported cell processors are: + <ul> + <li>ParseBigDecimal</li> + <li>ParseBool</li> + <li>ParseChar</li> + <li>ParseDate</li> + <li>ParseDouble</li> + <li>ParseInt</li> + <li>Optional</li> + <li>DMinMax</li> + <li>Equals</li> + <li>ForbidSubStr</li> + <li>LMinMax</li> + <li>NotNull</li> + <li>Null</li> + <li>RequireHashCode</li> + <li>RequireSubStr</li> + <li>Strlen</li> + <li>StrMinMax</li> + <li>StrNotNullOrEmpty</li> + <li>StrRegEx</li> + <li>Unique</li> + <li>UniqueHashCode</li> + <li>IsIncludedIn</li> + </ul> +</p> + +<p> + Here are some examples: + + <ul> + <b>Schema property:</b> Null, ParseDate("dd/MM/yyyy"), Optional(ParseDouble())<br /> + <b>Meaning:</b> the input CSV has three columns, the first one can be null and has no specification, the second one must be a date + formatted as expected, and the third one must a double or null (no value). + </ul> + + <ul> + <b>Schema property:</b> ParseBigDecimal(), ParseBool(), ParseChar(), ParseInt(), ParseLong()<br /> + <b>Meaning:</b> the input CSV has five columns, the first one must be a big decimal, the second one must be a boolean, + the third one must be a char, the fourth one must be an integer and the fifth one must be a long. + </ul> + + <ul> + <b>Schema property:</b> Equals(), NotNull(), StrNotNullOrEmpty()<br /> + <b>Meaning:</b> the input CSV has three columns, all the values of the first column must be equal to each other, all the values + of the second column must be not null, and all the values of the third column are not null/empty string values. + </ul> + + <ul> + <b>Schema property:</b> Strlen(4), StrMinMax(3,5), StrRegex("[a-z0-9\\._]+@[a-z0-9\\.]+")<br /> + <b>Meaning:</b> the input CSV has three columns, all the values of the first column must be 4-characters long, all the values + of the second column must be between 3 and 5 characters (inclusive), and all the values of the last column must match + the provided regular expression (email address). + </ul> + + <ul> + <b>Schema property:</b> Unique(), UniqueHashCode()<br /> + <b>Meaning:</b> the input CSV has two columns. All the values of the first column must be unique (all the values are stored in + memory and this can be consuming depending of the input). All the values of the second column must be unique (only hash + codes of the input values are stored to ensure uniqueness). + </ul> + + <ul> + <b>Schema property:</b> ForbidSubStr("test", "tset"), RequireSubStr("test")<br /> + <b>Meaning:</b> the input CSV has two columns. None of the values in the first column must contain one of the provided strings. + And all the values of the second column must contain the provided string. + </ul> +</p> +</body> +</html> Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateCsv/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateCsv/index.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateCsv/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateCsv/index.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ValidateCsv</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ValidateCsv</h1><h2>Description: </h2><p>Validates the contents of FlowFiles against a user-specified CSV schema. Take a look at the additional documentation of this processor for some schema examples.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>csv, schema, validation</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-l anguage-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Schema</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The schema to be used for validation. Is expected a comma-delimited string representing the cell processors to apply. The following cell processors are allowed in the schema definition: [ParseBigDecimal, ParseBool, ParseChar, ParseDate, ParseDouble, ParseInt, ParseLong, Optional, DMinMax, Equals, ForbidSubStr, LMinMax, NotNull, Null, RequireHashCode, RequireSubStr, Strlen, StrMinMax, StrNotNullOrEmpty, StrRegEx, Unique, UniqueHashCode, IsIncludedIn]. Note: cell processors cannot be nested except with Optional.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Header</strong></td><td id="de fault-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">True if the incoming flow file contains a header to ignore, false otherwise.</td></tr><tr><td id="name"><strong>Delimiter character</strong></td><td id="default-value">,</td><td id="allowable-values"></td><td id="description">Character used as 'delimiter' in the incoming data. Example: ,<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Quote character</strong></td><td id="default-value">"</td><td id="allowable-values"></td><td id="description">Character used as 'quote' in the incoming data. Example: "<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>End of line symbols</strong></td><td id="default-value">\n</td><td id="allowable-values"></td><td id="descrip tion">Symbols used as 'end of line' in the incoming data. Example: \n<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Validation strategy</strong></td><td id="default-value">FlowFile validation</td><td id="allowable-values"><ul><li>Line by line validation <img src="../../../../../html/images/iconInfo.png" alt="In case an error is found, the input CSV file will be split into two FlowFiles: one routed to the 'valid' relationship containing all the correct lines and one routed to the 'invalid' relationship containing all the incorrect lines. Take care if choosing this option while using Unique cell processors in schema definition:the first occurrence will be considered valid and the next ones as invalid." title="In case an error is found, the input CSV file will be split into two FlowFiles: one routed to the 'valid' relationship containing all the correct lines and one routed to the 'invalid' relationship containing all the incorrect lines. Take care if choosing this option while using Unique cell processors in schema definition:the first occurrence will be considered valid and the next ones as invalid."></img></li><li>FlowFile validation <img src="../../../../../html/images/iconInfo.png" alt="As soon as an error is found in the CSV file, the validation will stop and the whole flow file will be routed to the 'invalid' relationship. This option offers best performances." title="As soon as an error is found in the CSV file, the validation will stop and the whole flow file will be routed to the 'invalid' relationship. This option offers best performances."></img></li></ul></td><td id="description">Strategy to apply when routing input files to output relationships.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>valid</td><td>FlowFiles that are successfully validated against the schema are routed to this relationship</td></tr><tr><td>invalid</td><td>FlowFiles that are not valid according to the specified schema are routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>count.valid.lines</td><td>If line by line validation, number of valid lines extracted from the source data</td></tr><tr><td>count.invalid.lines</td><td>If line by line validation, number of invalid lines extracted from the source data</td></tr><tr><td>count.total.lines</td><td>If line by line validation, total number of lines in the source data</td></tr><tr><td>validation.error.message</td><td>For flow files routed to invalid, message of the first validation error</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relati onship.<h3>System Resource Considerations:</h3>None specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateRecord/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateRecord/index.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateRecord/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateRecord/index.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ValidateRecord</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ValidateRecord</h1><h2>Description: </h2><p>Validates the Records of an incoming FlowFile against a given schema. All records that adhere to the schema are routed to the "valid" relationship while records that do not adhere to the schema are routed to the "invalid" relationship. It is therefore possible for a single incoming FlowFile to be split into two individual FlowFiles if some records are valid according to the schema and others are not. Any FlowFile that is routed to the "invalid" relationship will emit a ROUTE Provenance Event with the Details field populated to explain why records were i nvalid. In addition, to gain further explanation of why records were invalid, DEBUG-level logging can be enabled for the "org.apache.nifi.processors.standard.ValidateRecord" logger.</p><h3>Tags: </h3><p>record, schema, validate</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Record Reader</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordReaderFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.syslog.Syslog5424Re ader/index.html">Syslog5424Reader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.grok.GrokReader/index.html">GrokReader</a><br/><a href="../../../nifi-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedReader/index.html">ScriptedReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.json.JsonTreeReader/index.html">JsonTreeReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.syslog.SyslogReader/index.html">SyslogReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.avro.AvroReader/index.html">AvroReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.xml.XMLReader/index.html">XMLReader</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.json.JsonPathReader/index.html">JsonPathReader</a><br/><a href="../../../nifi-record-serialization-servic es-nar/1.11.4/org.apache.nifi.csv.CSVReader/index.html">CSVReader</a><br/><a href="../../../nifi-parquet-nar/1.11.4/org.apache.nifi.parquet.ParquetReader/index.html">ParquetReader</a></td><td id="description">Specifies the Controller Service to use for reading incoming data</td></tr><tr><td id="name"><strong>Record Writer</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSetWriterFactory<br/><strong>Implementations: </strong><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.xml.XMLRecordSetWriter/index.html">XMLRecordSetWriter</a><br/><a href="../../../nifi-parquet-nar/1.11.4/org.apache.nifi.parquet.ParquetRecordSetWriter/index.html">ParquetRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.csv.CSVRecordSetWriter/index.html">CSVRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.tex t.FreeFormTextRecordSetWriter/index.html">FreeFormTextRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.json.JsonRecordSetWriter/index.html">JsonRecordSetWriter</a><br/><a href="../../../nifi-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html">ScriptedRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.avro.AvroRecordSetWriter/index.html">AvroRecordSetWriter</a></td><td id="description">Specifies the Controller Service to use for writing out the records. Regardless of the Controller Service schema access configuration, the schema that is used to validate record is used to write the valid results.</td></tr><tr><td id="name">Record Writer for Invalid Records</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSetWriterFactory<br/><strong>Implementations: </strong><a href="../../../nifi-rec ord-serialization-services-nar/1.11.4/org.apache.nifi.xml.XMLRecordSetWriter/index.html">XMLRecordSetWriter</a><br/><a href="../../../nifi-parquet-nar/1.11.4/org.apache.nifi.parquet.ParquetRecordSetWriter/index.html">ParquetRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.csv.CSVRecordSetWriter/index.html">CSVRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.text.FreeFormTextRecordSetWriter/index.html">FreeFormTextRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.json.JsonRecordSetWriter/index.html">JsonRecordSetWriter</a><br/><a href="../../../nifi-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html">ScriptedRecordSetWriter</a><br/><a href="../../../nifi-record-serialization-services-nar/1.11.4/org.apache.nifi.avro.AvroRecordSetWriter/index.html">AvroRecordSetWriter</a></td><td id="descri ption">If specified, this Controller Service will be used to write out any records that are invalid. If not specified, the writer specified by the "Record Writer" property will be used with the schema used to read the input records. This is useful, for example, when the configured Record Writer cannot write data that does not adhere to its schema (as is the case with Avro) or when it is desirable to keep invalid records in their original format while converting valid records to another format.</td></tr><tr><td id="name"><strong>Schema Access Strategy</strong></td><td id="default-value">reader-schema</td><td id="allowable-values"><ul><li>Use Reader's Schema <img src="../../../../../html/images/iconInfo.png" alt="The schema to validate the data against is determined by asking the configured Record Reader for its schema" title="The schema to validate the data against is determined by asking the configured Record Reader for its schema"></img></li><li>Use Schema Name Property <img src=". ./../../../../html/images/iconInfo.png" alt="The schema to validate the data against is determined by looking at the 'Schema Name' Property and looking up the schema in the configured Schema Registry" title="The schema to validate the data against is determined by looking at the 'Schema Name' Property and looking up the schema in the configured Schema Registry"></img></li><li>Use Schema Text Property <img src="../../../../../html/images/iconInfo.png" alt="The schema to validate the data against is determined by looking at the 'Schema Text' Property and parsing the schema as an Avro schema" title="The schema to validate the data against is determined by looking at the 'Schema Text' Property and parsing the schema as an Avro schema"></img></li></ul></td><td id="description">Specifies how to obtain the schema that should be used to validate records</td></tr><tr><td id="name">Schema Registry</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </stro ng><br/>SchemaRegistry<br/><strong>Implementations: </strong><a href="../../../nifi-confluent-platform-nar/1.11.4/org.apache.nifi.confluent.schemaregistry.ConfluentSchemaRegistry/index.html">ConfluentSchemaRegistry</a><br/><a href="../../../nifi-registry-nar/1.11.4/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html">AvroSchemaRegistry</a><br/><a href="../../../nifi-hwx-schema-registry-nar/1.11.4/org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry/index.html">HortonworksSchemaRegistry</a></td><td id="description">Specifies the Controller Service to use for the Schema Registry. This is necessary only if the Schema Access Strategy is set to "Use 'Schema Name' Property".</td></tr><tr><td id="name">Schema Name</td><td id="default-value">${schema.name}</td><td id="allowable-values"></td><td id="description">Specifies the name of the schema to lookup in the Schema Registry property<br/><strong>Supports Expression Language: true (will be evaluated using f low file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Text</td><td id="default-value">${avro.schema}</td><td id="allowable-values"></td><td id="description">The text of an Avro-formatted Schema<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Allow Extra Fields</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If the incoming data has fields that are not present in the schema, this property determines whether or not the Record is valid. If true, the Record is still valid. If false, the Record will be invalid due to the extra fields.</td></tr><tr><td id="name"><strong>Strict Type Checking</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">If the incoming data has a Record where a f ield is not of the correct type, this property determine whether how to handle the Record. If true, the Record will still be considered invalid. If false, the Record will be considered valid and the field will be coerced into the correct type (if possible, according to the type coercion supported by the Record Writer).</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>valid</td><td>Records that are valid according to the schema will be routed to this relationship</td></tr><tr><td>invalid</td><td>Records that are not valid according to the schema will be routed to this relationship</td></tr><tr><td>failure</td><td>If the records cannot be read, validated, or written, for any reason, the original FlowFile will be routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>mime.type</td> <td>Sets the mime.type attribute to the MIME Type specified by the Record Writer</td></tr><tr><td>record.count</td><td>The number of records in the FlowFile routed to a relationship</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateXml/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateXml/index.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateXml/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.ValidateXml/index.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>ValidateXml</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">ValidateXml</h1><h2>Description: </h2><p>Validates the contents of FlowFiles against a user-specified XML Schema file</p><h3>Tags: </h3><p>xml, schema, validation, xsd</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th> Description</th></tr><tr><td id="name"><strong>Schema File</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The path to the Schema file that is to be used for validation<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>valid</td><td>FlowFiles that are successfully validated against the schema are routed to this relationship</td></tr><tr><td>invalid</td><td>FlowFiles that are not valid according to the specified schema are routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>validatexml.invalid.error</td><td>If the flow file is routed to the invalid relationship the attribute will contain the error message resulting from the valida tion failure.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.Wait/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.Wait/additionalDetails.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.Wait/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.Wait/additionalDetails.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1,277 @@ +<!DOCTYPE html> +<html lang="en"> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> +<head> + <meta charset="utf-8"/> + <title>ValidateCsv</title> + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"/> + <style> + table td:first-child {text-align: center;} + </style> + +</head> + +<body> + +<h2>Best practices to handle multiple signal ids at a Wait processor</h2> + +When a Wait processor is expected to process multiple signal ids, by configuring 'Release Signal Identifier' with a FlowFile attribute Expression Language, there are few things to consider in order to get the expected result. Processor configuration can vary based on your requirement. + +Also, you will need to have high level understanding on how Wait processor works: +<ul> + <li>Wait processor only process a single signal id at a time</li> + <li>How frequent Wait processor runs is defined at 'Run Schedule'</li> + <li>Which FlowFile is processed is determined by Prioritizer</li> + <li>Not limited to Wait processor, but for all processors, the order of queued FlowFiles in a connection is undefined if no Prioritizer is set</li> +</ul> + + +See following sections for common patterns +<ul> + <li><a href="additionalDetails.html#asap">Release any FlowFile as soon as its signal is notified</a></li> + <li><a href="additionalDetails.html#higher-priority">Release higher priority FlowFiles in each signal id</a></li> +</ul> + +<h3 id="asap">Release any FlowFile as soon as its signal is notified</h3> + +This is the most common use case. +FlowFiles are independent and can be released in any order. + +<h4>Important configurations:</h4> +<ul> + <li>Use FirstInFirstOutPrioritizer (FIFO) at 'wait' relationship (or the incoming connection if 'Wait Mode' is 'Keep in the upstream connection)</li> +</ul> + +The following table illustrates the notified signal ids, queued FlowFiles and what will happen at each Wait run cycle. + +<table> + <tbody> + <tr> + <th># of Wait run</th> + <th>Notified Signals</th> + <th>Queue Index (FIFO)</th> + <th>FlowFile UUID</th> + <th>Signal ID</th> + <th> </th> + </tr> + <tr> + <td rowspan="3">1</td> + <td rowspan="3">B</td> + <td>1</td> + <td>a</td> + <td>A</td> + <td>This FlowFile is processed. But its signal is not found, and will be re-queued at the end of the queue.</td> + </tr> + <tr> + <td>2</td> + <td>b</td> + <td>B</td> + <td> </td> + </tr> + <tr> + <td>3</td> + <td>c</td> + <td>C</td> + <td> </td> + </tr> + <tr> + <td rowspan="3">2</td> + <td rowspan="3">B</td> + <td>1</td> + <td>b</td> + <td>B</td> + <td>This FlowFile is processed and since its signal is notified, this one will be released to 'success'.</td> + </tr> + <tr> + <td>2</td> + <td>c</td> + <td>C</td> + <td> </td> + </tr> + <tr> + <td>3</td> + <td>a</td> + <td>A</td> + <td> </td> + </tr> + <tr> + <td rowspan="3">3</td> + <td rowspan="3"> </td> + <td>1</td> + <td>c</td> + <td>C</td> + <td>This FlowFile will be processed at the next run.</td> + </tr> + <tr> + <td>2</td> + <td>a</td> + <td>A</td> + <td> </td> + </tr> + </tbody> +</table> + + + +<h3 id="higher-priority">Release higher priority FlowFiles in each signal id</h3> + +Multiple FlowFiles share the same signal id, and the order of releasing a FlowFile is important. + +<h4>Important configurations:</h4> +<ul> + <li>Use a (or set of a) Prioritizer(s) suites your need other than FIFO, at 'wait' relationship (or the incoming connection if 'Wait Mode' is 'Keep in the upstream connection), e.g. PriorityPrioritizer</li> + <li>Specify adequate 'Wait Penalty Duration', e.g. "3 sec", </li> + <li>'Wait Penalty Duration' should be grater than 'Run Schedule', e.g "3 sec" > "1 sec"</li> + <li>Increase 'Run Duration' to avoid the limitation of number of signal ids (see the <a href="additionalDetails.html#run-duration">note</a> below)</li> +</ul> + +The following table illustrates the notified signal ids, queued FlowFiles and what will happen at each Wait run cycle. +The example uses PriorityPrioritizer to control the order of processing FlowFiles within a signal id. + +If 'Wait Penalty Duration' is configured, Wait processor tracks unreleased signal ids and their penalty representing when they will be checked again. + +<table> + <tbody> + <tr> + <th># of Wait run</th> + <th>Notified Signals</th> + <th>Signal Penalties</th> + <th>Queue Index (via 'priority' attribute)</th> + <th>FlowFile UUID</th> + <th>Signal ID</th> + <th>'priority' attr</th> + <th> </th> + </tr> + <tr> + <td rowspan="3">1 (00:01)</td> + <td rowspan="3">B</td> + <td rowspan="3"> </td> + <td>1</td> + <td>a-1</td> + <td>A</td> + <td>1</td> + <td>This FlowFile is processed. But its signal is not found. Penalized.</td> + </tr> + <tr> + <td>2</td> + <td>b-1</td> + <td>B</td> + <td>1</td> + <td>Since a-1 and b-1 have the same priority '1', b-1 may be processed before a-1. You can add another Prioritizer to define more specific ordering.</td> + </tr> + <tr> + <td>3</td> + <td>b-2</td> + <td>B</td> + <td>2</td> + <td> </td> + </tr> + <tr> + <td rowspan="3">2 (00:02)</td> + <td rowspan="3">B</td> + <td rowspan="3">A (00:04)</td> + <td>1</td> + <td>a-1</td> + <td>A</td> + <td>1</td> + <td>This FlowFile is the first one according to the configured Prioritizer, but the signal id is penalized. So, this FlowFile is skipped at this execution.</td> + </tr> + <tr> + <td>2</td> + <td>b-1</td> + <td>B</td> + <td>1</td> + <td>This FlowFile is processed.</td> + </tr> + <tr> + <td>3</td> + <td>b-2</td> + <td>B</td> + <td>2</td> + <td> </td> + </tr> + <tr> + <td rowspan="2">3 (00:03)</td> + <td rowspan="2"> </td> + <td rowspan="2">A (00:04)</td> + <td>1</td> + <td>a-1</td> + <td>A</td> + <td>1</td> + <td>This FlowFile is the first one but is still penalized.</td> + </tr> + <tr> + <td>2</td> + <td>b-2</td> + <td>B</td> + <td>2</td> + <td>This FlowFile is processed, but its signal is not notified yet, thus will be penalized.</td> + </tr> + <tr> + <td rowspan="2">4 (00:04)</td> + <td rowspan="2"> </td> + <td rowspan="2">B (00:06)</td> + <td>1</td> + <td>a-1</td> + <td>A</td> + <td>1</td> + <td>This FlowFile is no longer penalized, and get processed. But its signal is not notified yet, thus will be penalized again.</td> + </tr> + <tr> + <td>2</td> + <td>b-2</td> + <td>B</td> + <td>2</td> + <td> </td> + </tr> + </tbody> +</table> + +<h4 id="run-duration">The importance of 'Run Duration' when 'Wait Penalty Duration' is used</h4> + +<p> +There are limitation of number of signals can be checked based on the combination of 'Run Schedule' and 'Wait Penalize Duration'. +If this limitation is engaged, some FlowFiles may not be processed and remain in the 'wait' relationship even if their signal ids are notified. +Let's say Wait is configured with: +</p> + +<ul> + <li>Run Schedule = 1 sec</li> + <li>Wait Penalize Duration = 3 sec</li> + <li>Release Signal Identifier = ${uuid}</li> +</ul> + +<p> +And there are 5 FlowFiles F1, F2 ... F5 in the 'wait' relationship. +Then the signal for F5 is notified. +Wait will work as follows: +</p> + +<ul> + <li>At 00:00 Wait checks the signal for F1, not found, and penalize F1 (till 00:03)</li> + <li>At 00:01 Wait checks the signal for F2, not found, and penalize F2 (till 00:04)</li> + <li>At 00:02 Wait checks the signal for F3, not found, and penalize F3 (till 00:05)</li> + <li>At 00:03 Wait checks the signal for F4, not found, and penalize F4 (till 00:06)</li> + <li>At 00:04 Wait checks the signal for F1 again, because it's not penalized any longer</li> +</ul> +Repeat above cycle, thus F5 will not be released until one of F1 ... F4 is released. + +<p> +To mitigate such limitation, increasing 'Run Duration' is recommended. By increasing 'Run Duration', Wait processor can keep being scheduled for that duration. For example, with 'Run Duration' 500 ms, Wait should be able to loop through all 5 queued FlowFiles at a single run. +</p> + +</body> +</html> Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.Wait/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.Wait/index.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.Wait/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.Wait/index.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>Wait</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">Wait</h1><h2>Description: </h2><p>Routes incoming FlowFiles to the 'wait' relationship until a matching release signal is stored in the distributed cache from a corresponding Notify processor. When a matching release signal is identified, a waiting FlowFile is routed to the 'success' relationship, with attributes copied from the FlowFile that produced the release signal from the Notify processor. The release signal entry is then removed from the cache. Waiting FlowFiles will be routed to 'expired' if they exceed the Expiration Duration. If you need to wait for more than one signal, specify the desired num ber of signals via the 'Target Signal Count' property. This is particularly useful with processors that split a source FlowFile into multiple fragments, such as SplitText. In order to wait for all fragments to be processed, connect the 'original' relationship to a Wait processor, and the 'splits' relationship to a corresponding Notify processor. Configure the Notify and Wait processors to use the '${fragment.identifier}' as the value of 'Release Signal Identifier', and specify '${fragment.count}' as the value of 'Target Signal Count' in the Wait processor.It is recommended to use a prioritizer (for instance First In First Out) when using the 'wait' relationship as a loop.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>map, cache, wait, hold, distributed, signal, release</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The t able also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Release Signal Identifier</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the release signal cache key<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Target Signal Count</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">A value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to deter mine the target signal count. This processor checks whether the signal count has reached this number. If Signal Counter Name is specified, this processor checks a particular counter, otherwise checks against total count in a signal.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Signal Counter Name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the signal counter name. If not specified, this processor checks the total count in a signal.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Wait Buffer Count</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">Specif y the maximum number of incoming FlowFiles that can be buffered to check whether it can move forward. The more buffer can provide the better performance, as it reduces the number of interactions with cache service by grouping FlowFiles by signal identifier. Only a signal identifier can be processed at a processor execution.</td></tr><tr><td id="name"><strong>Releasable FlowFile Count</strong></td><td id="default-value">1</td><td id="allowable-values"></td><td id="description">A value, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the releasable FlowFile count. This specifies how many FlowFiles can be released when a target count reaches target signal count. Zero (0) has a special meaning, any number of FlowFiles can be released as long as signal count matches target.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr> <td id="name"><strong>Expiration Duration</strong></td><td id="default-value">10 min</td><td id="allowable-values"></td><td id="description">Indicates the duration after which waiting FlowFiles will be routed to the 'expired' relationship</td></tr><tr><td id="name"><strong>Distributed Cache Service</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>AtomicDistributedMapCacheClient<br/><strong>Implementations: </strong><a href="../../../nifi-couchbase-nar/1.11.4/org.apache.nifi.couchbase.CouchbaseMapCacheClient/index.html">CouchbaseMapCacheClient</a><br/><a href="../../../nifi-distributed-cache-services-nar/1.11.4/org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService/index.html">DistributedMapCacheClientService</a><br/><a href="../../../nifi-hbase_2-client-service-nar/1.11.4/org.apache.nifi.hbase.HBase_2_ClientMapCacheService/index.html">HBase_2_ClientMapCacheService</a><br/><a href="../../../nifi-redis- nar/1.11.4/org.apache.nifi.redis.service.RedisDistributedMapCacheClientService/index.html">RedisDistributedMapCacheClientService</a><br/><a href="../../../nifi-hbase_1_1_2-client-service-nar/1.11.4/org.apache.nifi.hbase.HBase_1_1_2_ClientMapCacheService/index.html">HBase_1_1_2_ClientMapCacheService</a></td><td id="description">The Controller Service that is used to check for release signals from a corresponding Notify processor</td></tr><tr><td id="name"><strong>Attribute Copy Mode</strong></td><td id="default-value">keeporiginal</td><td id="allowable-values"><ul><li>Replace if present <img src="../../../../../html/images/iconInfo.png" alt="When cached attributes are copied onto released FlowFiles, they replace any matching attributes." title="When cached attributes are copied onto released FlowFiles, they replace any matching attributes."></img></li><li>Keep original <img src="../../../../../html/images/iconInfo.png" alt="Attributes on released FlowFiles are not overwritten by copi ed cached attributes." title="Attributes on released FlowFiles are not overwritten by copied cached attributes."></img></li></ul></td><td id="description">Specifies how to handle attributes copied from FlowFiles entering the Notify processor</td></tr><tr><td id="name"><strong>Wait Mode</strong></td><td id="default-value">wait</td><td id="allowable-values"><ul><li>Transfer to wait relationship <img src="../../../../../html/images/iconInfo.png" alt="Transfer a FlowFile to the 'wait' relationship when whose release signal has not been notified yet. This mode allows other incoming FlowFiles to be enqueued by moving FlowFiles into the wait relationship. It is recommended to set a prioritizer (for instance First In First Out) on the 'wait' relationship." title="Transfer a FlowFile to the 'wait' relationship when whose release signal has not been notified yet. This mode allows other incoming FlowFiles to be enqueued by moving FlowFiles into the wait relationship. It is recommended to set a prioritizer (for instance First In First Out) on the 'wait' relationship."></img></li><li>Keep in the upstream connection <img src="../../../../../html/images/iconInfo.png" alt="Transfer a FlowFile to the upstream connection where it comes from when whose release signal has not been notified yet. This mode helps keeping upstream connection being full so that the upstream source processor will not be scheduled while back-pressure is active and limit incoming FlowFiles. " title="Transfer a FlowFile to the upstream connection where it comes from when whose release signal has not been notified yet. This mode helps keeping upstream connection being full so that the upstream source processor will not be scheduled while back-pressure is active and limit incoming FlowFiles. "></img></li></ul></td><td id="description">Specifies how to handle a FlowFile waiting for a notify signal</td></tr><tr><td id="name">Wait Penalty Duration</td><td id="default-value"></td><td id="allowable-values"></td> <td id="description">If configured, after a signal identifier got processed but did not meet the release criteria, the signal identifier is penalized and FlowFiles having the signal identifier will not be processed again for the specified period of time, so that the signal identifier will not block others to be processed. This can be useful for use cases where a Wait processor is expected to process multiple signal identifiers, and each signal identifier has multiple FlowFiles, and also the order of releasing FlowFiles is important within a signal identifier. The FlowFile order can be configured with Prioritizers. IMPORTANT: There is a limitation of number of queued signals can be processed, and Wait processor may not be able to check all queued signal ids. See additional details for the best practice.</td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>expired</td><td>A FlowFile that has exceeded the configured Expira tion Duration will be routed to this relationship</td></tr><tr><td>success</td><td>A FlowFile with a matching release signal in the cache will be routed to this relationship</td></tr><tr><td>wait</td><td>A FlowFile with no matching release signal in the cache will be routed to this relationship</td></tr><tr><td>failure</td><td>When the cache cannot be reached, or if the Release Signal Identifier evaluates to null or empty, FlowFiles will be routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>wait.start.timestamp</td><td>All FlowFiles will have an attribute 'wait.start.timestamp', which sets the initial epoch timestamp when the file first entered this processor. This is used to determine the expiration time of the FlowFile. This attribute is not written when the FlowFile is transferred to failure or success</td></tr><tr><td>wait.counter.< counterName></td><td>If a signal exists when the processor runs, each count value in the signal is copied.</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../../../nifi-distributed-cache-services-nar/1.11.4/org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService/index.html">DistributedMapCacheClientService</a>, <a href="../../../nifi-distributed-cache-services-nar/1.11.4/org.apache.nifi.distributed.cache.server.map.DistributedMapCacheServer/index.html">DistributedMapCacheServer</a>, <a href="../org.apache.nifi.processors.standard.Notify/index.html">Notify</a></p></body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/additionalDetails.html?rev=1875518&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.reporting.ganglia.StandardGangliaReporter/additionalDetails.html Sun Mar 22 13:18:15 2020 @@ -0,0 +1,41 @@ +<!DOCTYPE html> +<html lang="en"> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <head> + <meta charset="utf-8" /> + <title>StandardGangliaReporter</title> + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css" /> + </head> + + <body> + <h2>Description:</h2> + + <p>Reporting Task that reports metrics to a Ganglia server. The following metrics are reported: + </p> + + <ul> + <li><strong>FlowFiles In (5 mins)</strong>: The number of FlowFiles received via Site-to-Site in the last 5 minutes</li> + <li><strong>Bytes In (5 mins)</strong>: The number of bytes received via Site-to-Site in the last 5 minutes</li> + <li><strong>FlowFiles Out (5 mins)</strong>: The number of FlowFiles pulled from Output Ports via Site-to-Site in the last 5 minutes</li> + <li><strong>Bytes Out (5 mins)</strong>: The number of bytes pulled from Output Ports via Site-to-Site in the last 5 minutes</li> + <li><strong>Bytes Read (5 mins)</strong>: The number of bytes read from disk by NiFi in the last 5 minutes</li> + <li><strong>Bytes Written (5 mins)</strong>: The number of bytes written to disk by NiFi in the last 5 minutes</li> + <li><strong>FlowFiles Queued</strong>: The total number of FlowFiles currently queued on the system at the point in time at which the Reporting Task is run</li> + <li><strong>Bytes Queued</strong>: The total number of bytes allocated by the FlowFiles that are currently queued on the system at the point in time at which the Reporting Task is run</li> + <li><strong>Active Threads</strong>: The number of threads actively running at the point in time at which the Reporting Task is run</li> + </ul> + </body> +</html>
