Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/additionalDetails.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/additionalDetails.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1,531 @@ +<!DOCTYPE html> +<html lang="en"> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <head> + <meta charset="utf-8"/> + <title>XMLReader</title> + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"/> + </head> + + <body> + <p> + The XMLReader Controller Service reads XML content and creates Record objects. The Controller Service + must be configured with a schema that describes the structure of the XML data. Fields in the XML data + that are not defined in the schema will be skipped. Depending on whether the property "Expect Records as Array" + is set to "false" or "true", the reader either expects a single record or an array of records for each FlowFile. + </p> + + <p> + Example: Single record + </p> + <code> + <pre> + <record> + <field1>content</field1> + <field2>content</field2> + </record> + </pre> + </code> + + <p> + An array of records has to be enclosed by a root tag. + Example: Array of records + </p> + + <code> + <pre> + <root> + <record> + <field1>content</field1> + <field2>content</field2> + </record> + <record> + <field1>content</field1> + <field2>content</field2> + </record> + </root> + </pre> + </code> + + <h2>Example: Simple Fields</h2> + + <p> + The simplest kind of data within XML data are tags / fields only containing content (no attributes, no embedded tags). + They can be described in the schema by simple types (e. g. INT, STRING, ...). + </p> + + <code> + <pre> + <root> + <record> + <simple_field>content</simple_field> + </record> + </root> + </pre> + </code> + + <p> + This record can be described by a schema containing one field (e. g. of type string). By providing this schema, + the reader expects zero or one occurrences of "simple_field" in the record. + </p> + + <code> + <pre> + { + "namespace": "nifi", + "name": "test", + "type": "record", + "fields": [ + { "name": "simple_field", "type": "string" } + ] + } + </pre> + </code> + + <h2>Example: Arrays with Simple Fields</h2> + + <p> + Arrays are considered as repetitive tags / fields in XML data. For the following XML data, "array_field" is considered + to be an array enclosing simple fields, whereas "simple_field" is considered to be a simple field not enclosed in + an array. + </p> + + <code> + <pre> + <record> + <array_field>content</array_field> + <array_field>content</array_field> + <simple_field>content</simple_field> + </record> + </pre> + </code> + + <p> + This record can be described by the following schema: + </p> + + <code> + <pre> + { + "namespace": "nifi", + "name": "test", + "type": "record", + "fields": [ + { "name": "array_field", "type": + { "type": "array", "items": string } + }, + { "name": "simple_field", "type": "string" } + ] + } + </pre> + </code> + + <p> + If a field in a schema is embedded in an array, the reader expects zero, one or more occurrences of the field + in a record. The field "array_field" principally also could be defined as a simple field, but then the second occurrence + of this field would replace the first in the record object. Moreover, the field "simple_field" could also be defined + as an array. In this case, the reader would put it into the record object as an array with one element. + </p> + + <h2>Example: Tags with Attributes</h2> + + <p> + XML fields frequently not only contain content, but also attributes. The following record contains a field with + an attribute "attr" and content: + </p> + + <code> + <pre> + <record> + <field_with_attribute attr="attr_content">content of field</field_with_attribute> + </record> + </pre> + </code> + + <p> + To parse the content of the field "field_with_attribute" together with the attribute "attr", two requirements have + to be fulfilled: + </p> + + <ul> + <li>In the schema, the field has to be defined as record.</li> + <li>The property "Field Name for Content" has to be set.</li> + <li>As an option, the property "Attribute Prefix" also can be set.</li> + </ul> + + <p> + For the example above, the following property settings are assumed: + </p> + + <table> + <tr> + <th>Property Name</th> + <th>Property Value</th> + </tr> + <tr> + <td>Field Name for Content</td> + <td><code>field_name_for_content</code></td> + </tr> + <tr> + <td>Attribute Prefix</td> + <td><code>prefix_</code></td> + </tr> + </table> + + <p> + The schema can be defined as follows: + </p> + + <code> + <pre> + { + "name": "test", + "namespace": "nifi", + "type": "record", + "fields": [ + { + "name": "field_with_attribute", + "type": { + "name": "RecordForTag", + "type": "record", + "fields" : [ + {"name": "attr", "type": "string"}, + {"name": "field_name_for_content", "type": "string"} + ] + } + ] + } + </pre> + </code> + + <p> + Note that the field "field_name_for_content" not only has to be defined in the property section, but also in the + schema, whereas the prefix for attributes is not part of the schema. It will be appended when an attribute named + "attr" is found at the respective position in the XML data and added to the record. The record object of the above + example will be structured as follows: + </p> + + <code> + <pre> + Record ( + Record "field_with_attribute" ( + RecordField "prefix_attr" = "attr_content", + RecordField "field_name_for_content" = "content of field" + ) + ) + </pre> + </code> + + <p> + Principally, the field "field_with_attribute" could also be defined as a simple field. In this case, the attributes + simply would be ignored. Vice versa, the simple field in example 1 above could also be defined as a record (assuming that + the property "Field Name for Content" is set. + </p> + + <h2>Example: Tags within tags</h2> + + <p> + XML data is frequently nested. In this case, tags enclose other tags: + </p> + + <code> + <pre> + <record> + <field_with_embedded_fields attr="attr_content"> + <embedded_field>embedded content</embedded_field> + <another_embedded_field>another embedded content</another_embedded_field> + </field_with_embedded_fields> + </record> + </pre> + </code> + + <p> + The enclosing fields always have to be defined as records, irrespective whether they include attributes to be + parsed or not. In this example, the tag "field_with_embedded_fields" encloses the fields "embedded_field" and + "another_embedded_field", which are both simple fields. The schema can be defined as follows: + </p> + + <code> + <pre> + { + "name": "test", + "namespace": "nifi", + "type": "record", + "fields": [ + { + "name": "field_with_embedded_fields", + "type": { + "name": "RecordForEmbedded", + "type": "record", + "fields" : [ + {"name": "attr", "type": "string"}, + {"name": "embedded_field", "type": "string"}, + {"name": "another_embedded_field", "type": "string"} + ] + } + ] + } + </pre> + </code> + + <p> + Notice that this case does not require the property "Field Name for Content" to be set as this is only required + for tags containing attributes and content. + </p> + + <h2>Example: Array of records</h2> + + <p> + For further explanation of the logic of this reader, an example of an array of records shall be demonstrated. + The following record contains the field "array_field", which repeatedly occurs. The field contains two + embedded fields. + </p> + + <code> + <pre> + <record> + <array_field> + <embedded_field>embedded content 1</embedded_field> + <another_embedded_field>another embedded content 1</another_embedded_field> + </array_field> + <array_field> + <embedded_field>embedded content 2</embedded_field> + <another_embedded_field>another embedded content 2</another_embedded_field> + </array_field> + </record> + </pre> + </code> + + <p> + This XML data can be parsed similarly to the data in example 4. However, the record defined in the schema of + example 4 has to be embedded in an array. + </p> + + <code> + <pre> + { + "namespace": "nifi", + "name": "test", + "type": "record", + "fields": [ + { "name": "array_field", + "type": { + "type": "array", + "items": { + "name": "RecordInArray", + "type": "record", + "fields" : [ + {"name": "embedded_field", "type": "string"}, + {"name": "another_embedded_field", "type": "string"} + ] + } + } + } + ] + } + </pre> + </code> + + <h2>Example: Array in record</h2> + + <p> + In XML data, arrays are frequently enclosed by tags: + </p> + + <code> + <pre> + <record> + <field_enclosing_array> + <element>content 1</element> + <element>content 2</element> + </field_enclosing_array> + <field_without_array> content 3</field_without_array> + </record> + </pre> + </code> + + <p> + For the schema, embedded tags have to be described by records. Therefore, the field "field_enclosing_array" + is a record that embeds an array with elements of type string: + </p> + + <code> + <pre> + { + "namespace": "nifi", + "name": "test", + "type": "record", + "fields": [ + { "name": "field_enclosing_array", + "type": { + "name": "EmbeddedRecord", + "type": "record", + "fields" : [ + { + "name": "element", + "type": { + "type": "array", + "items": "string" + } + } + ] + } + }, + { "name": "field_without_array", "type": "string" } + ] + } + </pre> + </code> + + + <h2>Example: Maps</h2> + + <p> + A map is a field embedding fields with different names: + </p> + + <code> + <pre> + <record> + <map_field> + <field1>content</field1> + <field2>content</field2> + ... + </map_field> + <simple_field>content</simple_field> + </record> + </pre> + </code> + + <p> + This data can be processed using the following schema: + </p> + + <code> + <pre> + { + "namespace": "nifi", + "name": "test", + "type": "record", + "fields": [ + { "name": "map_field", "type": + { "type": "map", "items": string } + }, + { "name": "simple_field", "type": "string" } + ] + } + </pre> + </code> + + + <h2>Schema Inference</h2> + + <p> + While NiFi's Record API does require that each Record have a schema, it is often convenient to infer the schema based on the values in the data, + rather than having to manually create a schema. This is accomplished by selecting a value of "Infer Schema" for the "Schema Access Strategy" property. + When using this strategy, the Reader will determine the schema by first parsing all data in the FlowFile, keeping track of all fields that it has encountered + and the type of each field. Once all data has been parsed, a schema is formed that encompasses all fields that have been encountered. + </p> + + <p> + A common concern when inferring schemas is how to handle the condition of two values that have different types. For example, consider a FlowFile with the following two records: + </p> + <code><pre> +<root> + <record> + <name>John</name> + <age>8</age> + <values>N/A</values> + </record> + <record> + <name>Jane</name> + <age>Ten</age> + <values>8</values> + <values>Ten</values> + </record> +</root> +</pre></code> + + <p> + It is clear that the "name" field will be inferred as a STRING type. However, how should we handle the "age" field? Should the field be an CHOICE between INT and STRING? Should we + prefer LONG over INT? Should we just use a STRING? Should the field be considered nullable? + </p> + + <p> + To help understand how this Record Reader infers schemas, we have the following list of rules that are followed in the inference logic: + </p> + + <ul> + <li>All fields are inferred to be nullable.</li> + <li> + When two values are encountered for the same field in two different records (or two values are encountered for an ARRAY type), the inference engine prefers + to use a "wider" data type over using a CHOICE data type. A data type "A" is said to be wider than data type "B" if and only if data type "A" encompasses all + values of "B" in addition to other values. For example, the LONG type is wider than the INT type but not wider than the BOOLEAN type (and BOOLEAN is also not wider + than LONG). INT is wider than SHORT. The STRING type is considered wider than all other types with the Exception of MAP, RECORD, ARRAY, and CHOICE. + </li> + <li> + If two values are encountered for the same field in two different records (or two values are encountered for an ARRAY type), but neither value is of a type that + is wider than the other, then a CHOICE type is used. In the example above, the "values" field will be inferred as a CHOICE between a STRING or an ARRRAY<STRING>. + </li> + <li> + If the "Time Format," "Timestamp Format," or "Date Format" properties are configured, any value that would otherwise be considered a STRING type is first checked against + the configured formats to see if it matches any of them. If the value matches the Timestamp Format, the value is considered a Timestamp field. If it matches the Date Format, + it is considered a Date field. If it matches the Time Format, it is considered a Time field. In the unlikely event that the value matches more than one of the configured + formats, they will be matched in the order: Timestamp, Date, Time. I.e., if a value matched both the Timestamp Format and the Date Format, the type that is inferred will be + Timestamp. Because parsing dates and times can be expensive, it is advisable not to configure these formats if dates, times, and timestamps are not expected, or if processing + the data as a STRING is acceptable. For use cases when this is important, though, the inference engine is intelligent enough to optimize the parsing by first checking several + very cheap conditions. For example, the string's length is examined to see if it is too long or too short to match the pattern. This results in far more efficient processing + than would result if attempting to parse each string value as a timestamp. + </li> + <li>The MAP type is never inferred. Instead, the RECORD type is used.</li> + <li>If two elements exist with the same name and the same parent (i.e., two sibling elements have the same name), the field will be inferred to be of type ARRAY.</li> + <li>If a field exists but all values are null, then the field is inferred to be of type STRING.</li> + </ul> + + + + <h2>Caching of Inferred Schemas</h2> + + <p> + This Record Reader requires that if a schema is to be inferred, that all records be read in order to ensure that the schema that gets inferred is applicable for all + records in the FlowFile. However, this can become expensive, especially if the data undergoes many different transformations. To alleviate the cost of inferring schemas, + the Record Reader can be configured with a "Schema Inference Cache" by populating the property with that name. This is a Controller Service that can be shared by Record + Readers and Record Writers. + </p> + + <p> + Whenever a Record Writer is used to write data, if it is configured with a "Schema Cache," it will also add the schema to the Schema Cache. This will result in an + identifier for that schema being added as an attribute to the FlowFile. + </p> + + <p> + Whenever a Record Reader is used to read data, if it is configured with a "Schema Inference Cache", it will first look for a "schema.cache.identifier" attribute on the FlowFile. + If the attribute exists, it will use the value of that attribute to lookup the schema in the schema cache. If it is able to find a schema in the cache with that identifier, + then it will use that schema instead of reading, parsing, and analyzing the data to infer the schema. If the attribute is not available on the FlowFile, or if the attribute is + available but the cache does not have a schema with that identifier, then the Record Reader will proceed to infer the schema as described above. + </p> + + <p> + The end result is that users are able to chain together many different Processors to operate on Record-oriented data. Typically, only the first such Processor in the chain will + incur the "penalty" of inferring the schema. For all other Processors in the chain, the Record Reader is able to simply lookup the schema in the Schema Cache by identifier. + This allows the Record Reader to infer a schema accurately, since it is inferred based on all data in the FlowFile, and still allows this to happen efficiently since the schema + will typically only be inferred once, regardless of how many Processors handle the data. + </p> + + + + </body> +</html>
Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/index.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLReader/index.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>XMLReader</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">XMLReader</h1><h2>Description: </h2><p>Reads XML content and creates Record objects. Records are expected in the second level of XML data, embedded in an enclosing root tag.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>xml, record, reader, parser</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi E xpression Language</a>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Schema Access Strategy</strong></td><td id="default-value">infer-schema</td><td id="allowable-values"><ul><li>Use 'Schema Name' Property <img src="../../../../../html/images/iconInfo.png" alt="The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service." title="The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service."></img></li><li>Use 'Schema Text' Property <img src="../../../../../html/images/iconInfo.png" alt="The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions." title="The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions."></img></li><li>HWX Schema Reference Attributes <img src="../../../../../html/images/iconInfo.png" alt="The FlowFile contains 3 Attributes that will be used to lookup a Schema from the configured Schema Registry: 'schema.identifier', 'schema.version', and 'schema.protocol.version'" title="The FlowFile contains 3 Attributes that will be used to lookup a Schema from the configured Schema Registry: 'schema.identifier', 'schema.version', and 'schema.protocol.version'"></img></li><li>HWX Content-Encoded Schema Reference <img src="../../../../../html/images/iconInfo.png" alt="The content of the FlowFile contains a reference to a schema in the Schema Registry service. T he reference is encoded as a single byte indicating the 'protocol version', followed by 8 bytes indicating the schema identifier, and finally 4 bytes indicating the schema version, as per the Hortonworks Schema Registry serializers and deserializers, found at https://github.com/hortonworks/registry"; title="The content of the FlowFile contains a reference to a schema in the Schema Registry service. The reference is encoded as a single byte indicating the 'protocol version', followed by 8 bytes indicating the schema identifier, and finally 4 bytes indicating the schema version, as per the Hortonworks Schema Registry serializers and deserializers, found at https://github.com/hortonworks/registry";></img></li><li>Confluent Content-Encoded Schema Reference <img src="../../../../../html/images/iconInfo.png" alt="The content of the FlowFile contains a reference to a schema in the Schema Registry service. The reference is encoded as a single 'Magic Byte' followed by 4 bytes representing the identifier of the schema, as outlined at http://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html. This is based on version 3.2.x of the Confluent Schema Registry." title="The content of the FlowFile contains a reference to a schema in the Schema Registry service. The reference is encoded as a single 'Magic Byte' followed by 4 bytes representing the identifier of the schema, as outlined at http://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html. This is based on version 3.2.x of the Confluent Schema Registry."></img></li><li>Infer Schema <img src="../../../../../html/images/iconInfo.png" alt="The Schema of the data will be inferred automatically when the data is read. See component Usage and Additional Details for information about how the schema is inferred." title="The Schema of the data will be inferred automatically when the data is read. See component Usage and Additional Details for information about how the schema is inferred.">< /img></li></ul></td><td id="description">Specifies how to obtain the schema that is to be used for interpreting the data.</td></tr><tr><td id="name">Schema Registry</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SchemaRegistry<br/><strong>Implementations: </strong><a href="../../../nifi-hwx-schema-registry-nar/1.9.0/org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry/index.html">HortonworksSchemaRegistry</a><br/><a href="../../../nifi-confluent-platform-nar/1.9.0/org.apache.nifi.confluent.schemaregistry.ConfluentSchemaRegistry/index.html">ConfluentSchemaRegistry</a><br/><a href="../../../nifi-registry-nar/1.9.0/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html">AvroSchemaRegistry</a></td><td id="description">Specifies the Controller Service to use for the Schema Registry</td></tr><tr><td id="name">Schema Name</td><td id="default-value">${schema.name}</td><td id="allowable-values"></td><td i d="description">Specifies the name of the schema to lookup in the Schema Registry property<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Version</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the version of the schema to lookup in the Schema Registry. If not specified then the latest version of the schema will be retrieved.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Branch</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the name of the branch to use when looking up the schema in the Schema Registry property. If the chosen Schema Registry does not support branching, this value will be ignored.<br/><strong>Supports Expression Language: true (will be evaluated using flo w file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Text</td><td id="default-value">${avro.schema}</td><td id="allowable-values"></td><td id="description">The text of an Avro-formatted Schema<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Inference Cache</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSchemaCacheService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.schema.inference.VolatileSchemaCache/index.html">VolatileSchemaCache</a></td><td id="description">Specifies a Schema Cache to use when inferring the schema. If not populated, the schema will be inferred each time. However, if a cache is specified, the cache will first be consulted and if the applicable schema can be found, it will be used instead of inferring the schema.</td></tr><tr><td id="name"><st rong>Expect Records as Array</strong></td><td id="default-value">false</td><td id="allowable-values"><ul><li>false <img src="../../../../../html/images/iconInfo.png" alt="Each FlowFile will consist of a single record without any sort of "wrapper"." title="Each FlowFile will consist of a single record without any sort of "wrapper"."></img></li><li>true <img src="../../../../../html/images/iconInfo.png" alt="Each FlowFile will consist of zero or more records. The outer-most XML element is expected to be a "wrapper" and will be ignored." title="Each FlowFile will consist of zero or more records. The outer-most XML element is expected to be a "wrapper" and will be ignored."></img></li><li>Use attribute 'xml.stream.is.array' <img src="../../../../../html/images/iconInfo.png" alt="Whether to treat a FlowFile as a single Record or an array of multiple Records is determined by the value of the 'xml.stream.is.array' attribute. If the value of the attri bute is 'true' (case-insensitive), then the XML Reader will treat the FlowFile as a series of Records with the outer element being ignored. If the value of the attribute is 'false' (case-insensitive), then the FlowFile is treated as a single Record and no wrapper element is assumed. If the attribute is missing or its value is anything other than 'true' or 'false', then an Exception will be thrown and no records will be parsed." title="Whether to treat a FlowFile as a single Record or an array of multiple Records is determined by the value of the 'xml.stream.is.array' attribute. If the value of the attribute is 'true' (case-insensitive), then the XML Reader will treat the FlowFile as a series of Records with the outer element being ignored. If the value of the attribute is 'false' (case-insensitive), then the FlowFile is treated as a single Record and no wrapper element is assumed. If the attribute is missing or its value is anything other than 'true' or 'false', then an Exception wi ll be thrown and no records will be parsed."></img></li></ul></td><td id="description">This property defines whether the reader expects a FlowFile to consist of a single Record or a series of Records with a "wrapper element". Because XML does not provide for a way to read a series of XML documents from a stream directly, it is common to combine many XML documents by concatenating them and then wrapping the entire XML blob with a "wrapper element". This property dictates whether the reader expects a FlowFile to consist of a single Record or a series of Records with a "wrapper element" that will be ignored.</td></tr><tr><td id="name">Attribute Prefix</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">If this property is set, the name of attributes will be prepended with a prefix when they are added to a record.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Field Name for Content</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">If tags with content (e. g. <field>content</field>) are defined as nested records in the schema, the name of the tag will be used as name for the record and the value of this property will be used as name for the field. If tags with content shall be parsed together with attributes (e. g. <field attribute="123">content</field>), they have to be defined as records. For additional information, see the section of processor usage.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Date Format</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Date fields. If not specified, Date fields will be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters, as in 01/01/2017).</td></tr><tr><td id="name">Time Format</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Time fields. If not specified, Time fields will be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, HH:mm:ss for a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 18:04:15).</td></tr><tr><td id="name">Timestamp Format</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Timestamp fields. If not specified, Timestamp fields wil l be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy HH:mm:ss for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters; and then followed by a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 01/01/2017 18:04:15).</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>System Resource Considerations:</h3>None specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLRecordSetWriter/additionalDetails.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLRecordSetWriter/additionalDetails.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLRecordSetWriter/additionalDetails.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLRecordSetWriter/additionalDetails.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1,211 @@ +<!DOCTYPE html> +<html lang="en"> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <head> + <meta charset="utf-8"/> + <title>XMLRecordSetWriter</title> + <link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"/> + </head> + + <body> + <p> + The XMLRecordSetWriter Controller Service writes record objects to XML. The Controller Service must be configured + with a schema that describes the structure of the record objects. Multiple records are wrapped by a root node. + The name of the root node can be configured via property. If no root node is configured, the writer expects + only one record for each FlowFile (that will not be wrapped). As Avro does not support defining attributes for + records, this writer currently does not support writing XML attributes. + </p> + + <h2> + Example: Simple records + </h2> + + <code> + <pre> + RecordSet ( + Record ( + Field "name1" = "value1", + Field "name2" = 42 + ), + Record ( + Field "name1" = "value2", + Field "name2" = 84 + ) + ) + </pre> + </code> + + <p> + This record can be described by the following schema: + </p> + + <code> + <pre> + { + "name": "test", + "namespace": "nifi", + "type": "record", + "fields": [ + { "name": "name1", "type": "string" }, + { "name": "name2", "type": "int" } + ] + } + </pre> + </code> + + <p> + Assuming that "root_name" has been configured as the name for the root node and "record_name" has been configured + as the name for the record nodes, the writer will write the following XML: + </p> + + <code> + <pre> + <root_name> + <record_name> + <name1>value1</name1> + <name2>42</name2> + </record_name> + <record_name> + <name1>value2</name1> + <name2>84</name2> + </record_name> + </root_name> + </pre> + </code> + + <p> + The writer furthermore can be configured how to treat null or missing values in records: + </p> + + <code> + <pre> + RecordSet ( + Record ( + Field "name1" = "value1", + Field "name2" = null + ), + Record ( + Field "name1" = "value2", + ) + ) + </pre> + </code> + + <p> + If the writer is configured always to suppress missing or null values, only the field of name "name1" will appear + in the XML. If the writer ist configured only to suppress missing values, the field of name "name2" will appear in + the XML as a node without content for the first record. If the writer is configured never to suppress anything, + the field of name "name2" will appear in the XML as a node without content for both records. + </p> + + <h2> + Example: Arrays + </h2> + + <p> + The writer furthermore can be configured how to write arrays: + </p> + + <code> + <pre> + RecordSet ( + Record ( + Field "name1" = "value1", + Field "array_field" = [ 1, 2, 3 ] + ) + ) + </pre> + </code> + + <p> + This record can be described by the following schema: + </p> + + <code> + <pre> + { + "name": "test", + "namespace": "nifi", + "type": "record", + "fields": [ + { "name": "array_field", "type": + { "type": "array", "items": int } + }, + { "name": "name1", "type": "string" } + ] + } + </pre> + </code> + + <p> + If the writer is configured not to wrap arrays, it will transform the record to the following XML: + </p> + + <code> + <pre> + <root_name> + <record_name> + <name1>value1</name1> + <array_field>1</array_field> + <array_field>2</array_field> + <array_field>3</array_field> + </record_name> + </root_name> + </pre> + </code> + + <p> + If the writer is configured to wrap arrays using the field name as wrapper and "elem" as tag name for element nodes, + it will transform the record to the following XML: + </p> + + <code> + <pre> + <root_name> + <record_name> + <name1>value1</field2> + <array_field> + <elem>1</elem> + <elem>2</elem> + <elem>3</elem> + </array_field> + </record_name> + </root_name> + </pre> + </code> + + <p> + If the writer is configured to wrap arrays using "wrap" as wrapper and the field name as tag name for element nodes, + it will transform the record to the following XML: + </p> + + <code> + <pre> + <root_name> + <record_name> + <name1>value1</field2> + <wrap> + <array_field>1</array_field> + <array_field>2</array_field> + <array_field>3</array_field> + </wrap> + </record_name> + </root_name> + </pre> + </code> + + </body> +</html> Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLRecordSetWriter/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLRecordSetWriter/index.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLRecordSetWriter/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.9.0/org.apache.nifi.xml.XMLRecordSetWriter/index.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>XMLRecordSetWriter</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">XMLRecordSetWriter</h1><h2>Description: </h2><p>Writes a RecordSet to XML. The records are wrapped by a root tag.</p><p><a href="additionalDetails.html">Additional Details...</a></p><h3>Tags: </h3><p>xml, resultset, writer, serialize, record, recordset, row</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a >.</p><table id="properties"><tr><th>Name</th><th>Default >Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td >id="name"><strong>Schema Write Strategy</strong></td><td >id="default-value">no-schema</td><td id="allowable-values"><ul><li>Set >'schema.name' Attribute <img src="../../../../../html/images/iconInfo.png" >alt="The FlowFile will be given an attribute named 'schema.name' and this >attribute will indicate the name of the schema in the Schema Registry. Note >that ifthe schema for a record is not obtained from a Schema Registry, then >no attribute will be added." title="The FlowFile will be given an attribute >named 'schema.name' and this attribute will indicate the name of the schema >in the Schema Registry. Note that ifthe schema for a record is not obtained >from a Schema Registry, then no attribute will be added."></img></li><li>Set >'avro.schema' Attribute <img src="../../../../../html/images/iconInfo.png" >alt="The FlowFile will be given an attribute named 'avro.schem a' and this attribute will contain the Avro Schema that describes the records in the FlowFile. The contents of the FlowFile need not be Avro, but the text of the schema will be used." title="The FlowFile will be given an attribute named 'avro.schema' and this attribute will contain the Avro Schema that describes the records in the FlowFile. The contents of the FlowFile need not be Avro, but the text of the schema will be used."></img></li><li>HWX Schema Reference Attributes <img src="../../../../../html/images/iconInfo.png" alt="The FlowFile will be given a set of 3 attributes to describe the schema: 'schema.identifier', 'schema.version', and 'schema.protocol.version'. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data." title="The FlowFile will be given a set of 3 attributes to describe the schema: 'schema.identifier', 'schema.version', and 'schema.protocol.version'. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data."></img></li><li>HWX Content-Encoded Schema Reference <img src="../../../../../html/images/iconInfo.png" alt="The content of the FlowFile will contain a reference to a schema in the Schema Registry service. The reference is encoded as a single byte indicating the 'protocol version', followed by 8 bytes indicating the schema identifier, and finally 4 bytes indicating the schema version, as per the Hortonworks Schema Registry serializers and deserializers, as found at https://github.com/hortonworks/registry. This will be prepended to each FlowFile. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data." title="The content of the FlowFile will contain a reference to a schema in the Schema Registry service. The reference is encoded as a single byte i ndicating the 'protocol version', followed by 8 bytes indicating the schema identifier, and finally 4 bytes indicating the schema version, as per the Hortonworks Schema Registry serializers and deserializers, as found at https://github.com/hortonworks/registry. This will be prepended to each FlowFile. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data."></img></li><li>Confluent Schema Registry Reference <img src="../../../../../html/images/iconInfo.png" alt="The content of the FlowFile will contain a reference to a schema in the Schema Registry service. The reference is encoded as a single 'Magic Byte' followed by 4 bytes representing the identifier of the schema, as outlined at http://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html. This will be prepended to each FlowFile. Note that if the schema for a record does not contain the necessary identifier and v ersion, an Exception will be thrown when attempting to write the data. This is based on the encoding used by version 3.2.x of the Confluent Schema Registry." title="The content of the FlowFile will contain a reference to a schema in the Schema Registry service. The reference is encoded as a single 'Magic Byte' followed by 4 bytes representing the identifier of the schema, as outlined at http://docs.confluent.io/current/schema-registry/docs/serializer-formatter.html. This will be prepended to each FlowFile. Note that if the schema for a record does not contain the necessary identifier and version, an Exception will be thrown when attempting to write the data. This is based on the encoding used by version 3.2.x of the Confluent Schema Registry."></img></li><li>Do Not Write Schema <img src="../../../../../html/images/iconInfo.png" alt="Do not add any schema-related information to the FlowFile." title="Do not add any schema-related information to the FlowFile."></img></li></ul></td><td id="description">Specifies how the schema for a Record should be added to the data.</td></tr><tr><td id="name">Schema Cache</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RecordSchemaCacheService<br/><strong>Implementation: </strong><a href="../org.apache.nifi.schema.inference.VolatileSchemaCache/index.html">VolatileSchemaCache</a></td><td id="description">Specifies a Schema Cache to add the Record Schema to so that Record Readers can quickly lookup the schema.</td></tr><tr><td id="name"><strong>Schema Access Strategy</strong></td><td id="default-value">inherit-record-schema</td><td id="allowable-values"><ul><li>Use 'Schema Name' Property <img src="../../../../../html/images/iconInfo.png" alt="The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service." title="The name of the Schema to use is specified by the 'Schem a Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service."></img></li><li>Inherit Record Schema <img src="../../../../../html/images/iconInfo.png" alt="The schema used to write records will be the same schema that was given to the Record when the Record was created." title="The schema used to write records will be the same schema that was given to the Record when the Record was created."></img></li><li>Use 'Schema Text' Property <img src="../../../../../html/images/iconInfo.png" alt="The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions." title="The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Te xt' property must be valid after substituting the expressions."></img></li></ul></td><td id="description">Specifies how to obtain the schema that is to be used for interpreting the data.</td></tr><tr><td id="name">Schema Registry</td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>SchemaRegistry<br/><strong>Implementations: </strong><a href="../../../nifi-hwx-schema-registry-nar/1.9.0/org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry/index.html">HortonworksSchemaRegistry</a><br/><a href="../../../nifi-confluent-platform-nar/1.9.0/org.apache.nifi.confluent.schemaregistry.ConfluentSchemaRegistry/index.html">ConfluentSchemaRegistry</a><br/><a href="../../../nifi-registry-nar/1.9.0/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html">AvroSchemaRegistry</a></td><td id="description">Specifies the Controller Service to use for the Schema Registry</td></tr><tr><td id="name">Schema Name</td><td id="defau lt-value">${schema.name}</td><td id="allowable-values"></td><td id="description">Specifies the name of the schema to lookup in the Schema Registry property<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Version</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the version of the schema to lookup in the Schema Registry. If not specified then the latest version of the schema will be retrieved.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Branch</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the name of the branch to use when looking up the schema in the Schema Registry property. If the chosen Schema Registry does not support branching, this value will be ignored.<br/><stron g>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Schema Text</td><td id="default-value">${avro.schema}</td><td id="allowable-values"></td><td id="description">The text of an Avro-formatted Schema<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name">Date Format</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Date fields. If not specified, Date fields will be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters, as in 01/01/2017).</td></tr><tr><td id="name">Time Format</td><td id="default- value"></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Time fields. If not specified, Time fields will be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, HH:mm:ss for a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 18:04:15).</td></tr><tr><td id="name">Timestamp Format</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the format to use when reading/writing Timestamp fields. If not specified, Timestamp fields will be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy HH:mm:ss for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/ ' characters; and then followed by a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 01/01/2017 18:04:15).</td></tr><tr><td id="name"><strong>Suppress Null Values</strong></td><td id="default-value">never-suppress</td><td id="allowable-values"><ul><li>Never Suppress <img src="../../../../../html/images/iconInfo.png" alt="Fields that are missing (present in the schema but not in the record), or that have a value of null, will be written out as a null value" title="Fields that are missing (present in the schema but not in the record), or that have a value of null, will be written out as a null value"></img></li><li>Always Suppress <img src="../../../../../html/images/iconInfo.png" alt="Fields that are missing (present in the schema but not in the record), or that have a value of null, will not be written out" title="Fields that are missing (present in the schema but not in the record), or that hav e a value of null, will not be written out"></img></li><li>Suppress Missing Values <img src="../../../../../html/images/iconInfo.png" alt="When a field has a value of null, it will be written out. However, if a field is defined in the schema and not present in the record, the field will not be written out." title="When a field has a value of null, it will be written out. However, if a field is defined in the schema and not present in the record, the field will not be written out."></img></li></ul></td><td id="description">Specifies how the writer should handle a null field</td></tr><tr><td id="name"><strong>Pretty Print XML</strong></td><td id="default-value">false</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Specifies whether or not the XML should be pretty printed</td></tr><tr><td id="name">Name of Root Tag</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the name of the XML root tag wrappi ng the record set. This property has to be defined if the writer is supposed to write multiple records in a single FlowFile.</td></tr><tr><td id="name">Name of Record Tag</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Specifies the name of the XML record tag wrapping the record fields. If this is not set, the writer will use the record name in the schema.</td></tr><tr><td id="name"><strong>Wrap Elements of Arrays</strong></td><td id="default-value">no-wrapping</td><td id="allowable-values"><ul><li>Use Property as Wrapper <img src="../../../../../html/images/iconInfo.png" alt="The value of the property "Array Tag Name" will be used as the tag name to wrap elements of an array. The field name of the array field will be used for the tag name of the elements." title="The value of the property "Array Tag Name" will be used as the tag name to wrap elements of an array. The field name of the array field will be used for the tag name of the elements."></img></li><li>Use Property for Elements <img src="../../../../../html/images/iconInfo.png" alt="The value of the property "Array Tag Name" will be used for the tag name of the elements of an array. The field name of the array field will be used as the tag name to wrap elements." title="The value of the property "Array Tag Name" will be used for the tag name of the elements of an array. The field name of the array field will be used as the tag name to wrap elements."></img></li><li>No Wrapping <img src="../../../../../html/images/iconInfo.png" alt="The elements of an array will not be wrapped" title="The elements of an array will not be wrapped"></img></li></ul></td><td id="description">Specifies how the writer wraps elements of fields of type array</td></tr><tr><td id="name">Array Tag Name</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Name of the tag used by property "Wrap Elements of Arrays" to write array s</td></tr><tr><td id="name"><strong>Character Set</strong></td><td id="default-value">UTF-8</td><td id="allowable-values"></td><td id="description">The Character set to use when writing the data to the FlowFile</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>System Resource Considerations:</h3>None specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-redis-nar/1.9.0/org.apache.nifi.redis.service.RedisConnectionPoolService/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-redis-nar/1.9.0/org.apache.nifi.redis.service.RedisConnectionPoolService/index.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-redis-nar/1.9.0/org.apache.nifi.redis.service.RedisConnectionPoolService/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-redis-nar/1.9.0/org.apache.nifi.redis.service.RedisConnectionPoolService/index.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>RedisConnectionPoolService</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">RedisConnectionPoolService</h1><h2>Description: </h2><p>A service that provides connections to Redis.</p><h3>Tags: </h3><p>redis, cache</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Redis Mode</strong></td><td id="default-value">Standalone</td><td id="allowable-values"><ul><li>Standalone <img src="../../../../../html/images/iconInfo.png" alt="A single standalone Redis instance." title="A single standalone Redis instance."></img></li><li>Sentinel <img src="../../../../../html/images/iconInfo.png" alt="Redis Sentinel which provides high-availability. Described further at https://redis.io/topics/sentinel"; title="Redis Sentinel which provides high-availability. Described further at https://redis.io/topics/sentinel";></img></li><li>Cluster <img src="../../../../../html/images/iconInfo.png" alt="Clustered Redis which provides sharding and replication. Described further at https:/ /redis.io/topics/cluster-spec" title="Clustered Redis which provides sharding and replication. Described further at https://redis.io/topics/cluster-spec";></img></li></ul></td><td id="description">The type of Redis being communicated with - standalone, sentinel, or clustered.</td></tr><tr><td id="name"><strong>Connection String</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The connection string for Redis. In a standalone instance this value will be of the form hostname:port. In a sentinel instance this value will be the comma-separated list of sentinels, such as host1:port1,host2:port2,host3:port3. In a clustered instance this value will be the comma-separated list of cluster masters, such as host1:port,host2:port,host3:port.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Database Index</strong></td><td id="default-value">0</td><td id="allowable- values"></td><td id="description">The database index to be used by connections created from this connection pool. See the databases property in redis.conf, by default databases 0-15 will be available.<br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Communication Timeout</strong></td><td id="default-value">10 seconds</td><td id="allowable-values"></td><td id="description">The timeout to use when attempting to communicate with Redis.</td></tr><tr><td id="name"><strong>Cluster Max Redirects</strong></td><td id="default-value">5</td><td id="allowable-values"></td><td id="description">The maximum number of redirects that can be performed when clustered.</td></tr><tr><td id="name">Sentinel Master</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The name of the sentinel master, require when Mode is set to Sentinel<br/><strong>Supports Expression Language: true (wil l be evaluated using variable registry only)</strong></td></tr><tr><td id="name">Password</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">The password used to authenticate to the Redis server. See the requirepass property in redis.conf.<br/><strong>Sensitive Property: true</strong><br/><strong>Supports Expression Language: true (will be evaluated using variable registry only)</strong></td></tr><tr><td id="name"><strong>Pool - Max Total</strong></td><td id="default-value">8</td><td id="allowable-values"></td><td id="description">The maximum number of connections that can be allocated by the pool (checked out to clients, or idle awaiting checkout). A negative value indicates that there is no limit.</td></tr><tr><td id="name"><strong>Pool - Max Idle</strong></td><td id="default-value">8</td><td id="allowable-values"></td><td id="description">The maximum number of idle connections that can be held in the pool, or a negative value if there is no limit. </td></tr><tr><td id="name"><strong>Pool - Min Idle</strong></td><td id="default-value">0</td><td id="allowable-values"></td><td id="description">The target for the minimum number of idle connections to maintain in the pool. If the configured value of Min Idle is greater than the configured value for Max Idle, then the value of Max Idle will be used instead.</td></tr><tr><td id="name"><strong>Pool - Block When Exhausted</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td id="description">Whether or not clients should block and wait when trying to obtain a connection from the pool when the pool has no available connections. Setting this to false means an error will occur immediately when a client requests a connection and none are available.</td></tr><tr><td id="name"><strong>Pool - Max Wait Time</strong></td><td id="default-value">10 seconds</td><td id="allowable-values"></td><td id="description">The amount of time to w ait for an available connection when Block When Exhausted is set to true.</td></tr><tr><td id="name"><strong>Pool - Min Evictable Idle Time</strong></td><td id="default-value">60 seconds</td><td id="allowable-values"></td><td id="description">The minimum amount of time an object may sit idle in the pool before it is eligible for eviction.</td></tr><tr><td id="name"><strong>Pool - Time Between Eviction Runs</strong></td><td id="default-value">30 seconds</td><td id="allowable-values"></td><td id="description">The amount of time between attempting to evict idle connections from the pool.</td></tr><tr><td id="name"><strong>Pool - Num Tests Per Eviction Run</strong></td><td id="default-value">-1</td><td id="allowable-values"></td><td id="description">The number of connections to tests per eviction attempt. A negative value indicates to test all connections.</td></tr><tr><td id="name"><strong>Pool - Test On Create</strong></td><td id="default-value">false</td><td id="allowable-values"><ul ><li>true</li><li>false</li></ul></td><td id="description">Whether or not >connections should be tested upon creation.</td></tr><tr><td >id="name"><strong>Pool - Test On Borrow</strong></td><td >id="default-value">false</td><td >id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td >id="description">Whether or not connections should be tested upon borrowing >from the pool.</td></tr><tr><td id="name"><strong>Pool - Test On >Return</strong></td><td id="default-value">false</td><td >id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td >id="description">Whether or not connections should be tested upon returning >to the pool.</td></tr><tr><td id="name"><strong>Pool - Test While >Idle</strong></td><td id="default-value">true</td><td >id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td >id="description">Whether or not connections should be tested while >idle.</td></tr></table><h3>State management: </h3>This component does not >store state.<h3>Restricted: </h3>Thi s component is not restricted.<h3>System Resource Considerations:</h3>None specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-redis-nar/1.9.0/org.apache.nifi.redis.service.RedisDistributedMapCacheClientService/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-redis-nar/1.9.0/org.apache.nifi.redis.service.RedisDistributedMapCacheClientService/index.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-redis-nar/1.9.0/org.apache.nifi.redis.service.RedisDistributedMapCacheClientService/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-redis-nar/1.9.0/org.apache.nifi.redis.service.RedisDistributedMapCacheClientService/index.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>RedisDistributedMapCacheClientService</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">RedisDistributedMapCacheClientService</h1><h2>Description: </h2><p>An implementation of DistributedMapCacheClient that uses Redis as the backing cache. This service relies on the WATCH, MULTI, and EXEC commands in Redis, which are not fully supported when Redis is clustered. As a result, this service can only be used with a Redis Connection Pool that is configured for standalone or sentinel mode. Sentinel mode can be used to provide high-availability configurations.</p><h3>Tags: </h3><p>redis, distributed, cache, map</p><h3>Properties: </h3><p>In the list below, the names o f required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>Redis Connection Pool</strong></td><td id="default-value"></td><td id="allowable-values"><strong>Controller Service API: </strong><br/>RedisConnectionPool<br/><strong>Implementation: </strong><a href="../org.apache.nifi.redis.service.RedisConnectionPoolService/index.html">RedisConnectionPoolService</a></td><td id="description">No Description Provided.</td></tr><tr><td id="name"><strong>TTL</strong></td><td id="default-value">0 secs</td><td id="allowable-values"></td><td id="description">Indicates how long the data should exist in Redis. Setting '0 secs' would mean the data would exist forever</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>System Resource Considerations:</h3>None specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-registry-nar/1.9.0/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-registry-nar/1.9.0/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-registry-nar/1.9.0/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-registry-nar/1.9.0/org.apache.nifi.schemaregistry.services.AvroSchemaRegistry/index.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>AvroSchemaRegistry</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">AvroSchemaRegistry</h1><h2>Description: </h2><p>Provides a service for registering and accessing schemas. You can register a schema as a dynamic property where 'name' represents the schema name and 'value' represents the textual representation of the actual schema following the syntax and semantics of Avro's Schema format.</p><h3>Tags: </h3><p>schema, registry, avro, json, csv</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values.</p ><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable >Values</th><th>Description</th></tr><tr><td id="name"><strong>Validate Field >Names</strong></td><td id="default-value">true</td><td >id="allowable-values"><ul><li>true</li><li>false</li></ul></td><td >id="description">Whether or not to validate the field names in the Avro >schema based on Avro naming rules. If set to true, all field names must be >valid Avro names, which must begin with [A-Za-z_], and subsequently contain >only [A-Za-z0-9_]. If set to false, no validation will be performed on the >field names.</td></tr></table><h3>State management: </h3>This component does >not store state.<h3>Restricted: </h3>This component is not >restricted.<h3>System Resource Considerations:</h3>None >specified.</body></html> \ No newline at end of file Added: nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-rethinkdb-nar/1.9.0/org.apache.nifi.processors.rethinkdb.DeleteRethinkDB/index.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-rethinkdb-nar/1.9.0/org.apache.nifi.processors.rethinkdb.DeleteRethinkDB/index.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-rethinkdb-nar/1.9.0/org.apache.nifi.processors.rethinkdb.DeleteRethinkDB/index.html (added) +++ nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi/nifi-rethinkdb-nar/1.9.0/org.apache.nifi.processors.rethinkdb.DeleteRethinkDB/index.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1 @@ +<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"></meta><title>DeleteRethinkDB</title><link rel="stylesheet" href="../../../../../css/component-usage.css" type="text/css"></link></head><script type="text/javascript">window.onload = function(){if(self==top) { document.getElementById('nameHeader').style.display = "inherit"; } }</script><body><h1 id="nameHeader" style="display: none;">DeleteRethinkDB</h1><h2>Description: </h2><p>Processor to remove a JSON document from RethinkDB (https://www.rethinkdb.com/) using the document id.</p><h3>Tags: </h3><p>rethinkdb, delete, remove</p><h3>Properties: </h3><p>In the list below, the names of required properties appear in <strong>bold</strong>. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the <a href="../../../../../html/expression-language-guide.html">NiFi Expression Language</a>, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the <strong>nifi.properties</strong> file has an entry for the property <strong>nifi.sensitive.props.key</strong>.</p><table id="properties"><tr><th>Name</th><th>Default Value</th><th>Allowable Values</th><th>Description</th></tr><tr><td id="name"><strong>DB Name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">RethinkDB database to connect to</td></tr><tr><td id="name"><strong>Hostname</strong></td><td id="default-value">localhost</td><td id="allowable-values"></td><td id="description">RethinkDB hostname</td></tr><tr><td id="name"><strong>DB Port</strong></td><td id="default-value">28015</td><td id="allowable-values"></td><td id="description">RethinkDB database port to connect to</td></tr><tr><td id="name">Username</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Username for accessing RethinkDB</td></tr><tr><td id="nam e">Password</td><td id="default-value"></td><td id="allowable-values"></td><td id="description">Password for user<br/><strong>Sensitive Property: true</strong></td></tr><tr><td id="name"><strong>Table name</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">RethinkDB table to connect to</td></tr><tr><td id="name"><strong>Character Set</strong></td><td id="default-value">UTF-8</td><td id="allowable-values"></td><td id="description">Specifies the character set of the document data.<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Document Identifier</strong></td><td id="default-value"></td><td id="allowable-values"></td><td id="description">A FlowFile attribute, or attribute expression used for determining RethinkDB key for the Flow File content<br/><strong>Supports Expression Language: true (will be evaluated using flow file attribute s and variable registry)</strong></td></tr><tr><td id="name"><strong>Return deleted value</strong></td><td id="default-value">true</td><td id="allowable-values"><ul><li>True <img src="../../../../../html/images/iconInfo.png" alt="Return changed document" title="Return changed document"></img></li><li>False <img src="../../../../../html/images/iconInfo.png" alt="Do not return changed document" title="Do not return changed document"></img></li></ul></td><td id="description">Return old value which were deleted<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr><tr><td id="name"><strong>Durablity of documents</strong></td><td id="default-value">hard</td><td id="allowable-values"><ul><li>Hard <img src="../../../../../html/images/iconInfo.png" alt="Save change to disk before ack" title="Save change to disk before ack"></img></li><li>Soft <img src="../../../../../html/images/iconInfo.png" alt="Don't save cha nges to disk before ack" title="Don't save changes to disk before ack"></img></li></ul></td><td id="description">Durability of documents being inserted<br/><strong>Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)</strong></td></tr></table><h3>Relationships: </h3><table id="relationships"><tr><th>Name</th><th>Description</th></tr><tr><td>success</td><td>Sucessful FlowFiles are routed to this relationship</td></tr><tr><td>not_found</td><td>Document not found are routed to this relationship</td></tr><tr><td>failure</td><td>Failed FlowFiles are routed to this relationship</td></tr></table><h3>Reads Attributes: </h3>None specified.<h3>Writes Attributes: </h3><table id="writes-attributes"><tr><th>Name</th><th>Description</th></tr><tr><td>rethinkdb.error.message</td><td>RethinkDB error message</td></tr><tr><td>rethinkdb.delete.errors</td><td>Error count while delete documents</td></tr><tr><td>rethinkdb.delete.deleted</td><td>Number of documents deleted</td></tr><tr><td>rethinkdb.delete.inserted</td><td>Number of documents inserted</td></tr><tr><td>rethinkdb.delete.replaced</td><td>Number of documents replaced</td></tr><tr><td>rethinkdb.delete.skipped</td><td>Number of documents skipped</td></tr><tr><td>rethinkdb.delete.unchanged</td><td>Number of documents unchanged since they already existed</td></tr></table><h3>State management: </h3>This component does not store state.<h3>Restricted: </h3>This component is not restricted.<h3>Input requirement: </h3>This component requires an incoming relationship.<h3>System Resource Considerations:</h3>None specified.<h3>See Also:</h3><p><a href="../org.apache.nifi.processors.rethinkdb.PutRethinkDB/index.html">PutRethinkDB</a>, <a href="../org.apache.nifi.processors.rethinkdb.GetRethinkDB/index.html">GetRethinkDB</a></p></body></html> \ No newline at end of file
