Modified: nifi/site/trunk/docs/nifi-docs/html/administration-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/administration-guide.html?rev=1834461&r1=1834460&r2=1834461&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/administration-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/administration-guide.html Wed Jun 27 00:37:46 2018 @@ -1,4 +1,20 @@ -<!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -528,6 +544,7 @@ body.book #toc,body.book #preamble,body. <li><a href="administration-guide.html#encrypted-write-ahead-provenance-repository-properties">Encrypted Write Ahead Provenance Repository Properties</a></li> <li><a href="administration-guide.html#component-status-repository">Component Status Repository</a></li> <li><a href="administration-guide.html#site_to_site_properties">Site to Site Properties</a></li> +<li><a href="administration-guide.html#site_to_site_reverse_proxy_properties">Site to Site Routing Properties for Reverse Proxies</a></li> <li><a href="administration-guide.html#web-properties">Web Properties</a></li> <li><a href="administration-guide.html#security-properties">Security Properties</a></li> <li><a href="administration-guide.html#identity-mapping-properties">Identity Mapping Properties</a></li> @@ -897,7 +914,7 @@ properties can be specified.</p> <i class="fa icon-note" title="Note"></i> </td> <td class="content"> -It is important when enabling HTTPS that the <code>nifi.web.http.port</code> property be unset. +It is important when enabling HTTPS that the <code>nifi.web.http.port</code> property be unset. NiFi only supports running on HTTP <strong>or</strong> HTTPS, not both simultaneously. </td> </tr> </table> @@ -2361,6 +2378,15 @@ memberUid: user2 <td class="tableblock halign-center valign-top"></td> <td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> </tr> +<tr> +<td class="tableblock halign-right valign-top"><p class="tableblock"><strong>view provenance</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"><p class="tableblock"><strong>*</strong></p></td> +<td class="tableblock halign-center valign-top"></td> +<td class="tableblock halign-center valign-top"></td> +</tr> </tbody> </table> <div class="paragraph"> @@ -2539,15 +2565,17 @@ Instructions requiring interaction with </div> <table class="tableblock frame-all grid-all spread"> <colgroup> -<col style="width: 33%;"> -<col style="width: 33%;"> -<col style="width: 33%;"> +<col style="width: 25%;"> +<col style="width: 25%;"> +<col style="width: 25%;"> +<col style="width: 25%;"> </colgroup> <thead> <tr> <th class="tableblock halign-left valign-top">Policy</th> <th class="tableblock halign-left valign-top">Privilege</th> <th class="tableblock halign-left valign-top">Global Menu Selection</th> +<th class="tableblock halign-left valign-top">Resource Descriptor</th> </tr> </thead> <tbody> @@ -2555,16 +2583,19 @@ Instructions requiring interaction with <td class="tableblock halign-left valign-top"><p class="tableblock">view the UI</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allow users to view the UI</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/flow</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">access the controller</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the controller including Reporting Tasks, Controller Services, and Nodes in the Cluster</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Controller Settings</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/controller</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">query provenance</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to submit a Provenance Search and request Event Lineage</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Data Provenance</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/provenance</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">access restricted components</p></td> @@ -2573,36 +2604,43 @@ components may indicate which specific p restrictions or be granted regardless of restrictions. If permission is granted regardless of restrictions, the user can create/modify all restricted components.</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/restricted-components</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">access all policies</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the policies for all components</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Policies</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/policies</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">access users/user groups</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify the users and user groups</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Users</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/tenants</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">retrieve site-to-site details</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows other NiFi instances to retrieve Site-To-Site details</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/site-to-site</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">view system diagnostics</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view System Diagnostics</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Summary</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/system</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">proxy user requests</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows proxy machines to send requests on the behalf of others</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/proxy</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">access counters</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view/modify Counters</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Counters</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/counters</code></p></td> </tr> </tbody> </table> @@ -2614,47 +2652,62 @@ the user can create/modify all restricte </div> <table class="tableblock frame-all grid-all spread"> <colgroup> -<col style="width: 50%;"> -<col style="width: 50%;"> +<col style="width: 33%;"> +<col style="width: 33%;"> +<col style="width: 33%;"> </colgroup> <thead> <tr> <th class="tableblock halign-left valign-top">Policy</th> <th class="tableblock halign-left valign-top">Privilege</th> +<th class="tableblock halign-left valign-top">Resource Descriptor & Action</th> </tr> </thead> <tbody> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">view the component</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view component configuration details</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/<component-type>/<component-UUID>" action="R"</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">modify the component</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to modify component configuration details</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/<component-type>/<component-UUID>" action="W"</code></p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">view provenance</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view provenance events generated by this component</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/provenance-data/<component-type>/<component-UUID>" action="R"</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">view the data</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Allows user to view metadata and content for this component through provenance data and flowfile queues in outbound connections</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view metadata and content for this component in flowfile queues in outbound connections and through provenance events</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/data/<component-type>/<component-UUID>" action="R"</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">modify the data</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Allows user to empty flowfile queues in outbound connections and submit replays</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to empty flowfile queues in outbound connections and submit replays through provenance events</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/data/<component-type>/<component-UUID>" action="W"</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">view the policies</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to view the list of users who can view/modify a component</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/policies/<component-type>/<component-UUID>" action="R"</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">modify the policies</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to modify the list of users who can view/modify a component</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/policies/<component-type>/<component-UUID>" action="W"</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">receive data via site-to-site</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows a port to receive data from NiFi instances</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/data-transfer/input-ports/<port-UUID>" action="W"</code></p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">send data via site-to-site</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">Allows a port to send data from NiFi instances</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><code>resource="/data-transfer/output-ports/<port-UUID>" action="W"</code></p></td> </tr> </tbody> </table> @@ -4187,8 +4240,8 @@ they must be set the same on every insta <div class="ulist"> <ul> <li> -<p>Under the <em>Web Properties</em> section, set either the http or https port that you want the Node to run on. -Also, consider whether you need to set the http or https host property.</p> +<p>Under the <em>Web Properties</em> section, set either the HTTP or HTTPS port that you want the Node to run on. +Also, consider whether you need to set the HTTP or HTTPS host property. All nodes in the cluster should use the same protocol setting.</p> </li> <li> <p>Under the <em>State Management section</em>, set the <code>nifi.state.management.provider.cluster</code> property @@ -4473,7 +4526,7 @@ NiFi ZooKeeper client and embedded ZooKe </div> <div class="paragraph"> <p>If Kerberos is not already setup in your environment, you can find information on installing and setting up a Kerberos Server at -<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Configuring_a_Kerberos_5_Server.html"; target="_blank">https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Configuring_a_Kerberos_5_Server.html</a>. This guide assumes that Kerberos already has been installed in the environment in which NiFi is running.</p> +<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Configuring_a_Kerberos_5_Server.html";>Red Hat Customer Portal: Configuring a Kerberos 5 Server</a>. This guide assumes that Kerberos already has been installed in the environment in which NiFi is running.</p> </div> <div class="paragraph"> <p>Note, the following procedures for kerberizing an Embedded ZooKeeper server in your NiFi Node and kerberizing a ZooKeeper NiFi client will require that @@ -4719,7 +4772,7 @@ in the following locations:</p> <div class="paragraph"> <p>One of the most important notes in the above Troubleshooting guide is the mechanism for turning on Debug output for Kerberos. This is done by setting the <code>sun.security.krb5.debug</code> environment variable. -In NiFi, this is accomplished by adding the following line to the _$NIFI_HOME/conf/bootstrap.conf` file:</p> +In NiFi, this is accomplished by adding the following line to the <em>$NIFI_HOME/conf/bootstrap.conf</em> file:</p> </div> <div class="listingblock"> <div class="content"> @@ -5428,7 +5481,7 @@ documentation of the proxy for guidance <li> <p>By default, if NiFi is running securely it will only accept HTTP requests with a Host header matching the host[:port] that it is bound to. If NiFi is to accept requests directed to a different host[:port] the expected values need to be configured. This may be required when running behind a proxy or in a containerized environment. This is configured in a comma -separated list in <em>nifi.properties</em> using the <code>nifi.web.proxy.host</code> property (e.g. localhost:18443, proxyhost:443). IPv6 addressed are accepted. Please refer to +separated list in <em>nifi.properties</em> using the <code>nifi.web.proxy.host</code> property (e.g. localhost:18443, proxyhost:443). IPv6 addresses are accepted. Please refer to RFC 5952 Sections <a href="https://tools.ietf.org/html/rfc5952#section-4";>4</a> and <a href="https://tools.ietf.org/html/rfc5952#section-6";>6</a> for additional details.</p> </li> <li> @@ -5439,6 +5492,16 @@ present in the whitelist, the "An unexpe </ul> </div> </li> +<li> +<p>Additional configurations at both proxy server and NiFi cluster are required to make NiFi Site-to-Site work behind reverse proxies. See <a href="administration-guide.html#site_to_site_reverse_proxy_properties">Site to Site Routing Properties for Reverse Proxies</a> for details.</p> +<div class="ulist"> +<ul> +<li> +<p>In order to transfer data via Site-to-Site protocol through reverse proxies, both proxy and Site-to-Site client NiFi users need to have following policies, 'retrieve site-to-site details', 'receive data via site-to-site' for input ports, and 'send data via site-to-site' for output ports.</p> +</li> +</ul> +</div> +</li> </ul> </div> </div> @@ -5579,16 +5642,16 @@ for example "10 secs" or "10 MB", not si </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.flow.configuration.archive.dir*</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the archive directory where backup copies of the flow.xml are saved. The default value is <code>./conf/archive</code>. NiFi removes old archive files to limit disk usage based on archived file lifespan, total size, and number of files, as specified with <code>nifi.flow.configuration.archive.max.time</code>, <code>max.storage</code> and <code>max.count</code> properties respectively. If none of these limitation for archiving is specified, NiFi uses default condition, that is "30 days" for max.time and "500 MB" for max.storage.<br> +<td class="tableblock halign-left valign-top"><p class="tableblock">The location of the archive directory where backup copies of the flow.xml are saved. The default value is <code>./conf/archive</code>. NiFi removes old archive files to limit disk usage based on archived file lifespan, total size, and number of files, as specified with <code>nifi.flow.configuration.archive.max.time</code>, <code>max.storage</code> and <code>max.count</code> properties respectively. If none of these limitation for archiving is specified, NiFi uses default conditions, that is <code>30 days</code> for max.time and <code>500 MB</code> for max.storage.<br> This cleanup mechanism takes into account only automatically created archived flow.xml files. If there are other files or directories in this archive directory, NiFi will ignore them. Automatically created archives have filename with ISO 8601 format timestamp prefix followed by '<em><original-filename>'. That is <year><month><day>T<hour><minute><second>+<timezone offset></em><original filename>. For example, <code>20160706T160719+0900_flow.xml.gz</code>. NiFi checks filenames when it cleans archive directory. If you would like to keep a particular archive in this directory without worrying about NiFi deleting it, you can do so by copying it with a different filename pattern.</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.flow.configuration.archive.max.time*</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The lifespan of archived flow.xml files. NiFi will delete expired archive files when it updates flow.xml if this property is specified. Expiration is determined based on current system time and the last modified timestamp of an archived flow.xml. If no archive limitation is specified in nifi.properties, NiFi removes archives older than "30 days".</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The lifespan of archived flow.xml files. NiFi will delete expired archive files when it updates flow.xml if this property is specified. Expiration is determined based on current system time and the last modified timestamp of an archived flow.xml. If no archive limitation is specified in nifi.properties, NiFi removes archives older than <code>30 days</code>.</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.flow.configuration.archive.max.storage*</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The total data size allowed for the archived flow.xml files. NiFi will delete the oldest archive files until the total archived file size becomes less than this configuration value, if this property is specified. If no archive limitation is specified in nifi.properties, NiFi uses "500 MB" for this.</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The total data size allowed for the archived flow.xml files. NiFi will delete the oldest archive files until the total archived file size becomes less than this configuration value, if this property is specified. If no archive limitation is specified in nifi.properties, NiFi uses <code>500 MB</code> for this.</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.flow.configuration.archive.max.count*</p></td> @@ -5615,6 +5678,14 @@ This cleanup mechanism takes into accoun <td class="tableblock halign-left valign-top"><p class="tableblock">When a component has no work to do (i.e., is "bored"), this is the amount of time it will wait before checking to see if it has new data to work on. This way, it does not use up CPU resources by checking for new work too often. When setting this property, be aware that it could add extra latency for components that do not constantly have work to do, as once they go into this "bored" state, they will wait this amount of time before checking for more work. The default value is <code>10 ms</code>.</p></td> </tr> <tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.queue.backpressure.count</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">When drawing a new connection between two components, this is the default value for that connection’s back pressure object threshold. The default is <code>10000</code> and the value must be an integer.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.queue.backpressure.size</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">When drawing a new connection between two components, this is the default value for that connection’s back pressure data size threshold. The default is <code>1 GB</code> and the value must be a data size including the unit of measure.</p></td> +</tr> +<tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.authorizer.configuration.file*</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">This is the location of the file that specifies how authorizers are defined. The default value is <code>./conf/authorizers.xml</code>.</p></td> </tr> @@ -5868,7 +5939,7 @@ FlowFile Repository, if also on that dis </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.content.claim.max.appendable.size</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum size for a content claim. The default value is <code>10 MB</code>.</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum size for a content claim. The default value is <code>1 MB</code>.</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.content.claim.max.flow.files</p></td> @@ -6333,6 +6404,384 @@ has many instances of Remote Process Gro </table> </div> <div class="sect2"> +<h3 id="site_to_site_reverse_proxy_properties"><a class="anchor" href="administration-guide.html#site_to_site_reverse_proxy_properties"></a>Site to Site Routing Properties for Reverse Proxies</h3> +<div class="paragraph"> +<p>Site-to-Site requires peer-to-peer communication between a client and a remote NiFi node. E.g. if a remote NiFi cluster has 3 nodes, nifi0, nifi1 and nifi2, then a client requests have to be reachable to each of those remote node.</p> +</div> +<div class="paragraph"> +<p>If a NiFi cluster is planned to receive/transfer data from/to Site-to-Site clients over the internet or a company firewall, a reverse proxy server can be deployed in front of the NiFi cluster nodes as a gateway to route client requests to upstream NiFi nodes, to reduce number of servers and ports those have to be exposed.</p> +</div> +<div class="paragraph"> +<p>In such environment, the same NiFi cluster would also be expected to be accessed by Site-to-Site clients within the same network. Sending FlowFiles to itself for load distribution among NiFi cluster nodes can be a typical example. In this case, client requests should be routed directly to a node without going through the reverse proxy.</p> +</div> +<div class="paragraph"> +<p>In order to support such deployments, remote NiFi clusters need to expose its Site-to-Site endpoints dynamically based on client request contexts. Following properties configure how peers should be exposed to clients. A routing definition consists of 4 properties, 'when', 'hostname', 'port', and 'secure', grouped by 'protocol' and 'name'. Multiple routing definitions can be configured. 'protocol' represents Site-to-Site transport protocol, i.e. raw or http.</p> +</div> +<table class="tableblock frame-all grid-all spread"> +<colgroup> +<col style="width: 50%;"> +<col style="width: 50%;"> +</colgroup> +<tbody> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Property</strong></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.remote.route.{protocol}.{name}.when</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Boolean value, 'true' or 'false'. Controls whether the routing definition for this name should be used.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.remote.route.{protocol}.{name}.hostname</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Specify hostname that will be introduced to Site-to-Site clients for further communications.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.remote.route.{protocol}.{name}.port</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Specify port number that will be introduced to Site-to-Site clients for further communications.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.remote.route.{protocol}.{name}.secure</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Boolean value, 'true' or 'false'. Specify whether the remote peer should be accessed via secure protocol. Defaults to 'false'.</p></td> +</tr> +</tbody> +</table> +<div class="paragraph"> +<p>All of above routing properties can use NiFi Expression Language to compute target peer description from request context. Available variables are:</p> +</div> +<table class="tableblock frame-all grid-all spread"> +<colgroup> +<col style="width: 50%;"> +<col style="width: 50%;"> +</colgroup> +<tbody> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Variable name</strong></p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">s2s.{source|target}.hostname</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Hostname of the source where the request came from, and the original target.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">s2s.{source|target}.port</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Same as above, for ports. Source port may not be useful as it is just a client side TCP port.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">s2s.{source|target}.secure</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Same as above, for secure or not.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">s2s.protocol</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The name of Site-to-Site protocol being used, RAW or HTTP.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">s2s.request</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The name of current request type, SiteToSiteDetail or Peers. See Site-to-Site protocol sequence below for detail.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">HTTP request headers</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">HTTP request header values can be referred by its name.</p></td> +</tr> +</tbody> +</table> +<div class="sect3"> +<h4 id="site-to-site-protocol-sequence"><a class="anchor" href="administration-guide.html#site-to-site-protocol-sequence"></a>Site to Site protocol sequence</h4> +<div class="paragraph"> +<p>Configuring these properties correctly would require some understandings on Site-to-Site protocol sequence.</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>A client initiates Site-to-Site protocol by sending a HTTP(S) request to the specified remote URL to get remote cluster Site-to-Site information. Specifically, to '/nifi-api/site-to-site'. This request is called 'SiteToSiteDetail'.</p> +</li> +<li> +<p>A remote NiFi node responds with its input and output ports, and TCP port numbers for RAW and TCP transport protocols.</p> +</li> +<li> +<p>The client sends another request to get remote peers using the TCP port number returned at #2. From this request, raw socket communication is used for RAW transport protocol, while HTTP keeps using HTTP(S). This request is called 'Peers'.</p> +</li> +<li> +<p>A remote NiFi node responds with list of available remote peers containing hostname, port, secure and workload such as the number of queued FlowFiles. From this point, further communication is done between the client and the remote NiFi node.</p> +</li> +<li> +<p>The client decides which peer to transfer data from/to, based on workload information.</p> +</li> +<li> +<p>The client sends a request to create a transaction to a remote NiFi node.</p> +</li> +<li> +<p>The remote NiFi node accepts the transaction.</p> +</li> +<li> +<p>Data is sent to the target peer. Multiple Data packets can be sent in batch manner.</p> +</li> +<li> +<p>When there is no more data to send, or reached to batch limit, the transaction is confirmed on both end by calculating CRC32 hash of sent data.</p> +</li> +<li> +<p>The transaction is committed on both end.</p> +</li> +</ol> +</div> +</div> +<div class="sect3"> +<h4 id="reverse-proxy-configurations"><a class="anchor" href="administration-guide.html#reverse-proxy-configurations"></a>Reverse Proxy Configurations</h4> +<div class="paragraph"> +<p>Most reverse proxy software implement HTTP and TCP proxy mode. For NiFi RAW Site-to-Site protocol, both HTTP and TCP proxy configurations are required, and at least 2 ports needed to be opened. NiFi HTTP Site-to-Site protocol can minimize the required number of open ports at the reverse proxy to 1.</p> +</div> +<div class="paragraph"> +<p>Setting correct HTTP headers at reverse proxies are crucial for NiFi to work correctly, not only routing requests but also authorize client requests. See also <a href="administration-guide.html#proxy_configuration">Proxy Configuration</a> for details.</p> +</div> +<div class="paragraph"> +<p>There are two types of requests-to-NiFi-node mapping techniques those can be applied at reverse proxy servers. One is 'Server name to Node' and the other is 'Port number to Node'.</p> +</div> +<div class="paragraph"> +<p>With 'Server name to Node', the same port can be used to route requests to different upstream NiFi nodes based on the requested server name (e.g. nifi0.example.com, nifi1.example.com). Host name resolution should be configured to map different host names to the same reverse proxy address, that can be done by adding /etc/hosts file or DNS server entries. Also, if clients to reverse proxy uses HTTPS, reverse proxy server certificate should have wildcard common name or SAN to be accessed by different host names.</p> +</div> +<div class="paragraph"> +<p>Some reverse proxy technologies do not support server name routing rules, in such case, use 'Port number to Node' technique. 'Port number to Node' mapping requires N open port at a reverse proxy for a NiFi cluster consists of N nodes.</p> +</div> +<div class="paragraph"> +<p>Refer following examples for actual configurations.</p> +</div> +</div> +<div class="sect3"> +<h4 id="site-to-site-and-reverse-proxy-examples"><a class="anchor" href="administration-guide.html#site-to-site-and-reverse-proxy-examples"></a>Site to Site and Reverse Proxy Examples</h4> +<div class="paragraph"> +<p>Here are some example reverse proxy and NiFi setups to illustrate how configuration files look like.</p> +</div> +<div class="paragraph"> +<p>Client1 in the following diagrams represents a client that does not have direct access to NiFi nodes, and it accesses through the reverse proxy, while Client2 has direct access.</p> +</div> +<div class="paragraph"> +<p>In this example, Nginx is used as a reverse proxy.</p> +</div> +<div class="sect4"> +<h5 id="example-1-raw-server-name-to-node-mapping"><a class="anchor" href="administration-guide.html#example-1-raw-server-name-to-node-mapping"></a>Example 1: RAW - Server name to Node mapping</h5> +<div class="paragraph"> +<p><span class="image"><img src="images/s2s-rproxy-servername.svg" alt="Server name to Node mapping"></span></p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>Client1 initiates Site-to-Site protocol, the request is routed to one of upstream NiFi nodes. The NiFi node computes Site-to-Site port for RAW. By the routing rule 'example1' in nifi.properties shown below, port 10443 is returned.</p> +</li> +<li> +<p>Client1 asks peers to 'nifi.example.com:10443', the request is routed to 'nifi0:8081'. The NiFi node computes available peers, by 'example1' routing rule, 'nifi0:8081' is converted to 'nifi0.example.com:10443', so are nifi1 and nifi2. As a result, 'nifi0.example.com:10443', 'nifi1.example.com:10443' and 'nifi2.example.com:10443' are returned.</p> +</li> +<li> +<p>Client1 decides to use 'nifi2.example.com:10443' for further communication.</p> +</li> +<li> +<p>On the other hand, Client2 has two URIs for Site-to-Site bootstrap URIs, and initiates the protocol using one of them. The 'example1' routing does not match this for this request, and port 8081 is returned.</p> +</li> +<li> +<p>Client2 asks peers from 'nifi1:8081'. The 'example1' does not match, so the original 'nifi0:8081', 'nifi1:8081' and 'nifi2:8081' are returned as they are.</p> +</li> +<li> +<p>Client2 decides to use 'nifi2:8081' for further communication.</p> +</li> +</ol> +</div> +<div class="paragraph"> +<p>Routing rule 'example1' is defined in nifi.properties (all node has the same routing configuration):</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre># S2S Routing for RAW, using server name to node +nifi.remote.route.raw.example1.when=\ +${X-ProxyHost:equals('nifi.example.com'):or(\ +${s2s.source.hostname:equals('nifi.example.com'):or(\ +${s2s.source.hostname:equals('192.168.99.100')})})} +nifi.remote.route.raw.example1.hostname=${s2s.target.hostname}.example.com +nifi.remote.route.raw.example1.port=10443 +nifi.remote.route.raw.example1.secure=true</pre> +</div> +</div> +<div class="paragraph"> +<p>nginx.conf</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>http { + + upstream nifi { + server nifi0:8443; + server nifi1:8443; + server nifi2:8443; + } + + # Use dnsmasq so that hostnames such as 'nifi0' can be resolved by /etc/hosts + resolver 127.0.0.1; + + server { + listen 443 ssl; + server_name nifi.example.com; + ssl_certificate /etc/nginx/nginx.crt; + ssl_certificate_key /etc/nginx/nginx.key; + + proxy_ssl_certificate /etc/nginx/nginx.crt; + proxy_ssl_certificate_key /etc/nginx/nginx.key; + proxy_ssl_trusted_certificate /etc/nginx/nifi-cert.pem; + + location / { + proxy_pass https://nifi; + proxy_set_header X-ProxyScheme https; + proxy_set_header X-ProxyHost nginx.example.com; + proxy_set_header X-ProxyPort 17590; + proxy_set_header X-ProxyContextPath /; + proxy_set_header X-ProxiedEntitiesChain $ssl_client_s_dn; + } + } +} + +stream { + + map $ssl_preread_server_name $nifi { + nifi0.example.com nifi0; + nifi1.example.com nifi1; + nifi2.example.com nifi2; + default nifi0; + } + + resolver 127.0.0.1; + + server { + listen 10443; + proxy_pass $nifi:8081; + } +}</pre> +</div> +</div> +</div> +<div class="sect4"> +<h5 id="example-2-raw-port-number-to-node-mapping"><a class="anchor" href="administration-guide.html#example-2-raw-port-number-to-node-mapping"></a>Example 2: RAW - Port number to Node mapping</h5> +<div class="paragraph"> +<p><span class="image"><img src="images/s2s-rproxy-portnumber.svg" alt="Port number to Node mapping"></span></p> +</div> +<div class="paragraph"> +<p>The 'example2' routing maps original host names (nifi0, 1 and 2) to different proxy ports (10443, 10444 and 10445) using 'equals and 'ifElse' expressions.</p> +</div> +<div class="paragraph"> +<p>nifi.properties (all node has the same routing configuration)</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre># S2S Routing for RAW, using port number to node +nifi.remote.route.raw.example2.when=\ +${X-ProxyHost:equals('nifi.example.com'):or(\ +${s2s.source.hostname:equals('nifi.example.com'):or(\ +${s2s.source.hostname:equals('192.168.99.100')})})} +nifi.remote.route.raw.example2.hostname=nifi.example.com +nifi.remote.route.raw.example2.port=\ +${s2s.target.hostname:equals('nifi0'):ifElse('10443',\ +${s2s.target.hostname:equals('nifi1'):ifElse('10444',\ +${s2s.target.hostname:equals('nifi2'):ifElse('10445',\ +'undefined')})})} +nifi.remote.route.raw.example2.secure=true</pre> +</div> +</div> +<div class="paragraph"> +<p>nginx.conf</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>http { + # Same as example 1. +} + +stream { + + map $ssl_preread_server_name $nifi { + nifi0.example.com nifi0; + nifi1.example.com nifi1; + nifi2.example.com nifi2; + default nifi0; + } + + resolver 127.0.0.1; + + server { + listen 10443; + proxy_pass nifi0:8081; + } + server { + listen 10444; + proxy_pass nifi1:8081; + } + server { + listen 10445; + proxy_pass nifi2:8081; + } +}</pre> +</div> +</div> +</div> +<div class="sect4"> +<h5 id="example-3-http-server-name-to-node-mapping"><a class="anchor" href="administration-guide.html#example-3-http-server-name-to-node-mapping"></a>Example 3: HTTP - Server name to Node mapping</h5> +<div class="paragraph"> +<p><span class="image"><img src="images/s2s-rproxy-http.svg" alt="Server name to Node mapping"></span></p> +</div> +<div class="paragraph"> +<p>nifi.properties (all node has the same routing configuration)</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre># S2S Routing for HTTP +nifi.remote.route.http.example3.when=${X-ProxyHost:contains('.example.com')} +nifi.remote.route.http.example3.hostname=${s2s.target.hostname}.example.com +nifi.remote.route.http.example3.port=443 +nifi.remote.route.http.example3.secure=true</pre> +</div> +</div> +<div class="paragraph"> +<p>nginx.conf</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>http { + upstream nifi_cluster { + server nifi0:8443; + server nifi1:8443; + server nifi2:8443; + } + + # If target node is not specified, use one from cluster. + map $http_host $nifi { + nifi0.example.com:443 "nifi0:8443"; + nifi1.example.com:443 "nifi1:8443"; + nifi2.example.com:443 "nifi2:8443"; + default "nifi_cluster"; + } + + resolver 127.0.0.1; + + server { + listen 443 ssl; + server_name ~^(.+\.example\.com)$; + ssl_certificate /etc/nginx/nginx.crt; + ssl_certificate_key /etc/nginx/nginx.key; + + proxy_ssl_certificate /etc/nginx/nginx.crt; + proxy_ssl_certificate_key /etc/nginx/nginx.key; + proxy_ssl_trusted_certificate /etc/nginx/nifi-cert.pem; + + location / { + proxy_pass https://$nifi; + proxy_set_header X-ProxyScheme https; + proxy_set_header X-ProxyHost $1; + proxy_set_header X-ProxyPort 443; + proxy_set_header X-ProxyContextPath /; + proxy_set_header X-ProxiedEntitiesChain $ssl_client_s_dn; + } + } +}</pre> +</div> +</div> +</div> +</div> +</div> +<div class="sect2"> <h3 id="web-properties"><a class="anchor" href="administration-guide.html#web-properties"></a>Web Properties</h3> <div class="paragraph"> <p>These properties pertain to the web-based User Interface.</p> @@ -6457,7 +6906,7 @@ Security Configuration section of this A </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.sensitive.props.additional.keys</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The comma separated list of properties to encrypt in addition to the default sensitive properties (see <a href="administration-guide.html#encrypt-config_tool">Encrypt-Config Tool</a>).</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The comma separated list of properties in <code>nifi.properties</code> to encrypt in addition to the default sensitive properties (see <a href="administration-guide.html#encrypt-config_tool">Encrypt-Config Tool</a>).</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.security.keystore*</p></td> @@ -6523,13 +6972,18 @@ in the file specified in <code>nifi.logi <div class="content"> <pre>nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$ nifi.security.identity.mapping.value.dn=$1@$2 +nifi.security.identity.mapping.transform.dn=NONE nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$ -nifi.security.identity.mapping.value.kerb=$1@$2</pre> +nifi.security.identity.mapping.value.kerb=$1@$2 +nifi.security.identity.mapping.transform.kerb=NONE</pre> </div> </div> <div class="paragraph"> <p>The last segment of each property is an identifier used to associate the pattern with the replacement value. When a user makes a request to NiFi, their identity is checked to see if it matches each of those patterns in lexicographical order. For the first one that matches, the replacement specified in the <code>nifi.security.identity.mapping.value.xxxx</code> property is used. So a login with <code>CN=localhost, OU=Apache NiFi, O=Apache, L=Santa Monica, ST=CA, C=US</code> matches the DN mapping pattern above and the DN mapping value <code>$1@$2</code> is applied. The user is normalized to <code>localhost@Apache NiFi</code>.</p> </div> +<div class="paragraph"> +<p>In addition to mapping a transform may be applied. The supported versions are NONE (no transform applied), LOWER (identity lowercased), and UPPER (identity uppercased). If not specified, the default value is NONE.</p> +</div> <div class="admonitionblock note"> <table> <tr> @@ -6537,7 +6991,29 @@ nifi.security.identity.mapping.value.ker <i class="fa icon-note" title="Note"></i> </td> <td class="content"> -These mappings are also applied to the "Initial Admin Identity" and "Cluster Node Identity" properties in the authorizers.xml file (See <a href="administration-guide.html#authorizers-setup">Authorizers.xml Setup</a>). +These mappings are also applied to the "Initial Admin Identity", "Cluster Node Identity", and any legacy users in the authorizers.xml file as well as users imported from LDAP (See <a href="administration-guide.html#authorizers-setup">Authorizers.xml Setup</a>). +</td> +</tr> +</table> +</div> +<div class="paragraph"> +<p>Group names can also be mapped. The following example will accept the existing group name but will lowercase it. This may be helpful when used in conjunction with an external authorizer.</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>nifi.security.group.mapping.pattern.anygroup=^(.*)$ +nifi.security.group.mapping.value.anygroup=$1 +nifi.security.group.mapping.transform.anygroup=LOWER</pre> +</div> +</div> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +These mappings are applied to any legacy groups referenced in the authorizers.xml as well as groups imported from LDAP. </td> </tr> </table> @@ -6621,6 +7097,10 @@ the connection a failure. The default va from the remote node before considering the communication with the node a failure. The default value is <code>5 secs</code>.</p></td> </tr> <tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.cluster.node.max.concurrent.requests</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of outstanding web requests that can be replicated to nodes in the cluster. If this number of requests is exceeded, the embedded Jetty server will return a "409: Conflict" response. This property defaults to <code>100</code>.</p></td> +</tr> +<tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.cluster.firewall.file</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">The location of the node firewall file. This is a file that may be used to list all the nodes that are allowed to connect to the cluster. It provides an additional layer of security. This value is blank by default, meaning that no firewall file is to be used.</p></td> @@ -6829,7 +7309,7 @@ that is specified.</p></td> </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-04-03 08:25:54 -07:00 +Last updated 2018-06-19 22:13:11 -07:00 </div> </div> </body>
Modified: nifi/site/trunk/docs/nifi-docs/html/developer-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/developer-guide.html?rev=1834461&r1=1834460&r2=1834461&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/developer-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/developer-guide.html Wed Jun 27 00:37:46 2018 @@ -1,4 +1,20 @@ -<!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -2643,6 +2659,13 @@ will always be set to <code>1</code>. Th as the thread that is executing <code>onTrigger</code> may change between invocations.</p> </li> <li> +<p><code>PrimaryNodeOnly</code>: Apache NiFi, when clustered, offers two modes of execution for Processors: "Primary Node" and +"All Nodes". Although running in all the nodes offers better parallelism, some Processors are known to cause unintended +behaviors when run in multiple nodes. For instance, some Processors list or read files from remote filesystems. If such +Processors are scheduled to run on "All Nodes", it will cause unnecessary duplication and even errors. Such Processors +should use this annotation. Applying this annotation will restrict the Processor to run only on the "Primary Node".</p> +</li> +<li> <p><code>TriggerWhenAnyDestinationAvailable</code>: By default, NiFi will not schedule a Processor to run if any of its outbound queues is full. This allows back-pressure to be applied all the way a chain of Processors. However, some Processors may need to run even if one of the outbound queues is full. This annotations indicates that the Processor should run @@ -3632,7 +3655,7 @@ worry about bothering us. Just ping the </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-04-03 08:25:54 -07:00 +Last updated 2018-06-19 22:13:11 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html?rev=1834461&r1=1834460&r2=1834461&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html Wed Jun 27 00:37:46 2018 @@ -1,4 +1,20 @@ -<!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -755,8 +771,37 @@ icon ( <span class="image"><img src="images/iconInfo.png" alt="Info"></span> ) next to the name of the Property. Hovering over this icon with the mouse will provide a tooltip that provides helpful information about the Property. This information includes a description of the Property, -the default value (if any), historically configured values (if any), and whether or not this Property -supports the expression language.</p> +the default value (if any), historically configured values (if any), and the evaluation scope of this +property for expression language. There are three values and the evaluation scope of the expression +language is hierarchical: NONE → VARIABLE_REGISTRY → FLOWFILE_ATTRIBUTES.</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>NONE - expression language is not supported for this property</p> +</li> +<li> +<p>VARIABLE_REGISTRY is hierarchically constructed as below:</p> +<div class="ulist"> +<ul> +<li> +<p>Variables defined at process group level and then, recursively, up to the higher process group until +the root process group.</p> +</li> +<li> +<p>Variables defined in custom properties files through the nifi.variable.registry.properties property +in nifi.properties file.</p> +</li> +<li> +<p>Environment variables defined at JVM level and system properties.</p> +</li> +</ul> +</div> +</li> +<li> +<p>FLOWFILE_ATTRIBUTES - will use attributes of each individual flow file.</p> +</li> +</ul> </div> <div class="sect2"> <h3 id="editor"><a class="anchor" href="expression-language-guide.html#editor"></a>Expression Language Editor</h3> @@ -3902,7 +3947,7 @@ an embedded Expression, though it does n </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-04-03 08:25:54 -07:00 +Last updated 2018-06-19 22:13:11 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/getting-started.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/getting-started.html?rev=1834461&r1=1834460&r2=1834461&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/getting-started.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/getting-started.html Wed Jun 27 00:37:46 2018 @@ -1,4 +1,20 @@ -<!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -1607,7 +1623,7 @@ work back to the Apache NiFi community s </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-04-03 08:25:54 -07:00 +Last updated 2018-06-19 22:13:11 -07:00 </div> </div> </body> Added: nifi/site/trunk/docs/nifi-docs/html/images/PG1_imported_diff.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/PG1_imported_diff.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/PG1_imported_diff.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/PG1_imported_same.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/PG1_imported_same.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/PG1_imported_same.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/PG1_process_group.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/PG1_process_group.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/PG1_process_group.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/PG1_variable_ref_2.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/PG1_variable_ref_2.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/PG1_variable_ref_2.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/PG1_variable_ref_PG.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/PG1_variable_ref_PG.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/PG1_variable_ref_PG.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/PG1_versioned_flow.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/PG1_versioned_flow.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/PG1_versioned_flow.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/abc-restricted-component-flow.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/abc-restricted-component-flow.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/abc-restricted-component-flow.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/abc-versioned-flow.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/abc-versioned-flow.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/abc-versioned-flow.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/getfile-permissions.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/getfile-permissions.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/getfile-permissions.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/import-xyz-flow-fails.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/import-xyz-flow-fails.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/import-xyz-flow-fails.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/keytabCredentialsService-pg.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/keytabCredentialsService-pg.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/keytabCredentialsService-pg.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/keytabCredentialsService-rpg.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/keytabCredentialsService-rpg.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/keytabCredentialsService-rpg.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/keytabcredentialsservice-permissions.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/keytabcredentialsservice-permissions.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/keytabcredentialsservice-permissions.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/primary-node-processor.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/primary-node-processor.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/primary-node-processor.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/primary-node-processors-summary.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/primary-node-processors-summary.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/primary-node-processors-summary.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-no-kerberosCS.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-no-kerberosCS.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-no-kerberosCS.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-permissions.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-permissions.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-permissions.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-properties.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-properties.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-properties.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-properties_2.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-properties_2.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/puthdfs-properties_2.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/revert-failure.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/revert-failure.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/revert-failure.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/revert-success.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/revert-success.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/revert-success.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/rpg-variable.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/rpg-variable.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/rpg-variable.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-http.svg URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-http.svg?rev=1834461&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-http.svg (added) +++ nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-http.svg Wed Jun 27 00:37:46 2018 @@ -0,0 +1,17 @@ +<?xml version="1.0" standalone="yes"?> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> [... 3 lines stripped ...] Added: nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-portnumber.svg URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-portnumber.svg?rev=1834461&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-portnumber.svg (added) +++ nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-portnumber.svg Wed Jun 27 00:37:46 2018 @@ -0,0 +1,17 @@ +<?xml version="1.0" standalone="yes"?> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> [... 3 lines stripped ...] Added: nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-servername.svg URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-servername.svg?rev=1834461&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-servername.svg (added) +++ nifi/site/trunk/docs/nifi-docs/html/images/s2s-rproxy-servername.svg Wed Jun 27 00:37:46 2018 @@ -0,0 +1,17 @@ +<?xml version="1.0" standalone="yes"?> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> [... 3 lines stripped ...] Added: nifi/site/trunk/docs/nifi-docs/html/images/sys_admin-restricted-component-access-policy.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/sys_admin-restricted-component-access-policy.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/sys_admin-restricted-component-access-policy.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-abc-flow.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-abc-flow.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-abc-flow.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-success.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-success.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-success.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-xyz-flow.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-xyz-flow.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/test_user-import-xyz-flow.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/test_user-restricted-component-read-filesystem.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/test_user-restricted-component-read-filesystem.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/test_user-restricted-component-read-filesystem.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/test_user-restricted-component-write-filesystem.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/test_user-restricted-component-write-filesystem.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/test_user-restricted-component-write-filesystem.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/test_user-revert-local-changes-2.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/test_user-revert-local-changes-2.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/test_user-revert-local-changes-2.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/test_user-revert-local-changes.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/test_user-revert-local-changes.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/test_user-revert-local-changes.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/xyz-flow.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/xyz-flow.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/xyz-flow.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: nifi/site/trunk/docs/nifi-docs/html/images/xyz-process-group.png URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/images/xyz-process-group.png?rev=1834461&view=auto ============================================================================== Binary file - no diff available. Propchange: nifi/site/trunk/docs/nifi-docs/html/images/xyz-process-group.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Modified: nifi/site/trunk/docs/nifi-docs/html/nifi-in-depth.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/nifi-in-depth.html?rev=1834461&r1=1834460&r2=1834461&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/nifi-in-depth.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/nifi-in-depth.html Wed Jun 27 00:37:46 2018 @@ -1,4 +1,20 @@ -<!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -849,7 +865,7 @@ This section relies heavily on informati </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-04-03 08:25:54 -07:00 +Last updated 2018-06-19 22:13:11 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/overview.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/overview.html?rev=1834461&r1=1834460&r2=1834461&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/overview.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/overview.html Wed Jun 27 00:37:46 2018 @@ -1,4 +1,20 @@ -<!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -901,7 +917,7 @@ Scalable Internet Services [online]. Re </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-04-03 08:25:54 -07:00 +Last updated 2018-06-19 22:13:11 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/record-path-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/record-path-guide.html?rev=1834461&r1=1834460&r2=1834461&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/record-path-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/record-path-guide.html Wed Jun 27 00:37:46 2018 @@ -1,4 +1,20 @@ -<!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -464,6 +480,8 @@ body.book #toc,body.book #preamble,body. <li><a href="record-path-guide.html#concat">concat</a></li> <li><a href="record-path-guide.html#fieldname">fieldName</a></li> <li><a href="record-path-guide.html#todate">toDate</a></li> +<li><a href="record-path-guide.html#tostring">toString</a></li> +<li><a href="record-path-guide.html#tobytes">toBytes</a></li> <li><a href="record-path-guide.html#format">format</a></li> </ul> </li> @@ -1244,6 +1262,77 @@ the value of the "city" field whose pare </div> </div> <div class="sect2"> +<h3 id="tostring"><a class="anchor" href="record-path-guide.html#tostring"></a>toString</h3> +<div class="paragraph"> +<p>Converts a value to a String, using the given character set if the input type is "bytes". For example, +given a schema such as:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>{ + "type": "record", + "name": "events", + "fields": [ + { "name": "name", "type": "string" }, + { "name": "bytes", "type" : "bytes"} + ] +}</pre> +</div> +</div> +<div class="paragraph"> +<p>and a record such as:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>{ + "name" : "My Event", + "bytes" : "Hello World!" +}</pre> +</div> +</div> +<div class="paragraph"> +<p>The following record path would parse the bytes field into a String:</p> +</div> +<div class="paragraph"> +<p><code>toString( /bytes, "UTF-8")</code></p> +</div> +</div> +<div class="sect2"> +<h3 id="tobytes"><a class="anchor" href="record-path-guide.html#tobytes"></a>toBytes</h3> +<div class="paragraph"> +<p>Converts a String to byte[], using the given character set. For example, given a schema such as:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>{ + "type": "record", + "name": "events", + "fields": [ + { "name": "name", "type": "string" }, + { "name": "s", "type" : "string"} + ] +}</pre> +</div> +</div> +<div class="paragraph"> +<p>and a record such as:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>{ + "name" : "My Event", + "s" : "Hello World!" +}</pre> +</div> +</div> +<div class="paragraph"> +<p>The following record path would convert the String field into a byte array using UTF-16 encoding:</p> +</div> +<div class="paragraph"> +<p><code>toBytes( /s, "UTF-16")</code></p> +</div> +</div> +<div class="sect2"> <h3 id="format"><a class="anchor" href="record-path-guide.html#format"></a>format</h3> <div class="paragraph"> <p>Converts a Date to a String in the given format.</p> @@ -1589,7 +1678,7 @@ only of white space (spaces, tabs, carri </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-04-03 08:25:54 -07:00 +Last updated 2018-06-19 22:13:11 -07:00 </div> </div> </body>
