Modified: nifi/site/trunk/docs/nifi-docs/html/administration-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/administration-guide.html?rev=1828578&r1=1828577&r2=1828578&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/administration-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/administration-guide.html Sat Apr 7 00:33:22 2018 @@ -1,20 +1,4 @@ -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - --> - <!DOCTYPE html> +<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -1201,7 +1185,7 @@ JKS keystores and truststores are recomm <h2 id="user_authentication"><a class="anchor" href="administration-guide.html#user_authentication"></a>User Authentication</h2> <div class="sectionbody"> <div class="paragraph"> -<p>NiFi supports user authentication via client certificates, via username/password, via Apache Knox, or via OpenId Connect (<a href="http://openid.net/connect"; class="bare">http://openid.net/connect</a>).</p> +<p>NiFi supports user authentication via client certificates, via username/password, via Apache Knox, or via <a href="http://openid.net/connect"; target="_blank">OpenId Connect</a>.</p> </div> <div class="paragraph"> <p>Username/password authentication is performed by a 'Login Identity Provider'. The Login Identity Provider is a pluggable mechanism for @@ -1235,7 +1219,7 @@ these concurrently. NiFi will require cl </table> </div> <div class="paragraph"> -<p>A secured instance of NiFi cannot be accessed anonymously unless configured to use an LDAP or Kerberos Login Identity Provider, which in turn must be configured to explicitly allow anonymous access. Anonymous access is not currently possible by the default FileAuthorizer (see <a href="administration-guide.html#authorizer-configuration">Authorizer Configuration</a>), but is a future effort (<a href="https://issues.apache.org/jira/browse/NIFI-2730";>NIFI-2730</a>).</p> +<p>A secured instance of NiFi cannot be accessed anonymously unless configured to use an LDAP or Kerberos Login Identity Provider, which in turn must be configured to explicitly allow anonymous access. Anonymous access is not currently possible by the default FileAuthorizer (see <a href="administration-guide.html#authorizer-configuration">Authorizer Configuration</a>), but is a future effort (<a href="https://issues.apache.org/jira/browse/NIFI-2730"; target="_blank">NIFI-2730</a>).</p> </div> <div class="admonitionblock note"> <table> @@ -1477,7 +1461,7 @@ JSON Web Key (JWK) provided through the <tbody> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.discovery.url</code></p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The discovery URL for the desired OpenId Connect Provider (<a href="http://openid.net/specs/openid-connect-discovery-1_0.html"; class="bare">http://openid.net/specs/openid-connect-discovery-1_0.html</a>).</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The discovery URL for the desired OpenId Connect Provider (<a href="http://openid.net/specs/openid-connect-discovery-1_0.html"; target="_blank">http://openid.net/specs/openid-connect-discovery-1_0.html</a>).</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock"><code>nifi.security.user.oidc.connect.timeout</code></p></td> @@ -2584,7 +2568,10 @@ Instructions requiring interaction with </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">access restricted components</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to create/modify restricted components assuming otherwise sufficient permissions</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to create/modify restricted components assuming other permissions are sufficient. The restricted +components may indicate which specific permissions are required. Permissions can be granted for specific +restrictions or be granted regardless of restrictions. If permission is granted regardless of restrictions, +the user can create/modify all restricted components.</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">N/A</p></td> </tr> <tr> @@ -2991,7 +2978,7 @@ You cannot modify the users/groups on an <div class="sect2"> <h3 id="key-derivation-functions"><a class="anchor" href="administration-guide.html#key-derivation-functions"></a>Key Derivation Functions</h3> <div class="paragraph"> -<p>Key Derivation Functions (KDF) are mechanisms by which human-readable information, usually a password or other secret information, is translated into a cryptographic key suitable for data protection. For further information, read <a href="https://en.wikipedia.org/wiki/Key_derivation_function";>the Wikipedia entry on Key Derivation Functions</a>. +<p>Key Derivation Functions (KDF) are mechanisms by which human-readable information, usually a password or other secret information, is translated into a cryptographic key suitable for data protection. For further information, read the <a href="https://en.wikipedia.org/wiki/Key_derivation_function"; target="_blank">Wikipedia entry on Key Derivation Functions</a>. Currently, KDFs are ingested by <code>CipherProvider</code> implementations and return a fully-initialized <code>Cipher</code> object to be used for encryption or decryption. Due to the use of a <code>CipherProviderFactory</code>, the KDFs are not customizable at this time. Future enhancements will include the ability to provide custom cost parameters to the KDF at initialization time. As a work-around, <code>CipherProvider</code> instances can be initialized with custom cost parameters in the constructor but this is not currently supported by the <code>CipherProviderFactory</code>. Here are the KDFs currently supported by NiFi (primarily in the <code>EncryptContent</code> processor for password-based encryption (PBE)) and relevant notes:</p> </div> @@ -3031,7 +3018,7 @@ Here are the KDFs currently supported by <p>This KDF was added in v0.5.0.</p> </li> <li> -<p><a href="https://en.wikipedia.org/wiki/Bcrypt";>Bcrypt</a> is an adaptive function based on the <a href="https://en.wikipedia.org/wiki/Blowfish_(cipher)">Blowfish</a> cipher. This KDF is strongly recommended as it automatically incorporates a random 16 byte salt, configurable cost parameter (or "work factor"), and is hardened against brute-force attacks using <a href="https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units";>GPGPU</a> (which share memory between cores) by requiring access to "large" blocks of memory during the key derivation. It is less resistant to <a href="https://en.wikipedia.org/wiki/Field-programmable_gate_array";>FPGA</a> brute-force attacks where the gate arrays have access to individual embedded RAM blocks.</p> +<p><a href="https://en.wikipedia.org/wiki/Bcrypt"; target="_blank">Bcrypt</a> is an adaptive function based on the <a href="https://en.wikipedia.org/wiki/Blowfish_(cipher)" target="_blank">Blowfish</a> cipher. This KDF is strongly recommended as it automatically incorporates a random 16 byte salt, configurable cost parameter (or "work factor"), and is hardened against brute-force attacks using <a href="https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units"; target="_blank">GPGPU</a> (which share memory between cores) by requiring access to "large" blocks of memory during the key derivation. It is less resistant to <a href="https://en.wikipedia.org/wiki/Field-programmable_gate_array"; target="_blank">FPGA</a> brute-force attacks where the gate arrays have access to individual embedded RAM blocks.</p> </li> <li> <p>Because the length of a Bcrypt-derived key is always 184 bits, the complete output is then fed to a <code>SHA-512</code> digest and truncated to the desired key length. This provides the benefit of the avalanche effect on the formatted input.</p> @@ -3044,7 +3031,7 @@ Here are the KDFs currently supported by <div class="ulist"> <ul> <li> -<p><code>2a</code> - the version of the format. An extensive explanation can be found <a href="http://blog.ircmaxell.com/2012/12/seven-ways-to-screw-up-bcrypt.html";>here</a>. NiFi currently uses <code>2a</code> for all salts generated internally.</p> +<p><code>2a</code> - the version of the format. An extensive explanation can be found <a href="http://blog.ircmaxell.com/2012/12/seven-ways-to-screw-up-bcrypt.html"; target="_blank">here</a>. NiFi currently uses <code>2a</code> for all salts generated internally.</p> </li> <li> <p><code>10</code> - the work factor. This is actually the log<sub>2</sub> value, so the total iteration count would be 2<sup>10</sup> in this case.</p> @@ -3066,7 +3053,7 @@ Here are the KDFs currently supported by <p>This KDF was added in v0.5.0.</p> </li> <li> -<p><a href="https://en.wikipedia.org/wiki/Scrypt";>Scrypt</a> is an adaptive function designed in response to <code>bcrypt</code>. This KDF is recommended as it requires relatively large amounts of memory for each derivation, making it resistant to hardware brute-force attacks.</p> +<p><a href="https://en.wikipedia.org/wiki/Scrypt"; target="_blank">Scrypt</a> is an adaptive function designed in response to <code>bcrypt</code>. This KDF is recommended as it requires relatively large amounts of memory for each derivation, making it resistant to hardware brute-force attacks.</p> </li> <li> <p>The recommended minimum cost is <code>N</code>=2<sup>14</sup>, <code>r</code>=8, <code>p</code>=1 (as of 2/1/2016 on commodity hardware) and should be increased to the threshold at which legitimate systems will encounter detrimental delays (see schedule below or use <code>ScryptCipherProviderGroovyTest#testDefaultConstructorShouldProvideStrongParameters()</code> to calculate safe minimums).</p> @@ -3105,7 +3092,7 @@ Here are the KDFs currently supported by <p>This KDF was added in v0.5.0.</p> </li> <li> -<p><a href="https://en.wikipedia.org/wiki/PBKDF2";>Password-Based Key Derivation Function 2</a> is an adaptive derivation function which uses an internal pseudorandom function (PRF) and iterates it many times over a password and salt (at least 16 bytes).</p> +<p><a href="https://en.wikipedia.org/wiki/PBKDF2"; target="_blank">Password-Based Key Derivation Function 2</a> is an adaptive derivation function which uses an internal pseudorandom function (PRF) and iterates it many times over a password and salt (at least 16 bytes).</p> </li> <li> <p>The PRF is recommended to be <code>HMAC/SHA-256</code> or <code>HMAC/SHA-512</code>. The use of an HMAC cryptographic hash function mitigates a length extension attack.</p> @@ -3139,40 +3126,40 @@ Here are the KDFs currently supported by <div class="ulist"> <ul> <li> -<p><a href="http://stackoverflow.com/a/30308723/70465";>Explanation of optimal scrypt cost parameters and relationships</a></p> +<p><a href="http://stackoverflow.com/a/30308723/70465"; target="_blank">Explanation of optimal scrypt cost parameters and relationships</a></p> </li> <li> -<p><a href="http://csrc.nist.gov/publications/nistpubs/800-132/nist-sp800-132.pdf";>NIST Special Publication 800-132</a></p> +<p><a href="http://csrc.nist.gov/publications/nistpubs/800-132/nist-sp800-132.pdf"; target="_blank">NIST Special Publication 800-132</a></p> </li> <li> -<p><a href="https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet#Work_Factor";>OWASP Password Storage Work Factor Calculations</a></p> +<p><a href="https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet#Work_Factor"; target="_blank">OWASP Password Storage Work Factor Calculations</a></p> </li> <li> -<p><a href="http://security.stackexchange.com/a/3993/16485";>PBKDF2 rounds calculations</a></p> +<p><a href="http://security.stackexchange.com/a/3993/16485"; target="_blank">PBKDF2 rounds calculations</a></p> </li> <li> -<p><a href="http://blog.ircmaxell.com/2014/03/why-i-dont-recommend-scrypt.html";>Scrypt as KDF vs password storage vulnerabilities</a></p> +<p><a href="http://blog.ircmaxell.com/2014/03/why-i-dont-recommend-scrypt.html"; target="_blank">Scrypt as KDF vs password storage vulnerabilities</a></p> </li> <li> -<p><a href="http://security.stackexchange.com/a/26253/16485";>Scrypt vs. Bcrypt (as of 2010)</a></p> +<p><a href="http://security.stackexchange.com/a/26253/16485"; target="_blank">Scrypt vs. Bcrypt (as of 2010)</a></p> </li> <li> -<p><a href="http://security.stackexchange.com/a/6415/16485";>Bcrypt vs PBKDF2</a></p> +<p><a href="http://security.stackexchange.com/a/6415/16485"; target="_blank">Bcrypt vs PBKDF2</a></p> </li> <li> -<p><a href="http://wildlyinaccurate.com/bcrypt-choosing-a-work-factor/";>Choosing a work factor for Bcrypt</a></p> +<p><a href="http://wildlyinaccurate.com/bcrypt-choosing-a-work-factor/"; target="_blank">Choosing a work factor for Bcrypt</a></p> </li> <li> -<p><a href="https://docs.spring.io/spring-security/site/docs/current/apidocs/org/springframework/security/crypto/bcrypt/BCrypt.html";>Spring Security Bcrypt</a></p> +<p><a href="https://docs.spring.io/spring-security/site/docs/current/apidocs/org/springframework/security/crypto/bcrypt/BCrypt.html"; target="_blank">Spring Security Bcrypt</a></p> </li> <li> -<p><a href="https://www.openssl.org/docs/man1.1.0/crypto/EVP_BytesToKey.html";>OpenSSL EVP BytesToKey PKCS#1v1.5</a></p> +<p><a href="https://www.openssl.org/docs/man1.1.0/crypto/EVP_BytesToKey.html"; target="_blank">OpenSSL EVP BytesToKey PKCS#1v1.5</a></p> </li> <li> -<p><a href="https://wiki.openssl.org/index.php/Manual:PKCS5_PBKDF2_HMAC(3)">OpenSSL PBKDF2 KDF</a></p> +<p><a href="https://wiki.openssl.org/index.php/Manual:PKCS5_PBKDF2_HMAC(3)" target="_blank">OpenSSL PBKDF2 KDF</a></p> </li> <li> -<p><a href="http://security.stackexchange.com/a/29139/16485";>OpenSSL KDF flaws description</a></p> +<p><a href="http://security.stackexchange.com/a/29139/16485"; target="_blank">OpenSSL KDF flaws description</a></p> </li> </ul> </div> @@ -3223,7 +3210,7 @@ Here are the KDFs currently supported by <div class="sect2"> <h3 id="java-cryptography-extension-jce-limited-strength-jurisdiction-policies"><a class="anchor" href="administration-guide.html#java-cryptography-extension-jce-limited-strength-jurisdiction-policies"></a>Java Cryptography Extension (JCE) Limited Strength Jurisdiction Policies</h3> <div class="paragraph"> -<p>Because of US export regulations, default JVMs have <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#importlimits";>limits imposed on the strength of cryptographic operations</a> available to them. For example, AES operations are limited to <code>128 bit keys</code> by default. While <code>AES-128</code> is cryptographically safe, this can have unintended consequences, specifically on Password-based Encryption (PBE).</p> +<p>Because of US export regulations, default JVMs have <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#importlimits"; target="_blank">limits imposed on the strength of cryptographic operations</a> available to them. For example, AES operations are limited to <code>128 bit keys</code> by default. While <code>AES-128</code> is cryptographically safe, this can have unintended consequences, specifically on Password-based Encryption (PBE).</p> </div> <div class="paragraph"> <p>PBE is the process of deriving a cryptographic key for encryption or decryption from <em>user-provided secret material</em>, usually a password. Rather than a human remembering a (random-appearing) 32 or 64 character hexadecimal string, a password or passphrase is used.</p> @@ -3341,7 +3328,7 @@ Here are the KDFs currently supported by <div class="ulist"> <ul> <li> -<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html";>JCE Unlimited Strength Jurisdiction Policy files for Java 8</a></p> +<p><a href="http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html"; target="_blank">JCE Unlimited Strength Jurisdiction Policy files for Java 8</a></p> </li> </ul> </div> @@ -3364,7 +3351,7 @@ Here are the KDFs currently supported by </table> </div> <div class="paragraph"> -<p>It is preferable to request upstream/downstream systems to switch to <a href="https://cwiki.apache.org/confluence/display/NIFI/Encryption+Information";>keyed encryption</a> or use a "strong" <a href="https://cwiki.apache.org/confluence/display/NIFI/Key+Derivation+Function+Explanations";>Key Derivation Function (KDF) supported by NiFi</a>.</p> +<p>It is preferable to request upstream/downstream systems to switch to <a href="https://cwiki.apache.org/confluence/display/NIFI/Encryption+Information"; target="_blank">keyed encryption</a> or use a "strong" <a href="https://cwiki.apache.org/confluence/display/NIFI/Key+Derivation+Function+Explanations"; target="_blank">Key Derivation Function (KDF) supported by NiFi</a>.</p> </div> </div> </div> @@ -4447,7 +4434,7 @@ echo 2 > state/zookeeper/myid</code>< </div> <div class="paragraph"> <p>For more information on the properties used to administer ZooKeeper, see the -<a href="https://zookeeper.apache.org/doc/current/zookeeperAdmin.html";>ZooKeeper Admin Guide</a>.</p> +<a href="https://zookeeper.apache.org/doc/current/zookeeperAdmin.html"; target="_blank">ZooKeeper Admin Guide</a>.</p> </div> <div class="paragraph"> <p>For information on securing the embedded ZooKeeper Server, see the <a href="administration-guide.html#securing_zookeeper">Securing ZooKeeper</a> section below.</p> @@ -4486,8 +4473,7 @@ NiFi ZooKeeper client and embedded ZooKe </div> <div class="paragraph"> <p>If Kerberos is not already setup in your environment, you can find information on installing and setting up a Kerberos Server at -<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Configuring_a_Kerberos_5_Server.html";><em>https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Configuring_a_Kerberos_5_Server.html</em></a> -. This guide assumes that Kerberos already has been installed in the environment in which NiFi is running.</p> +<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Configuring_a_Kerberos_5_Server.html"; target="_blank">https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Managing_Smart_Cards/Configuring_a_Kerberos_5_Server.html</a>. This guide assumes that Kerberos already has been installed in the environment in which NiFi is running.</p> </div> <div class="paragraph"> <p>Note, the following procedures for kerberizing an Embedded ZooKeeper server in your NiFi Node and kerberizing a ZooKeeper NiFi client will require that @@ -4727,8 +4713,8 @@ in the following locations:</p> </div> </div> <div class="paragraph"> -<p>If there are problems communicating or authenticating with Kerberos, -<a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html";>this Troubleshooting Guide</a> may be of value.</p> +<p>If there are problems communicating or authenticating with Kerberos, this +<a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html"; target="_blank">Troubleshooting Guide</a> may be of value.</p> </div> <div class="paragraph"> <p>One of the most important notes in the above Troubleshooting guide is the mechanism for turning on Debug output for Kerberos. @@ -4782,7 +4768,7 @@ This output can be rather verbose but pr </div> <div class="paragraph"> <p>The NiFi ZooKeeper Migrator is part of the NiFi Toolkit and is downloaded separately from the -<a href="https://nifi.apache.org/download.html";>Apache NiFi download page</a>.</p> +<a href="https://nifi.apache.org/download.html"; target="_blank">Apache NiFi download page</a>.</p> </div> <div class="sect3"> <h4 id="zk_migrator_command_line_parameters"><a class="anchor" href="administration-guide.html#zk_migrator_command_line_parameters"></a>zk-migrator.sh Command Line Parameters</h4> @@ -5433,6 +5419,28 @@ documentation of the proxy for guidance ...</pre> </div> </div> +<div class="ulist"> +<ul> +<li> +<p>Additional NiFi proxy configuration must be updated to allow expected Host and context paths HTTP headers.</p> +<div class="ulist"> +<ul> +<li> +<p>By default, if NiFi is running securely it will only accept HTTP requests with a Host header matching the host[:port] that it is bound to. If NiFi is to accept requests directed to a different +host[:port] the expected values need to be configured. This may be required when running behind a proxy or in a containerized environment. This is configured in a comma +separated list in <em>nifi.properties</em> using the <code>nifi.web.proxy.host</code> property (e.g. localhost:18443, proxyhost:443). IPv6 addressed are accepted. Please refer to +RFC 5952 Sections <a href="https://tools.ietf.org/html/rfc5952#section-4";>4</a> and <a href="https://tools.ietf.org/html/rfc5952#section-6";>6</a> for additional details.</p> +</li> +<li> +<p>NiFi will only accept HTTP requests with a X-ProxyContextPath or X-Forwarded-Context header if the value is whitelisted in the <code>nifi.web.proxy.context.path</code> property in +<em>nifi.properties</em>. This property accepts a comma separated list of expected values. In the event an incoming request has an X-ProxyContextPath or X-Forwarded-Context header value that is not +present in the whitelist, the "An unexpected error has occurred" page will be shown and an error will be written to the nifi-app.log.</p> +</li> +</ul> +</div> +</li> +</ul> +</div> </div> </div> <div class="sect1"> @@ -5482,13 +5490,13 @@ documentation of the proxy for guidance <p>Kerberos is case-sensitive in many places and the error messages (or lack thereof) may not be sufficiently explanatory. Check the case sensitivity of the service principal in your configuration files. Convention is <code>HTTP/fully.qualified.domain@REALM</code>.</p> </li> <li> -<p>Browsers have varying levels of restriction when dealing with SPNEGO negotiations. Some will provide the local Kerberos ticket to any domain that requests it, while others whitelist the trusted domains. See <a href="http://docs.spring.io/autorepo/docs/spring-security-kerberos/1.0.2.BUILD-SNAPSHOT/reference/htmlsingle/#browserspnegoconfig";>Spring Security Kerberos - Reference Documentation: Appendix E. Configure browsers for SPNEGO Negotiation</a> for common browsers.</p> +<p>Browsers have varying levels of restriction when dealing with SPNEGO negotiations. Some will provide the local Kerberos ticket to any domain that requests it, while others whitelist the trusted domains. See <a href="http://docs.spring.io/autorepo/docs/spring-security-kerberos/1.0.2.BUILD-SNAPSHOT/reference/htmlsingle/#browserspnegoconfig"; target="_blank">Spring Security Kerberos - Reference Documentation: Appendix E. Configure browsers for SPNEGO Negotiation</a> for common browsers.</p> </li> <li> <p>Some browsers (legacy IE) do not support recent encryption algorithms such as AES, and are restricted to legacy algorithms (DES). This should be noted when generating keytabs.</p> </li> <li> -<p>The KDC must be configured and a service principal defined for NiFi and a keytab exported. Comprehensive instructions for Kerberos server configuration and administration are beyond the scope of this document (see <a href="http://web.mit.edu/kerberos/krb5-current/doc/admin/index.html";>MIT Kerberos Admin Guide</a>), but an example is below:</p> +<p>The KDC must be configured and a service principal defined for NiFi and a keytab exported. Comprehensive instructions for Kerberos server configuration and administration are beyond the scope of this document (see <a href="http://web.mit.edu/kerberos/krb5-current/doc/admin/index.html"; target="_blank">MIT Kerberos Admin Guide</a>), but an example is below:</p> </li> </ul> </div> @@ -5710,7 +5718,7 @@ for components to persist state. See the <td class="tableblock halign-left valign-top"><p class="tableblock"><strong>Description</strong></p></td> </tr> <tr> -<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.database.directory</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.database.directory*</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">The location of the H2 database directory. The default value is <code>./database_repository</code>.</p></td> </tr> <tr> @@ -5739,7 +5747,18 @@ to configure it on a separate drive if a </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.flowfile.repository.implementation</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">The FlowFile Repository implementation. The default value is <code>org.apache.nifi.controller.repository.WriteAheadFlowFileRepository</code> and should only be changed with caution. To store flowfiles in memory instead of on disk (at the risk of data loss in the event of power/machine failure), set this property to <code>org.apache.nifi.controller.repository.VolatileFlowFileRepository</code>.</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">The FlowFile Repository implementation. The default value is <code>org.apache.nifi.controller.repository.WriteAheadFlowFileRepository</code> and should only be changed with caution. To store flowfiles in memory instead of on disk (accepting data loss in the event of power/machine failure or a restart of NiFi), set this property to <code>org.apache.nifi.controller.repository.VolatileFlowFileRepository</code>.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.flowfile.repository.wal.implementation</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">If the repository implementation is configured to use the <code>WriteAheadFlowFileRepository</code>, this property can be used to specify which implementation of the +Write-Ahead Log should be used. The default value is <code>org.apache.nifi.wali.SequentialAccessWriteAheadLog</code>. This version of the write-ahead log was added in version 1.6.0 of Apache NiFi and was developed +in order to address an issue that exists in the older implementation. In the event of power loss or an operating system crash, the old implementation was susceptible to recovering FlowFiles +incorrectly. This could potentially lead to the wrong attributes or content being assigned to a FlowFile upon restart, following the power loss or OS crash. However, one can still choose to opt into +using the previous implementation and accept that risk, if desired (for example, if the new implementation were to exhibit some unexpected error). +To do so, set the value of this property to <code>org.wali.MinimalLockingWriteAheadLog</code>. +If the value of this property is changed, upon restart, NiFi will still recover the records written using the previously configured repository and delete the files written by the previously configured +implementation.</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.flowfile.repository.directory*</p></td> @@ -6394,6 +6413,17 @@ Providing three total network interfaces <td class="tableblock halign-left valign-top"><p class="tableblock">nifi.web.max.header.size</p></td> <td class="tableblock halign-left valign-top"><p class="tableblock">The maximum size allowed for request and response headers. The default value is 16 KB.</p></td> </tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.web.proxy.host</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">A comma separated list of allowed HTTP Host header values to consider when NiFi is running securely and will be receiving requests to a different host[:port] than it is bound to. +For example, when running in a Docker container or behind a proxy (e.g. localhost:18443, proxyhost:443). By default, this value is blank meaning NiFi should only allow requests sent to the +host[:port] that NiFi is bound to.</p></td> +</tr> +<tr> +<td class="tableblock halign-left valign-top"><p class="tableblock">nifi.web.proxy.context.path</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">A comma separated list of allowed HTTP X-ProxyContextPath or X-Forwarded-Context header values to consider. By default, this value is +blank meaning all requests containing a proxy context path are rejected. Configuring this property would allow requests where the proxy path is contained in this listing.</p></td> +</tr> </tbody> </table> </div> @@ -6799,7 +6829,7 @@ that is specified.</p></td> </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-01-12 13:36:55 -05:00 +Last updated 2018-04-03 08:25:54 -07:00 </div> </div> </body>
Modified: nifi/site/trunk/docs/nifi-docs/html/developer-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/developer-guide.html?rev=1828578&r1=1828577&r2=1828578&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/developer-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/developer-guide.html Sat Apr 7 00:33:22 2018 @@ -1,20 +1,4 @@ -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - --> - <!DOCTYPE html> +<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -2914,7 +2898,7 @@ needed for any number of operational con </ul> </div> <div class="paragraph"> -<p>Custom UIs can be created to provide configuration options beyond the standard property/value tables available in most processor settings. Examples of processors with Custom UIs are <a href="https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-update-attribute-bundle";>UpdateAttribute</a> and <a href="https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-standard-bundle";>JoltTransformJSON</a>.</p> +<p>Custom UIs can be created to provide configuration options beyond the standard property/value tables available in most processor settings. Examples of processors with Custom UIs are <a href="https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-update-attribute-bundle"; target="_blank">UpdateAttribute</a> and <a href="https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-standard-bundle"; target="_blank">JoltTransformJSON</a>.</p> </div> <div class="paragraph"> <p>Content Viewers can be created to extend the types of data that can be viewed within NiFi. NiFi comes with NARs in the lib directory which contain content viewers for data types such as csv, xml, avro, json (standard-nar) and image types such as png, jpeg and gif (media-nar).</p> @@ -3580,7 +3564,7 @@ logged to help avoid this bad practice.< </div> <div class="listingblock"> <div class="content"> -<pre class="highlight"><code class="language-java" data-lang="java"> @DeprecationNotice(alternatives = {ListenSyslog.class}, classNames = {"org.apache.nifi.processors.standard.ListenRELP"}, reason = "Technologyhas been superseded", ) +<pre class="highlight"><code class="language-java" data-lang="java"> @DeprecationNotice(alternatives = {ListenSyslog.class}, classNames = {"org.apache.nifi.processors.standard.ListenRELP"}, reason = "Technology has been superseded", ) public class ListenOldProtocol extends AbstractProcessor {</code></pre> </div> </div> @@ -3606,20 +3590,20 @@ used to provide a user interface. We dep among others. We make use of Apache Maven for our builds and Git for our version control system.</p> </div> <div class="paragraph"> -<p>Documentation is created in <a href="http://asciidoctor.org";>AsciiDoc</a>.</p> +<p>Documentation is created in <a href="http://asciidoctor.org"; target="_blank">AsciiDoc</a>.</p> </div> </div> <div class="sect2"> <h3 id="where-to-start"><a class="anchor" href="developer-guide.html#where-to-start"></a>Where to Start?</h3> <div class="paragraph"> -<p><a href="http://issues.apache.org/jira/browse/NIFI";>NiFi’s JIRA page</a> can be used to find tickets that are tagged as "beginner", +<p><a href="http://issues.apache.org/jira/browse/NIFI"; target="_blank">NiFi’s JIRA page</a> can be used to find tickets that are tagged as "beginner", or you can dig into any of the tickets for creating Processors. Processors should be self-contained and not rely on other outside components (except for Controller Services), so they make for excellent starting points for new NiFi developers to get started. This exposes the developer to the NiFi API and is the most extensible part of the dataflow system.</p> </div> <div class="paragraph"> <p>System-level and overview documentation is located in '<code checkout location>/nifi/nifi-docs/src/main/asciidoc'. -Tools available to facilitate documentation generation are available at <a href="http://asciidoctor.org/docs/editing-asciidoc-with-live-preview/";>Editing AsciiDoc with Live Preview</a>.</p> +Tools available to facilitate documentation generation are available at <a href="http://asciidoctor.org/docs/editing-asciidoc-with-live-preview/"; target="_blank">Editing AsciiDoc with Live Preview</a>.</p> </div> </div> <div class="sect2"> @@ -3648,7 +3632,7 @@ worry about bothering us. Just ping the </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-01-12 13:36:55 -05:00 +Last updated 2018-04-03 08:25:54 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html?rev=1828578&r1=1828577&r2=1828578&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/expression-language-guide.html Sat Apr 7 00:33:22 2018 @@ -1,20 +1,4 @@ -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - --> - <!DOCTYPE html> +<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -825,7 +809,7 @@ the number of milliseconds since midnigh <p><strong>Decimal</strong>: A Decimal is a numeric value that can support decimals and larger values with minimal loss of precision. More precisely it is a double-precision 64-bit IEEE 754 floating point. Due to this minimal loss of precision this data type should not be used for very precise values, such as currency. For more documentation on the range of values stored in this data type -refer to this <a href="https://docs.oracle.com/javase/specs/jls/se7/html/jls-4.html#jls-4.2.3";>link</a>. The following are some examples of the forms of +refer to this <a href="https://docs.oracle.com/javase/specs/jls/se7/html/jls-4.html#jls-4.2.3"; target="_blank">link</a>. The following are some examples of the forms of literal decimals that are supported in expression language (the "E" can also be lower-case):</p> <div class="ulist"> <ul> @@ -2912,14 +2896,7 @@ Divide. This is to preserve backwards co <div class="sect2 function"> <h3 id="toradix"><a class="anchor" href="expression-language-guide.html#toradix"></a>toRadix</h3> <div class="paragraph"> -<p><strong>Description</strong>: [.description]#Converts the Subject from a Base 10 number to a different Radix (or number base). An optional - second argument can be used to indicate the minimum number of characters to be used. If the converted value - has fewer than this number of characters, the number will be padded with leading zeroes.</p> -</div> -<div class="literalblock"> -<div class="content"> -<pre>If a decimal is passed as the subject, it will first be converted to a whole number and then processed.#</pre> -</div> +<p><strong>Description</strong>: <span class="description">Converts the Subject from a Base 10 number to a different Radix (or number base). An optional second argument can be used to indicate the minimum number of characters to be used. If the converted value has fewer than this number of characters, the number will be padded with leading zeroes. If a decimal is passed as the subject, it will first be converted to a whole number and then processed.</span></p> </div> <div class="paragraph"> <p><strong>Subject Type</strong>: <span class="subject">Number</span></p> @@ -2989,13 +2966,7 @@ Divide. This is to preserve backwards co <div class="sect2 function"> <h3 id="fromradix"><a class="anchor" href="expression-language-guide.html#fromradix"></a>fromRadix</h3> <div class="paragraph"> -<p><strong>Description</strong>: [.description]#Converts the Subject from a specified Radix (or number base) to a base ten whole number. The subject will converted as is, without interpretation, and all characters -must be valid for the base being converted from. For example converting "0xFF" from hex will not work due to "x" being a invalid hex character.</p> -</div> -<div class="literalblock"> -<div class="content"> -<pre>If a decimal is passed as the subject, it will first be converted to a whole number and then processed.#</pre> -</div> +<p><strong>Description</strong>: <span class="description">Converts the Subject from a specified Radix (or number base) to a base ten whole number. The subject will converted as is, without interpretation, and all characters must be valid for the base being converted from. For example converting "0xFF" from hex will not work due to "x" being a invalid hex character. If a decimal is passed as the subject, it will first be converted to a whole number and then processed.</span></p> </div> <div class="paragraph"> <p><strong>Subject Type</strong>: <span class="subject">String</span></p> @@ -3064,7 +3035,7 @@ must be valid for the base being convert <div class="sect2 function"> <h3 id="math"><a class="anchor" href="expression-language-guide.html#math"></a>math</h3> <div class="paragraph"> -<p><strong>Description</strong>: <span class="description">ADVANCED FEATURE. This expression is designed to be used by advanced users only. It utilizes Java Reflection to run arbitrary java.lang.Math static methods. The exact API will depend on the version of Java you are running. The Java 8 API can be found here: <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html"; class="bare">https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html</a> +<p><strong>Description</strong>: <span class="description">ADVANCED FEATURE. This expression is designed to be used by advanced users only. It utilizes Java Reflection to run arbitrary java.lang.Math static methods. The exact API will depend on the version of Java you are running. The Java 8 API can be found here: <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html"; target="_blank">https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html</a> <br> In order to run the correct method, the parameter types must be correct. The Expression Language "Number" (whole number) type is interpreted as a Java "long". The "Decimal" type is interpreted as a Java "double". Running the desired method may require calling "toNumber()" or "toDecimal()" in order to "cast" the value to the desired type. This also is important to remember when cascading "math()" calls since the return type depends on the method that was run.</span></p> </div> @@ -3931,7 +3902,7 @@ an embedded Expression, though it does n </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-01-12 13:36:55 -05:00 +Last updated 2018-04-03 08:25:54 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/getting-started.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/getting-started.html?rev=1828578&r1=1828577&r2=1828578&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/getting-started.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/getting-started.html Sat Apr 7 00:33:22 2018 @@ -1,20 +1,4 @@ -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - --> - <!DOCTYPE html> +<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -565,7 +549,7 @@ dataflows.</p> <h2 id="downloading-and-installing-nifi"><a class="anchor" href="getting-started.html#downloading-and-installing-nifi"></a>Downloading and Installing NiFi</h2> <div class="sectionbody"> <div class="paragraph"> -<p>NiFi can be downloaded from the <a href="http://nifi.apache.org/download.html";>NiFi Downloads Page</a>. There are two packaging options +<p>NiFi can be downloaded from the <a href="http://nifi.apache.org/download.html"; target="_blank">NiFi Downloads Page</a>. There are two packaging options available: a "tarball" that is tailored more to Linux and a zip file that is more applicable for Windows users. Mac OS X users may also use the tarball or can install via Homebrew.</p> </div> @@ -634,7 +618,7 @@ and <code>sudo service nifi stop</code>. <div class="sectionbody"> <div class="paragraph"> <p>Now that NiFi has been started, we can bring up the User Interface (UI) in order to create and monitor our dataflow. -To get started, open a web browser and navigate to <code><a href="http://localhost:8080/nifi"; class="bare">http://localhost:8080/nifi</a></code>. The port can be changed by +To get started, open a web browser and navigate to <a href="http://localhost:8080/nifi"; target="_blank"><code>http://localhost:8080/nifi</code></a>. The port can be changed by editing the <code>nifi.properties</code> file in the NiFi <code>conf</code> directory, but the default port is 8080.</p> </div> <div class="paragraph"> @@ -1600,18 +1584,18 @@ comprehensive overview of the different NiFi components and common Processor idioms to help aid in understanding the logic behind many of the existing NiFi components.</p> </li> <li> -<p><a href="https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide";>Contributor’s Guide</a> - A guide for explaining how to contribute +<p><a href="https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide"; target="_blank">Contributor’s Guide</a> - A guide for explaining how to contribute work back to the Apache NiFi community so that others can make use of it.</p> </li> </ul> </div> <div class="paragraph"> <p>Several blog postings have also been added to the Apache NiFi blog site: -<a href="https://blogs.apache.org/nifi/";>https://blogs.apache.org/nifi/</a></p> +<a href="https://blogs.apache.org/nifi/"; target="_blank">https://blogs.apache.org/nifi/</a></p> </div> <div class="paragraph"> <p>In addition to the blog and guides provided here, you can browse the different -<a href="https://nifi.apache.org/mailing_lists.html";>NiFi Mailing Lists</a> or send an e-mail to one of the mailing lists at +<a href="https://nifi.apache.org/mailing_lists.html"; target="_blank">NiFi Mailing Lists</a> or send an e-mail to one of the mailing lists at <a href="mailto:us...@nifi.apache.org";>us...@nifi.apache.org</a> or <a href="mailto:d...@nifi.apache.org";>d...@nifi.apache.org</a>.</p> </div> @@ -1623,7 +1607,7 @@ work back to the Apache NiFi community s </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-01-12 13:36:55 -05:00 +Last updated 2018-04-03 08:25:54 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/nifi-in-depth.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/nifi-in-depth.html?rev=1828578&r1=1828577&r2=1828578&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/nifi-in-depth.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/nifi-in-depth.html Sat Apr 7 00:33:22 2018 @@ -1,20 +1,4 @@ -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - --> - <!DOCTYPE html> +<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -542,7 +526,7 @@ body.book #toc,body.book #preamble,body. <p>If the Node was in the middle of writing content when it went down, nothing is corrupted, thanks to the Copy On Write (mentioned below) and Immutability (mentioned above) paradigms. Since FlowFile transactions never modify the original content (pointed to by the content pointer), the original is safe. When NiFi goes down, the write claim for the change is orphaned and then cleaned up by the background garbage collection. This provides a ârollbackâ to the last known stable state.</p> </div> <div class="paragraph"> -<p>The Node then restores its state from the FlowFile. For a more in-depth, step-by-step explanation of the process, see this link: <a href="https://cwiki.apache.org/confluence/display/NIFI/NiFi%27s+Write-Ahead+Log+Implementation"; class="bare">https://cwiki.apache.org/confluence/display/NIFI/NiFi%27s+Write-Ahead+Log+Implementation</a> .</p> +<p>The Node then restores its state from the FlowFile. For a more in-depth, step-by-step explanation of the process, see <a href="https://cwiki.apache.org/confluence/display/NIFI/NiFi%27s+Write-Ahead+Log+Implementation"; target="_blank">NiFi’s Write-Ahead Log Implementation</a>.</p> </div> <div class="paragraph"> <p>This setup, in terms of transactional units of work, allows NiFi to be very resilient in the face of adversity, ensuring that even if NiFi is suddenly killed, it can pick back up without any loss of data.</p> @@ -606,7 +590,7 @@ Since provenance events are snapshots of </table> </div> <div class="paragraph"> -<p>For a look at the design decisions behind the Provenance Repository check out this link: <a href="https://cwiki.apache.org/confluence/display/NIFI/Persistent+Provenance+Repository+Design"; class="bare">https://cwiki.apache.org/confluence/display/NIFI/Persistent+Provenance+Repository+Design</a></p> +<p>For a look at the design decisions behind the Provenance Repository check out <a href="https://cwiki.apache.org/confluence/display/NIFI/Persistent+Provenance+Repository+Design"; target="_blank">Persistent Provenance Repository Design</a>.</p> </div> <div class="sect3"> <h4 id="deeper-view-provenance-log-files"><a class="anchor" href="nifi-in-depth.html#deeper-view-provenance-log-files"></a>Deeper View: Provenance Log Files</h4> @@ -645,7 +629,7 @@ Since provenance events are snapshots of <h2 id="life-of-a-flowfile"><a class="anchor" href="nifi-in-depth.html#life-of-a-flowfile"></a>Life of a FlowFile</h2> <div class="sectionbody"> <div class="paragraph"> -<p>To better understand how the repos interact with one another, the underlying functionality of NiFi, and the life of a FlowFile; this next section will include examples of a FlowFile at different points in a real flow. The flow is a template called "WebCrawler.xml" and is available here: <a href="https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates"; class="bare">https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates</a>.</p> +<p>To better understand how the repos interact with one another, the underlying functionality of NiFi, and the life of a FlowFile; this next section will include examples of a FlowFile at different points in a real flow. The flow is a template called "WebCrawler.xml" and is available here: <a href="https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates"; target="_blank">https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates</a>.</p> </div> <div class="paragraph"> <p>At a high level, this template reaches out to a seed URL configured in the GetHTTP processor, then analyzes the response using the RouteText processor to find instances of a keyword (in this case "nifi"), and potential URLs to hit. Then InvokeHTTP executes a HTTP Get request using the URLs found in the original seed web page. The response is routed based on the status code attribute and only 200-202 status codes are routed back to the original RouteText processor for analysis.</p> @@ -865,7 +849,7 @@ This section relies heavily on informati </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-01-12 13:36:55 -05:00 +Last updated 2018-04-03 08:25:54 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/overview.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/overview.html?rev=1828578&r1=1828577&r2=1828578&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/overview.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/overview.html Sat Apr 7 00:33:22 2018 @@ -1,20 +1,4 @@ -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - --> - <!DOCTYPE html> +<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -871,7 +855,7 @@ about loading, and to exchange data on s </dd> <dt class="hdlist1">Scale-up & down</dt> <dd> -<p>NiFi is also designed to scale-up and down in a very flexible manner. In terms of increasing throughput from the standpoint of the NiFi framework, it is possible to increase the number of concurrent tasks on the processor under the Scheduling tab when configuring. This allows more processes to execute simultaneously, providing greater throughput. On the other side of the spectrum, you can perfectly scale NiFi down to be suitable to run on edge devices where a small footprint is desired due to limited hardware resources. To specifically solve the first mile data collection challenge and edge use cases, you can find more details here: <a href="https://cwiki.apache.org/confluence/display/NIFI/MiNiFi"; class="bare">https://cwiki.apache.org/confluence/display/NIFI/MiNiFi</a> regarding a child project effort of Apache NiFi, MiNiFi (pronounced "minify", [min-uh-fahy]).</p> +<p>NiFi is also designed to scale-up and down in a very flexible manner. In terms of increasing throughput from the standpoint of the NiFi framework, it is possible to increase the number of concurrent tasks on the processor under the Scheduling tab when configuring. This allows more processes to execute simultaneously, providing greater throughput. On the other side of the spectrum, you can perfectly scale NiFi down to be suitable to run on edge devices where a small footprint is desired due to limited hardware resources. To specifically solve the first mile data collection challenge and edge use cases, you can find more details here: <a href="https://cwiki.apache.org/confluence/display/NIFI/MiNiFi"; target="_blank">https://cwiki.apache.org/confluence/display/NIFI/MiNiFi</a> regarding a child project effort of Apache NiFi, MiNiFi (pronounced "minify", [min-uh-fahy]).</p> </dd> </dl> </div> @@ -886,28 +870,29 @@ about loading, and to exchange data on s <div class="ulist bibliography"> <ul class="bibliography"> <li> -<p><a id="eip"></a>[eip] Gregor Hohpe. Enterprise Integration Patterns [online]. Retrieved: 27 Dec 2014, from: <a href="http://www.enterpriseintegrationpatterns.com/"; class="bare">http://www.enterpriseintegrationpatterns.com/</a></p> +<p><a id="eip"></a>[eip] Gregor Hohpe. Enterprise Integration Patterns [online]. Retrieved: 27 Dec 2014, from: <a href="http://www.enterpriseintegrationpatterns.com"; target="_blank">http://www.enterpriseintegrationpatterns.com</a></p> </li> <li> -<p><a id="soa"></a>[soa] Wikipedia. Service Oriented Architecture [online]. Retrieved: 27 Dec 2014, from: <a href="http://en.wikipedia.org/wiki/Service-oriented_architecture"; class="bare">http://en.wikipedia.org/wiki/Service-oriented_architecture</a></p> +<p><a id="soa"></a>[soa] Wikipedia. Service Oriented Architecture [online]. Retrieved: 27 Dec 2014, from: <a href="http://en.wikipedia.org/wiki/Service-oriented_architecture"; target="_blank">http://en.wikipedia.org/wiki/Service-oriented_architecture</a></p> </li> <li> -<p><a id="api"></a>[api] Eric Savitz. Welcome to the API Economy [online]. Forbes.com. Retrieved: 27 Dec 2014, from: <a href="http://www.forbes.com/sites/ciocentral/2012/08/29/welcome-to-the-api-economy/"; class="bare">http://www.forbes.com/sites/ciocentral/2012/08/29/welcome-to-the-api-economy/</a></p> +<p><a id="api"></a>[api] Eric Savitz. Welcome to the API Economy [online]. Forbes.com. Retrieved: 27 Dec 2014, from: <a href="http://www.forbes.com/sites/ciocentral/2012/08/29/welcome-to-the-api-economy"; target="_blank">http://www.forbes.com/sites/ciocentral/2012/08/29/welcome-to-the-api-economy</a></p> </li> <li> -<p><a id="api2"></a>[api2] Adam Duvander. The rise of the API economy and consumer-led ecosystems [online]. thenextweb.com. Retrieved: 27 Dec 2014, from: <a href="http://thenextweb.com/dd/2014/03/28/api-economy/"; class="bare">http://thenextweb.com/dd/2014/03/28/api-economy/</a></p> +<p><a id="api2"></a>[api2] Adam Duvander. The rise of the API economy and consumer-led ecosystems [online]. thenextweb.com. Retrieved: 27 Dec 2014, from: <a href="http://thenextweb.com/dd/2014/03/28/api-economy"; target="_blank">http://thenextweb.com/dd/2014/03/28/api-economy</a></p> </li> <li> -<p><a id="iot"></a>[iot] Wikipedia. Internet of Things [online]. Retrieved: 27 Dec 2014, from: <a href="http://en.wikipedia.org/wiki/Internet_of_Things"; class="bare">http://en.wikipedia.org/wiki/Internet_of_Things</a></p> +<p><a id="iot"></a>[iot] Wikipedia. Internet of Things [online]. Retrieved: 27 Dec 2014, from: <a href="http://en.wikipedia.org/wiki/Internet_of_Things"; target="_blank">http://en.wikipedia.org/wiki/Internet_of_Things</a></p> </li> <li> -<p><a id="bigdata"></a>[bigdata] Wikipedia. Big Data [online]. Retrieved: 27 Dec 2014, from: <a href="http://en.wikipedia.org/wiki/Big_data"; class="bare">http://en.wikipedia.org/wiki/Big_data</a></p> +<p><a id="bigdata"></a>[bigdata] Wikipedia. Big Data [online]. Retrieved: 27 Dec 2014, from: <a href="http://en.wikipedia.org/wiki/Big_data"; target="_blank">http://en.wikipedia.org/wiki/Big_data</a></p> </li> <li> -<p><a id="fbp"></a>[fbp] Wikipedia. Flow Based Programming [online]. Retrieved: 28 Dec 2014, from: <a href="http://en.wikipedia.org/wiki/Flow-based_programming#Concepts"; class="bare">http://en.wikipedia.org/wiki/Flow-based_programming#Concepts</a></p> +<p><a id="fbp"></a>[fbp] Wikipedia. Flow Based Programming [online]. Retrieved: 28 Dec 2014, from: <a href="http://en.wikipedia.org/wiki/Flow-based_programming#Concepts"; target="_blank">http://en.wikipedia.org/wiki/Flow-based_programming#Concepts</a></p> </li> <li> -<p><a id="seda"></a>[seda] Matt Welsh. Harvard. SEDA: An Architecture for Highly Concurrent Server Applications [online]. Retrieved: 28 Dec 2014, from: <a href="http://www.eecs.harvard.edu/~mdw/proj/seda/"; class="bare">http://www.eecs.harvard.edu/~mdw/proj/seda/</a></p> +<p><a id="seda"></a>[seda] Matt Welsh. Berkeley. SEDA: An Architecture for Well-Conditioned, +Scalable Internet Services [online]. Retrieved: 18 Jan 2018, from: <a href="http://www.mdw.la/papers/seda-sosp01.pdf"; target="_blank">http://www.mdw.la/papers/seda-sosp01.pdf</a></p> </li> </ul> </div> @@ -916,7 +901,7 @@ about loading, and to exchange data on s </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-01-12 13:36:55 -05:00 +Last updated 2018-04-03 08:25:54 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/record-path-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/record-path-guide.html?rev=1828578&r1=1828577&r2=1828578&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/record-path-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/record-path-guide.html Sat Apr 7 00:33:22 2018 @@ -1,20 +1,4 @@ -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - --> - <!DOCTYPE html> +<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -1605,7 +1589,7 @@ only of white space (spaces, tabs, carri </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-01-12 13:36:55 -05:00 +Last updated 2018-04-03 08:25:54 -07:00 </div> </div> </body> Modified: nifi/site/trunk/docs/nifi-docs/html/user-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/user-guide.html?rev=1828578&r1=1828577&r2=1828578&view=diff ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/user-guide.html (original) +++ nifi/site/trunk/docs/nifi-docs/html/user-guide.html Sat Apr 7 00:33:22 2018 @@ -1,20 +1,4 @@ -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - --> - <!DOCTYPE html> +<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> @@ -739,10 +723,16 @@ able to view and modify the dataflow. Fo </div> <div class="paragraph"> <p>The Components Toolbar runs across the top left portion of your screen. It consists of the components you can drag onto the -canvas to build your dataflow. Each component is described in more detail in <a href="http://localhost:8080/nifi-docs/html/building-dataflow.html";>Building a Dataflow</a>.</p> +canvas to build your dataflow. Each component is described in more detail in <a href="user-guide.html#building-dataflow">Building a DataFlow</a>.</p> </div> <div class="paragraph"> -<p>The Status Bar is under the Components Toolbar. The Status bar provides information about the number of threads that are currently active in the flow, the amount of data that currently exists in the flow, how many Remote Process Groups exist on the canvas in each state (Transmitting, Not Transmitting), how many Processors exist on the canvas in each state (Stopped, Running, Invalid, Disabled), how many versioned Process Groups exist on the canvas in each state (Up to date, Locally modified, Stale, Locally modified and stale, Sync failure) and the timestamp at which all of this information was last refreshed. Additionally, if the instance of NiFi is clustered, the Status bar shows how many nodes are in the cluster and how many are currently connected.</p> +<p>The Status Bar is under the Components Toolbar. The Status bar provides information about the number of threads that are +currently active in the flow, the amount of data that currently exists in the flow, how many Remote Process Groups exist +on the canvas in each state (Transmitting, Not Transmitting), how many Processors exist on the canvas in each state +(Stopped, Running, Invalid, Disabled), how many versioned Process Groups exist on the canvas in each state (Up to date, +Locally modified, Stale, Locally modified and stale, Sync failure) and the timestamp at which all of this information +was last refreshed. Additionally, if the instance of NiFi is clustered, the Status bar shows how many nodes are in the +cluster and how many are currently connected.</p> </div> <div class="paragraph"> <p>The Operate Palette sits to the left-hand side of the screen. It consists of buttons that are @@ -813,7 +803,9 @@ is visible and available to you, dependi </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">access restricted components</p></td> -<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to create/modify restricted components assuming otherwise sufficient permissions</p></td> +<td class="tableblock halign-left valign-top"><p class="tableblock">Allows users to create/modify restricted components assuming other permissions are sufficient. The restricted +components may indicate which specific permissions are required. Permissions can be granted for specific restrictions or be granted regardless +of restrictions. If permission is granted regardless of restrictions, the user can create/modify all restricted components.</p></td> </tr> <tr> <td class="tableblock halign-left valign-top"><p class="tableblock">access all policies</p></td> @@ -963,14 +955,19 @@ Processors that allow us to ingest files <div class="paragraph"> <p>Restricted components will be marked with a <span class="image"><img src="images/restricted.png" alt="Restricted"></span> -icon next to their name. These are components that can be used to execute arbitrary unsanitized code provided by the operator -through the NiFi REST API/UI or can be used to obtain or alter data on the NiFi host system using the NiFi OS credentials. -These components could be used by an otherwise authorized NiFi user to go beyond the intended use of the application, escalate -privilege, or could expose data about the internals of the NiFi process or the host system. All of these capabilities should -be considered privileged, and admins should be aware of these capabilities and explicitly enable them for a subset of trusted users.</p> +icon next to their name. Hovering over the tooltip will display the specific restrictions this component requires. If the component +does not list any specific restrictions it will require access to restricted components regardless of restrictions. These are components +that can be used to execute arbitrary unsanitized code provided by the operator through the NiFi REST API/UI or can be used to obtain +or alter data on the NiFi host system using the NiFi OS credentials. These components could be used by an otherwise authorized NiFi +user to go beyond the intended use of the application, escalate privilege, or could expose data about the internals of the NiFi process +or the host system. All of these capabilities should be considered privileged, and admins should be aware of these capabilities and +explicitly enable them for a subset of trusted users.</p> </div> <div class="paragraph"> -<p>Before a user is allowed to create and modify restricted components they must be granted access to restricted components. For more information refer to +<p>Before a user is allowed to create and modify restricted components they must be granted access to restricted components. This can be +assigned regardless of restrictions. In this case, the user will have access to all restricted components. Alternatively, users can +be assigned access to specific restrictions. If the user has been granted access to all restrictions a component requires, they will +have access to that component assuming otherwise sufficient permissions. For more information refer to <a href="user-guide.html#UI-with-multi-tenant-authorization">Accessing the UI with Multi-Tenant Authorization</a>.</p> </div> <div class="paragraph"> @@ -1591,7 +1588,7 @@ example, 1L indicates the last Sunday of </ul> </div> <div class="paragraph"> -<p>For additional information and examples, see the <a href="http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/crontrigger.html";>Chron Trigger Tutorial</a> in the Quartz documentation.</p> +<p>For additional information and examples, see the <a href="http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/crontrigger.html"; target="_blank">Chron Trigger Tutorial</a> in the Quartz documentation.</p> </div> <div class="paragraph"> <p>Next, the Scheduling Tab provides a configuration option named <code>Concurrent tasks</code>. This controls how many threads the Processor @@ -3357,7 +3354,7 @@ will remove the selection.</p> <h2 id="versioning_dataflow"><a class="anchor" href="user-guide.html#versioning_dataflow"></a>Versioning a DataFlow</h2> <div class="sectionbody"> <div class="paragraph"> -<p>When NiFi is connected to a NiFi Registry, dataflows can be version controlled on the process group level. For more information about NiFi Registry usage and configuration, see the documentation at <a href="https://nifi.apache.org/docs/nifi-registry-docs/index.html"; class="bare">https://nifi.apache.org/docs/nifi-registry-docs/index.html</a>.</p> +<p>When NiFi is connected to a NiFi Registry, dataflows can be version controlled on the process group level. For more information about NiFi Registry usage and configuration, see the documentation at <a href="https://nifi.apache.org/docs/nifi-registry-docs/index.html"; target="_blank">https://nifi.apache.org/docs/nifi-registry-docs/index.html</a>.</p> </div> <div class="sect2"> <h3 id="connecting-to-a-nifi-registry"><a class="anchor" href="user-guide.html#connecting-to-a-nifi-registry"></a>Connecting to a NiFi Registry</h3> @@ -3400,7 +3397,7 @@ will remove the selection.</p> <i class="fa icon-note" title="Note"></i> </td> <td class="content"> -Versioned flows are stored and organized in registry buckets. Bucket Policies and Special Privileges configured by the registry administrator determine which buckets a user can import versioned flows from and which buckets a user can save versioned flows to. Information on Bucket Policies and Special Privileges can be found in the NiFi Registry User Guide (<a href="https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html"; class="bare">https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html</a>). +Versioned flows are stored and organized in registry buckets. Bucket Policies and Special Privileges configured by the registry administrator determine which buckets a user can import versioned flows from and which buckets a user can save versioned flows to. Information on Bucket Policies and Special Privileges can be found in the NiFi Registry User Guide (<a href="https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html"; target="_blank">https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html</a>). </td> </tr> </table> @@ -3847,7 +3844,7 @@ filter the templates to see only those o <div class="sect3"> <h4 id="Export_Template"><a class="anchor" href="user-guide.html#Export_Template"></a>Exporting a Template</h4> <div class="paragraph"> -<p>Once a Template has been created, it can be shared with others in the Template Management page (see <a href="user-guide.html#Manage_Templates">Managing Templates</a>). +<p>Once a Template has been created, it can be shared with others in the Template Management page. To export a Template, locate the Template in the table. The Filter in the top-right corner can be used to help find the appropriate Template if several are available. Then click the Export or Download button ( <span class="image"><img src="images/iconExport.png" alt="Export"></span> @@ -3858,8 +3855,7 @@ into other instances of NiFi (see <a hre <div class="sect3"> <h4 id="removing-a-template"><a class="anchor" href="user-guide.html#removing-a-template"></a>Removing a Template</h4> <div class="paragraph"> -<p>Once it is decided that a Template is no longer needed, it can be easily removed from the Template Management page -(see <a href="user-guide.html#Manage_Templates">Managing Templates</a>). To delete a Template, locate it in the table (the Filter in the top-right corner +<p>Once it is decided that a Template is no longer needed, it can be easily removed from the Template Management page. To delete a Template, locate it in the table (the Filter in the top-right corner may be used to find the appropriate Template if several are available) and click the Delete button ( <span class="image"><img src="images/iconDelete.png" alt="Delete"></span> ). This will prompt for confirmation. After confirming the deletion, the Template will be removed from this table @@ -4271,7 +4267,7 @@ key5=c6FzfnKm7UR7xqI2NFpZ+fEKBfSU7+1NvRw <div class="ulist"> <ul> <li> -<p>If a user has an existing repository (<code>WriteAheadProvenanceRepository</code> only — <strong>not</strong> <code>PersistentProvenanceRepository</code>) that is not encrypted and switches their configuration to use an encrypted repository, the application writes an error to the log but starts up. However, previous events are not accessible through the provenance query interface and new events will overwrite the existing events. The same behavior occurs if a user switches from an encrypted repository to an unencrypted repository. Automatic roll-over is a future effort (<a href="https://issues.apache.org/jira/browse/NIFI-3722";>NIFI-3722</a>) but NiFi is not intended for long-term storage of provenance events so the impact should be minimal. There are two scenarios for roll-over:</p> +<p>If a user has an existing repository (<code>WriteAheadProvenanceRepository</code> only — <strong>not</strong> <code>PersistentProvenanceRepository</code>) that is not encrypted and switches their configuration to use an encrypted repository, the application writes an error to the log but starts up. However, previous events are not accessible through the provenance query interface and new events will overwrite the existing events. The same behavior occurs if a user switches from an encrypted repository to an unencrypted repository. Automatic roll-over is a future effort (<a href="https://issues.apache.org/jira/browse/NIFI-3722"; target="_blank">NIFI-3722</a>) but NiFi is not intended for long-term storage of provenance events so the impact should be minimal. There are two scenarios for roll-over:</p> <div class="ulist"> <ul> <li> @@ -4284,7 +4280,7 @@ key5=c6FzfnKm7UR7xqI2NFpZ+fEKBfSU7+1NvRw </div> </li> <li> -<p>There is also a future effort to provide a standalone tool in NiFi Toolkit to encrypt/decrypt an existing provenance repository to make the transition easier. The translation process could take a long time depending on the size of the existing repository, and being able to perform this task outside of application startup would be valuable (<a href="https://issues.apache.org/jira/browse/NIFI-3723";>NIFI-3723</a>).</p> +<p>There is also a future effort to provide a standalone tool in NiFi Toolkit to encrypt/decrypt an existing provenance repository to make the transition easier. The translation process could take a long time depending on the size of the existing repository, and being able to perform this task outside of application startup would be valuable (<a href="https://issues.apache.org/jira/browse/NIFI-3723"; target="_blank">NIFI-3723</a>).</p> </li> </ul> </div> @@ -4323,7 +4319,7 @@ the <a href="administration-guide.html"> </div> <div id="footer"> <div id="footer-text"> -Last updated 2018-01-12 13:36:55 -05:00 +Last updated 2018-04-03 08:25:54 -07:00 </div> </div> </body>
