Added: nifi/site/trunk/docs/nifi-docs/html/toolkit-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/toolkit-guide.html?rev=1854109&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/toolkit-guide.html (added) +++ nifi/site/trunk/docs/nifi-docs/html/toolkit-guide.html Fri Feb 22 01:03:44 2019 @@ -0,0 +1,2927 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> +<html lang="en"> +<head> +<meta charset="UTF-8"> +<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]--> +<meta name="viewport" content="width=device-width, initial-scale=1.0"> +<meta name="generator" content="Asciidoctor 1.5.2"> +<meta name="author" content="Apache NiFi Team"> +<title>Apache NiFi Toolkit Guide</title> +<style> +/* Asciidoctor default stylesheet | MIT License | http://asciidoctor.org */ +/* Copyright (C) 2012-2015 Dan Allen, Ryan Waldron and the Asciidoctor Project + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. */ +/* Remove the comments around the @import statement below when using this as a custom stylesheet */ +@import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400";; +article,aside,details,figcaption,figure,footer,header,hgroup,main,nav,section,summary{display:block} +audio,canvas,video{display:inline-block} +audio:not([controls]){display:none;height:0} +[hidden],template{display:none} +script{display:none!important} +html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%} +body{margin:0} +a{background:transparent} +a:focus{outline:thin dotted} +a:active,a:hover{outline:0} +h1{font-size:2em;margin:.67em 0} +abbr[title]{border-bottom:1px dotted} +b,strong{font-weight:bold} +dfn{font-style:italic} +hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0} +mark{background:#ff0;color:#000} +code,kbd,pre,samp{font-family:monospace;font-size:1em} +pre{white-space:pre-wrap} +q{quotes:"\201C" "\201D" "\2018" "\2019"} +small{font-size:80%} +sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline} +sup{top:-.5em} +sub{bottom:-.25em} +img{border:0} +svg:not(:root){overflow:hidden} +figure{margin:0} +fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em} +legend{border:0;padding:0} +button,input,select,textarea{font-family:inherit;font-size:100%;margin:0} +button,input{line-height:normal} +button,select{text-transform:none} +button,html input[type="button"],input[type="reset"],input[type="submit"]{-webkit-appearance:button;cursor:pointer} +button[disabled],html input[disabled]{cursor:default} +input[type="checkbox"],input[type="radio"]{box-sizing:border-box;padding:0} +input[type="search"]{-webkit-appearance:textfield;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;box-sizing:content-box} +input[type="search"]::-webkit-search-cancel-button,input[type="search"]::-webkit-search-decoration{-webkit-appearance:none} +button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0} +textarea{overflow:auto;vertical-align:top} +table{border-collapse:collapse;border-spacing:0} +*,*:before,*:after{-moz-box-sizing:border-box;-webkit-box-sizing:border-box;box-sizing:border-box} +html,body{font-size:100%} +body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;font-weight:400;font-style:normal;line-height:1;position:relative;cursor:auto} +a:hover{cursor:pointer} +img,object,embed{max-width:100%;height:auto} +object,embed{height:100%} +img{-ms-interpolation-mode:bicubic} +#map_canvas img,#map_canvas embed,#map_canvas object,.map_canvas img,.map_canvas embed,.map_canvas object{max-width:none!important} +.left{float:left!important} +.right{float:right!important} +.text-left{text-align:left!important} +.text-right{text-align:right!important} +.text-center{text-align:center!important} +.text-justify{text-align:justify!important} +.hide{display:none} +.antialiased,body{-webkit-font-smoothing:antialiased} +img{display:inline-block;vertical-align:middle} +textarea{height:auto;min-height:50px} +select{width:100%} +p.lead,.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{font-size:1.21875em;line-height:1.6} +.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em} +div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0;direction:ltr} +a{color:#2156a5;text-decoration:underline;line-height:inherit} +a:hover,a:focus{color:#1d4b8f} +a img{border:none} +p{font-family:inherit;font-weight:400;font-size:1em;line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility} +p aside{font-size:.875em;line-height:1.35;font-style:italic} +h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em} +h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0} +h1{font-size:2.125em} +h2{font-size:1.6875em} +h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em} +h4,h5{font-size:1.125em} +h6{font-size:1em} +hr{border:solid #ddddd8;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em;height:0} +em,i{font-style:italic;line-height:inherit} +strong,b{font-weight:bold;line-height:inherit} +small{font-size:60%;line-height:inherit} +code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9);padding-right: 1px;} +ul,ol,dl{font-size:1em;line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit} +ul,ol,ul.no-bullet,ol.no-bullet{margin-left:1.5em} +ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0;font-size:1em} +ul.square li ul,ul.circle li ul,ul.disc li ul{list-style:inherit} +ul.square{list-style-type:square} +ul.circle{list-style-type:circle} +ul.disc{list-style-type:disc} +ul.no-bullet{list-style:none} +ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0} +dl dt{margin-bottom:.3125em;font-weight:bold} +dl dd{margin-bottom:1.25em} +abbr,acronym{text-transform:uppercase;font-size:90%;color:rgba(0,0,0,.8);border-bottom:1px dotted #ddd;cursor:help} +abbr{text-transform:none} +blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd} +blockquote cite{display:block;font-size:.9375em;color:rgba(0,0,0,.6)} +blockquote cite:before{content:"\2014 \0020"} +blockquote cite a,blockquote cite a:visited{color:rgba(0,0,0,.6)} +blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)} +@media only screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2} +h1{font-size:2.75em} +h2{font-size:2.3125em} +h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em} +h4{font-size:1.4375em}}table{background:#fff;margin-bottom:1.25em;border:solid 1px #dedede} +table thead,table tfoot{background:#f7f8f7;font-weight:bold} +table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left} +table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)} +table tr.even,table tr.alt,table tr:nth-of-type(even){background:#f8f8f7} +table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{display:table-cell;line-height:1.6} +h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em} +h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400} +.clearfix:before,.clearfix:after,.float-group:before,.float-group:after{content:" ";display:table} +.clearfix:after,.float-group:after{clear:both} +*:not(pre)>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;word-spacing:-.15em;background-color:#f7f7f8;-webkit-border-radius:4px;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed} +pre,pre>code{line-height:1.45;color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;text-rendering:optimizeSpeed} +.keyseq{color:rgba(51,51,51,.8)} +kbd{display:inline-block;color:rgba(0,0,0,.8);font-size:.75em;line-height:1.4;background-color:#f7f7f7;border:1px solid #ccc;-webkit-border-radius:3px;border-radius:3px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em white inset;box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em #fff inset;margin:-.15em .15em 0 .15em;padding:.2em .6em .2em .5em;vertical-align:middle;white-space:nowrap} +.keyseq kbd:first-child{margin-left:0} +.keyseq kbd:last-child{margin-right:0} +.menuseq,.menu{color:rgba(0,0,0,.8)} +b.button:before,b.button:after{position:relative;top:-1px;font-weight:400} +b.button:before{content:"[";padding:0 3px 0 2px} +b.button:after{content:"]";padding:0 2px 0 3px} +p a>code:hover{color:rgba(0,0,0,.9)} +#header,#content,#footnotes,#footer{width:100%;margin-left:auto;margin-right:auto;margin-top:0;margin-bottom:0;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em} +#header:before,#header:after,#content:before,#content:after,#footnotes:before,#footnotes:after,#footer:before,#footer:after{content:" ";display:table} +#header:after,#content:after,#footnotes:after,#footer:after{clear:both} +#content{margin-top:1.25em} +#content:before{content:none} +#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0} +#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #ddddd8} +#header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #ddddd8;padding-bottom:8px} +#header .details{border-bottom:1px solid #ddddd8;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:-ms-flexbox;display:-webkit-flex;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap} +#header .details span:first-child{margin-left:-.125em} +#header .details span.email a{color:rgba(0,0,0,.85)} +#header .details br{display:none} +#header .details br+span:before{content:"\00a0\2013\00a0"} +#header .details br+span.author:before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)} +#header .details br+span#revremark:before{content:"\00a0|\00a0"} +#header #revnumber{text-transform:capitalize} +#header #revnumber:after{content:"\00a0"} +#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #ddddd8;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem} +#toc{border-bottom:1px solid #efefed;padding-bottom:.5em} +#toc>ul{margin-left:.125em} +#toc ul.sectlevel0>li>a{font-style:italic} +#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0} +#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none} +#toc a{text-decoration:none} +#toc a:active{text-decoration:underline} +#toctitle{color:#7a2518;font-size:1.2em} +@media only screen and (min-width:768px){#toctitle{font-size:1.375em} +body.toc2{padding-left:15em;padding-right:0} +#toc.toc2{margin-top:0!important;background-color:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #efefed;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto} +#toc.toc2 #toctitle{margin-top:0;font-size:1.2em} +#toc.toc2>ul{font-size:.9em;margin-bottom:0} +#toc.toc2 ul ul{margin-left:0;padding-left:1em} +#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em} +body.toc2.toc-right{padding-left:0;padding-right:15em} +body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #efefed;left:auto;right:0}}@media only screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0} +#toc.toc2{width:20em} +#toc.toc2 #toctitle{font-size:1.375em} +#toc.toc2>ul{font-size:.95em} +#toc.toc2 ul ul{padding-left:1.25em} +body.toc2.toc-right{padding-left:0;padding-right:20em}}#content #toc{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px} +#content #toc>:first-child{margin-top:0} +#content #toc>:last-child{margin-bottom:0} +#footer{max-width:100%;background-color:rgba(0,0,0,.8);padding:1.25em} +#footer-text{color:rgba(255,255,255,.8);line-height:1.44} +.sect1{padding-bottom:.625em} +@media only screen and (min-width:768px){.sect1{padding-bottom:1.25em}}.sect1+.sect1{border-top:1px solid #efefed} +#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400} +#content h1>a.anchor:before,h2>a.anchor:before,h3>a.anchor:before,#toctitle>a.anchor:before,.sidebarblock>.content>.title>a.anchor:before,h4>a.anchor:before,h5>a.anchor:before,h6>a.anchor:before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em} +#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible} +#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none} +#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221} +.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em} +.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic} +table.tableblock>caption.title{white-space:nowrap;overflow:visible;max-width:0} +.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{color:rgba(0,0,0,.85)} +table.tableblock #preamble>.sectionbody>.paragraph:first-of-type p{font-size:inherit} +.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%} +.admonitionblock>table td.icon{text-align:center;width:80px} +.admonitionblock>table td.icon img{max-width:none} +.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase} +.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #ddddd8;color:rgba(0,0,0,.6)} +.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0} +.exampleblock>.content{border-style:solid;border-width:1px;border-color:#e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;-webkit-border-radius:4px;border-radius:4px} +.exampleblock>.content>:first-child{margin-top:0} +.exampleblock>.content>:last-child{margin-bottom:0} +.sidebarblock{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px} +.sidebarblock>:first-child{margin-top:0} +.sidebarblock>:last-child{margin-bottom:0} +.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center} +.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0} +.literalblock pre,.listingblock pre:not(.highlight),.listingblock pre[class="highlight"],.listingblock pre[class^="highlight "],.listingblock pre.CodeRay,.listingblock pre.prettyprint{background:#f7f7f8} +.sidebarblock .literalblock pre,.sidebarblock .listingblock pre:not(.highlight),.sidebarblock .listingblock pre[class="highlight"],.sidebarblock .listingblock pre[class^="highlight "],.sidebarblock .listingblock pre.CodeRay,.sidebarblock .listingblock pre.prettyprint{background:#f2f1f1} +.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{-webkit-border-radius:4px;border-radius:4px;word-wrap:break-word;padding:1em;font-size:.8125em} +.literalblock pre.nowrap,.literalblock pre[class].nowrap,.listingblock pre.nowrap,.listingblock pre[class].nowrap{overflow-x:auto;white-space:pre;word-wrap:normal} +@media only screen and (min-width:768px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:.90625em}}@media only screen and (min-width:1280px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:1em}}.literalblock.output pre{color:#f7f7f8;background-color:rgba(0,0,0,.9)} +.listingblock pre.highlightjs{padding:0} +.listingblock pre.highlightjs>code{padding:1em;-webkit-border-radius:4px;border-radius:4px} +.listingblock pre.prettyprint{border-width:0} +.listingblock>.content{position:relative} +.listingblock code[data-lang]:before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:#999} +.listingblock:hover code[data-lang]:before{display:block} +.listingblock.terminal pre .command:before{content:attr(data-prompt);padding-right:.5em;color:#999} +.listingblock.terminal pre .command:not([data-prompt]):before{content:"$"} +table.pyhltable{border-collapse:separate;border:0;margin-bottom:0;background:none} +table.pyhltable td{vertical-align:top;padding-top:0;padding-bottom:0} +table.pyhltable td.code{padding-left:.75em;padding-right:0} +pre.pygments .lineno,table.pyhltable td:not(.code){color:#999;padding-left:0;padding-right:.5em;border-right:1px solid #ddddd8} +pre.pygments .lineno{display:inline-block;margin-right:.25em} +table.pyhltable .linenodiv{background:none!important;padding-right:0!important} +.quoteblock{margin:0 1em 1.25em 1.5em;display:table} +.quoteblock>.title{margin-left:-1.5em;margin-bottom:.75em} +.quoteblock blockquote,.quoteblock blockquote p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify} +.quoteblock blockquote{margin:0;padding:0;border:0} +.quoteblock blockquote:before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)} +.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0} +.quoteblock .attribution{margin-top:.5em;margin-right:.5ex;text-align:right} +.quoteblock .quoteblock{margin-left:0;margin-right:0;padding:.5em 0;border-left:3px solid rgba(0,0,0,.6)} +.quoteblock .quoteblock blockquote{padding:0 0 0 .75em} +.quoteblock .quoteblock blockquote:before{display:none} +.verseblock{margin:0 1em 1.25em 1em} +.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility} +.verseblock pre strong{font-weight:400} +.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex} +.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic} +.quoteblock .attribution br,.verseblock .attribution br{display:none} +.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.05em;color:rgba(0,0,0,.6)} +.quoteblock.abstract{margin:0 0 1.25em 0;display:block} +.quoteblock.abstract blockquote,.quoteblock.abstract blockquote p{text-align:left;word-spacing:0} +.quoteblock.abstract blockquote:before,.quoteblock.abstract blockquote p:first-of-type:before{display:none} +table.tableblock{max-width:100%;border-collapse:separate} +table.tableblock td>.paragraph:last-child p>p:last-child,table.tableblock th>p:last-child,table.tableblock td>p:last-child{margin-bottom:0} +table.spread{width:100%} +table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede} +table.grid-all th.tableblock,table.grid-all td.tableblock{border-width:0 1px 1px 0} +table.grid-all tfoot>tr>th.tableblock,table.grid-all tfoot>tr>td.tableblock{border-width:1px 1px 0 0} +table.grid-cols th.tableblock,table.grid-cols td.tableblock{border-width:0 1px 0 0} +table.grid-all *>tr>.tableblock:last-child,table.grid-cols *>tr>.tableblock:last-child{border-right-width:0} +table.grid-rows th.tableblock,table.grid-rows td.tableblock{border-width:0 0 1px 0} +table.grid-all tbody>tr:last-child>th.tableblock,table.grid-all tbody>tr:last-child>td.tableblock,table.grid-all thead:last-child>tr>th.tableblock,table.grid-rows tbody>tr:last-child>th.tableblock,table.grid-rows tbody>tr:last-child>td.tableblock,table.grid-rows thead:last-child>tr>th.tableblock{border-bottom-width:0} +table.grid-rows tfoot>tr>th.tableblock,table.grid-rows tfoot>tr>td.tableblock{border-width:1px 0 0 0} +table.frame-all{border-width:1px} +table.frame-sides{border-width:0 1px} +table.frame-topbot{border-width:1px 0} +th.halign-left,td.halign-left{text-align:left} +th.halign-right,td.halign-right{text-align:right} +th.halign-center,td.halign-center{text-align:center} +th.valign-top,td.valign-top{vertical-align:top} +th.valign-bottom,td.valign-bottom{vertical-align:bottom} +th.valign-middle,td.valign-middle{vertical-align:middle} +table thead th,table tfoot th{font-weight:bold} +tbody tr th{display:table-cell;line-height:1.6;background:#f7f8f7} +tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold} +p.tableblock>code:only-child{background:none;padding:0} +p.tableblock{font-size:1em} +td>div.verse{white-space:pre} +ol{margin-left:1.75em} +ul li ol{margin-left:1.5em} +dl dd{margin-left:1.125em} +dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0} +ol>li p,ul>li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em} +ul.unstyled,ol.unnumbered,ul.checklist,ul.none{list-style-type:none} +ul.unstyled,ol.unnumbered,ul.checklist{margin-left:.625em} +ul.checklist li>p:first-child>.fa-square-o:first-child,ul.checklist li>p:first-child>.fa-check-square-o:first-child{width:1em;font-size:.85em} +ul.checklist li>p:first-child>input[type="checkbox"]:first-child{width:1em;position:relative;top:1px} +ul.inline{margin:0 auto .625em auto;margin-left:-1.375em;margin-right:0;padding:0;list-style:none;overflow:hidden} +ul.inline>li{list-style:none;float:left;margin-left:1.375em;display:block} +ul.inline>li>*{display:block} +.unstyled dl dt{font-weight:400;font-style:normal} +ol.arabic{list-style-type:decimal} +ol.decimal{list-style-type:decimal-leading-zero} +ol.loweralpha{list-style-type:lower-alpha} +ol.upperalpha{list-style-type:upper-alpha} +ol.lowerroman{list-style-type:lower-roman} +ol.upperroman{list-style-type:upper-roman} +ol.lowergreek{list-style-type:lower-greek} +.hdlist>table,.colist>table{border:0;background:none} +.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none} +td.hdlist1{padding-right:.75em;font-weight:bold} +td.hdlist1,td.hdlist2{vertical-align:top} +.literalblock+.colist,.listingblock+.colist{margin-top:-.5em} +.colist>table tr>td:first-of-type{padding:0 .75em;line-height:1} +.colist>table tr>td:last-of-type{padding:.25em 0} +.thumb,.th{line-height:0;display:inline-block;border:solid 4px #fff;-webkit-box-shadow:0 0 0 1px #ddd;box-shadow:0 0 0 1px #ddd} +.imageblock.left,.imageblock[style*="float: left"]{margin:.25em .625em 1.25em 0} +.imageblock.right,.imageblock[style*="float: right"]{margin:.25em 0 1.25em .625em} +.imageblock>.title{margin-bottom:0} +.imageblock.thumb,.imageblock.th{border-width:6px} +.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em} +.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0} +.image.left{margin-right:.625em} +.image.right{margin-left:.625em} +a.image{text-decoration:none} +span.footnote,span.footnoteref{vertical-align:super;font-size:.875em} +span.footnote a,span.footnoteref a{text-decoration:none} +span.footnote a:active,span.footnoteref a:active{text-decoration:underline} +#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em} +#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em 0;border-width:1px 0 0 0} +#footnotes .footnote{padding:0 .375em;line-height:1.3;font-size:.875em;margin-left:1.2em;text-indent:-1.2em;margin-bottom:.2em} +#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none} +#footnotes .footnote:last-of-type{margin-bottom:0} +#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0} +.gist .file-data>table{border:0;background:#fff;width:100%;margin-bottom:0} +.gist .file-data>table td.line-data{width:99%} +div.unbreakable{page-break-inside:avoid} +.big{font-size:larger} +.small{font-size:smaller} +.underline{text-decoration:underline} +.overline{text-decoration:overline} +.line-through{text-decoration:line-through} +.aqua{color:#00bfbf} +.aqua-background{background-color:#00fafa} +.black{color:#000} +.black-background{background-color:#000} +.blue{color:#0000bf} +.blue-background{background-color:#0000fa} +.fuchsia{color:#bf00bf} +.fuchsia-background{background-color:#fa00fa} +.gray{color:#606060} +.gray-background{background-color:#7d7d7d} +.green{color:#006000} +.green-background{background-color:#007d00} +.lime{color:#00bf00} +.lime-background{background-color:#00fa00} +.maroon{color:#600000} +.maroon-background{background-color:#7d0000} +.navy{color:#000060} +.navy-background{background-color:#00007d} +.olive{color:#606000} +.olive-background{background-color:#7d7d00} +.purple{color:#600060} +.purple-background{background-color:#7d007d} +.red{color:#bf0000} +.red-background{background-color:#fa0000} +.silver{color:#909090} +.silver-background{background-color:#bcbcbc} +.teal{color:#006060} +.teal-background{background-color:#007d7d} +.white{color:#bfbfbf} +.white-background{background-color:#fafafa} +.yellow{color:#bfbf00} +.yellow-background{background-color:#fafa00} +span.icon>.fa{cursor:default} +.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default} +.admonitionblock td.icon .icon-note:before{content:"\f05a";color:#19407c} +.admonitionblock td.icon .icon-tip:before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111} +.admonitionblock td.icon .icon-warning:before{content:"\f071";color:#bf6900} +.admonitionblock td.icon .icon-caution:before{content:"\f06d";color:#bf3400} +.admonitionblock td.icon .icon-important:before{content:"\f06a";color:#bf0000} +.conum[data-value]{display:inline-block;color:#fff!important;background-color:rgba(0,0,0,.8);-webkit-border-radius:100px;border-radius:100px;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold} +.conum[data-value] *{color:#fff!important} +.conum[data-value]+b{display:none} +.conum[data-value]:after{content:attr(data-value)} +pre .conum[data-value]{position:relative;top:-.125em} +b.conum *{color:inherit!important} +.conum:not([data-value]):empty{display:none} +h1,h2{letter-spacing:-.01em} +dt,th.tableblock,td.content{text-rendering:optimizeLegibility} +p,td.content{letter-spacing:-.01em} +p strong,td.content strong{letter-spacing:-.005em} +p,blockquote,dt,td.content{font-size:1.0625rem} +p{margin-bottom:1.25rem} +.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em} +.exampleblock>.content{background-color:#fffef7;border-color:#e0e0dc;-webkit-box-shadow:0 1px 4px #e0e0dc;box-shadow:0 1px 4px #e0e0dc} +.print-only{display:none!important} +@media print{@page{margin:1.25cm .75cm} +*{-webkit-box-shadow:none!important;box-shadow:none!important;text-shadow:none!important} +a{color:inherit!important;text-decoration:underline!important} +a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important} +a[href^="http:"]:not(.bare):after,a[href^="https:"]:not(.bare):after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em} +abbr[title]:after{content:" (" attr(title) ")"} +pre,blockquote,tr,img{page-break-inside:avoid} +thead{display:table-header-group} +img{max-width:100%!important} +p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3} +h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid} +#toc,.sidebarblock,.exampleblock>.content{background:none!important} +#toc{border-bottom:1px solid #ddddd8!important;padding-bottom:0!important} +.sect1{padding-bottom:0!important} +.sect1+.sect1{border:0!important} +#header>h1:first-child{margin-top:1.25rem} +body.book #header{text-align:center} +body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em 0} +body.book #header .details{border:0!important;display:block;padding:0!important} +body.book #header .details span:first-child{margin-left:0!important} +body.book #header .details br{display:block} +body.book #header .details br+span:before{content:none!important} +body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important} +body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always} +.listingblock code[data-lang]:before{display:block} +#footer{background:none!important;padding:0 .9375em} +#footer-text{color:rgba(0,0,0,.6)!important;font-size:.9em} +.hide-on-print{display:none!important} +.print-only{display:block!important} +.hide-for-print{display:none!important} +.show-for-print{display:inherit!important}} +</style> +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.2.0/css/font-awesome.min.css";> +</head> +<body class="article"> +<div id="header"> +<h1>Apache NiFi Toolkit Guide</h1> +<div class="details"> +<span id="author" class="author">Apache NiFi Team</span><br> +<span id="email" class="email"><a href="mailto:[email protected]";>[email protected]</a></span><br> +</div> +<div id="toc" class="toc"> +<div id="toctitle">Table of Contents</div> +<ul class="sectlevel1"> +<li><a href="toolkit-guide.html#overview">Overview</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#prerequisites-for-running-in-a-secure-environment">Prerequisites for Running in a Secure Environment</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#nifi_CLI">NiFi CLI</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#usage">Usage</a></li> +<li><a href="toolkit-guide.html#property-argument-handling">Property/Argument Handling</a></li> +<li><a href="toolkit-guide.html#security-configuration">Security Configuration</a> +<ul class="sectlevel3"> +<li><a href="toolkit-guide.html#example-secure-nifi-registry-without-proxied-entity">Example - Secure NiFi Registry without Proxied-Entity</a></li> +<li><a href="toolkit-guide.html#example-secure-nifi-registry-with-proxied-entity">Example - Secure NiFi Registry with Proxied-Entity</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#interactive-usage">Interactive Usage</a></li> +<li><a href="toolkit-guide.html#output">Output</a></li> +<li><a href="toolkit-guide.html#back-referencing">Back-Referencing</a></li> +<li><a href="toolkit-guide.html#adding-commands">Adding Commands</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#encrypt_config_tool">Encrypt-Config Tool</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#usage-2">Usage</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#file-manager">File Manager</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#usage-3">Usage</a></li> +<li><a href="toolkit-guide.html#expected-behavior">Expected Behavior</a> +<ul class="sectlevel3"> +<li><a href="toolkit-guide.html#backup">Backup</a></li> +<li><a href="toolkit-guide.html#install">Install</a></li> +<li><a href="toolkit-guide.html#restore">Restore</a></li> +</ul> +</li> +</ul> +</li> +<li><a href="toolkit-guide.html#flow-analyzer">Flow Analyzer</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#usage-4">Usage</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#node-manager">Node Manager</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#usage-5">Usage</a></li> +<li><a href="toolkit-guide.html#expected-behavior-2">Expected Behavior</a> +<ul class="sectlevel3"> +<li><a href="toolkit-guide.html#status">Status</a></li> +<li><a href="toolkit-guide.html#disconnect">Disconnect</a></li> +<li><a href="toolkit-guide.html#connect">Connect</a></li> +<li><a href="toolkit-guide.html#remove">Remove</a></li> +</ul> +</li> +</ul> +</li> +<li><a href="toolkit-guide.html#notify">Notify</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#usage-6">Usage</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#s2s">S2S</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#usage-7">Usage</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#tls_toolkit">TLS Toolkit</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#wildcard_certificates">Wildcard Certificates</a> +<ul class="sectlevel3"> +<li><a href="toolkit-guide.html#potential-issues-with-wildcard-certificates">Potential issues with wildcard certificates</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#tls_operation_modes">Operation Modes</a> +<ul class="sectlevel3"> +<li><a href="toolkit-guide.html#standalone">Standalone</a></li> +<li><a href="toolkit-guide.html#client-server">Client/Server</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#tls_intermediate_ca">Using An Existing Intermediate Certificate Authority (CA)</a> +<ul class="sectlevel3"> +<li><a href="toolkit-guide.html#nifi-cert-pem">nifi-cert.pem</a></li> +<li><a href="toolkit-guide.html#nifi-key-key">nifi-key.key</a></li> +<li><a href="toolkit-guide.html#tls_external-signed_ca">Signing with Externally-signed CA Certificates</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#additional_certificate_commands">Additional Certificate Commands</a></li> +</ul> +</li> +<li><a href="toolkit-guide.html#zookeeper_migrator">ZooKeeper Migrator</a> +<ul class="sectlevel2"> +<li><a href="toolkit-guide.html#usage-11">Usage</a></li> +<li><a href="toolkit-guide.html#migrating-between-source-and-destination-zookeepers">Migrating Between Source and Destination ZooKeepers</a> +<ul class="sectlevel3"> +<li><a href="toolkit-guide.html#zookeeper-migration-steps">ZooKeeper Migration Steps</a></li> +</ul> +</li> +</ul> +</li> +</ul> +</div> +</div> +<div id="content"> +<div class="sect1"> +<h2 id="overview"><a class="anchor" href="toolkit-guide.html#overview"></a>Overview</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>CLI — The <code>cli</code> tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes.</p> +</li> +<li> +<p>Encrypt Config — The <code>encrypt-config</code> tool encrypts the sensitive keys in the <em>nifi.properties</em> file to facilitate the setup of a secure NiFi instance.</p> +</li> +<li> +<p>File Manager — The <code>file-manager</code> tool enables administrators to backup, install or restore a NiFi installation from backup.</p> +</li> +<li> +<p>Flow Analyzer — The <code>flow-analyzer</code> tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow.</p> +</li> +<li> +<p>Node Manager — The <code>node-manager</code> tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster.</p> +</li> +<li> +<p>Notify — The <code>notify</code> tool enables administrators to send bulletins to the NiFi UI.</p> +</li> +<li> +<p>S2S — The <code>s2s</code> tool enables administrators to send data into or out of NiFi flows over site-to-site.</p> +</li> +<li> +<p>TLS Toolkit — The <code>tls-toolkit</code> utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance.</p> +</li> +<li> +<p>ZooKeeper Migrator — The <code>zk-migrator</code> tool enables administrators to:</p> +<div class="ulist"> +<ul> +<li> +<p>move ZooKeeper information from one ZooKeeper cluster to another</p> +</li> +<li> +<p>migrate ZooKeeper node ownership</p> +</li> +</ul> +</div> +</li> +</ul> +</div> +<div class="paragraph"> +<p>The utilities are executed with scripts found in the <code>bin</code> folder of your NiFi Toolkit installation.</p> +</div> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +The NiFi Toolkit is downloaded separately from NiFi (see the <a href="https://nifi.apache.org/download.html"; target="_blank">Apache NiFi Downloads</a> page). +</td> +</tr> +</table> +</div> +<div class="sect2"> +<h3 id="prerequisites-for-running-in-a-secure-environment"><a class="anchor" href="toolkit-guide.html#prerequisites-for-running-in-a-secure-environment"></a>Prerequisites for Running in a Secure Environment</h3> +<div class="paragraph"> +<p>For secured nodes and clusters, two policies should be configured in advance:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Access the controller â A user that will have access to these utilities should be authorized in NiFi by creating an âaccess the controllerâ policy (<code>/controller</code>) with both view and modify rights</p> +</li> +<li> +<p>Proxy user request â If not previously set, nodeâs identity (the DN value of the nodeâs certificate) should be authorized to proxy requests on behalf of a user</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>When executing either the Notify or Node Manager tools in a secured environment the <code>proxyDN</code> flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored.</p> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="nifi_CLI"><a class="anchor" href="toolkit-guide.html#nifi_CLI"></a>NiFi CLI</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes.</p> +</div> +<div class="sect2"> +<h3 id="usage"><a class="anchor" href="toolkit-guide.html#usage"></a>Usage</h3> +<div class="paragraph"> +<p>The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell.</p> +</div> +<div class="paragraph"> +<p>To execute a single command:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh <command> <args></pre> +</div> +</div> +<div class="paragraph"> +<p>To launch the interactive shell:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh</pre> +</div> +</div> +<div class="paragraph"> +<p>To show help:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh -h</pre> +</div> +</div> +<div class="paragraph"> +<p>The following are available commands:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>demo quick-import +nifi current-user +nifi cluster-summary +nifi connect-node +nifi delete-node +nifi disconnect-node +nifi get-root-id +nifi get-node +nifi get-nodes +nifi offload-node +nifi list-reg-clients +nifi create-reg-client +nifi update-reg-client +nifi get-reg-client-id +nifi pg-import +nifi pg-start +nifi pg-stop +nifi pg-get-vars +nifi pg-set-var +nifi pg-get-version +nifi pg-change-version +nifi pg-get-all-versions +nifi pg-list +nifi pg-status +nifi pg-get-services +nifi pg-enable-services +nifi pg-disable-services +registry current-user +registry list-buckets +registry create-bucket +registry delete-bucket +registry list-flows +registry create-flow +registry delete-flow +registry list-flow-versions +registry export-flow-version +registry import-flow-version +registry sync-flow-versions +registry transfer-flow-version +session keys +session show +session get +session set +session remove +session clear +exit +help</pre> +</div> +</div> +<div class="paragraph"> +<p>To display extensive help for a specific command:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh <command> -h</pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="property-argument-handling"><a class="anchor" href="toolkit-guide.html#property-argument-handling"></a>Property/Argument Handling</h3> +<div class="paragraph"> +<p>Most commands will require specifying a baseUrl for the NiFi or NiFi Registry instance.</p> +</div> +<div class="paragraph"> +<p>An example command to list the buckets in a NiFi Registry instance would be the following:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh registry list-buckets -u http://localhost:18080</pre> +</div> +</div> +<div class="paragraph"> +<p>In order to avoid specifying the URL (and possibly other optional arguments for TLS) on every command, you can define a properties file containing the repetitive arguments.</p> +</div> +<div class="paragraph"> +<p>An example properties file for a local NiFi Registry instance would look like the following:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code> baseUrl=http://localhost:18080 + keystore= + keystoreType= + keystorePasswd= + keyPasswd= + truststore= + truststoreType= + truststorePasswd= + proxiedEntity=</code></pre> +</div> +</div> +<div class="paragraph"> +<p>This properties file can then be used on a command by specifying <code>-p</code>:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh registry list-buckets -p /path/to/local-nifi-registry.properties</pre> +</div> +</div> +<div class="paragraph"> +<p>You could then maintain a properties file for each environment you plan to interact with, such as Dev, QA, and Prod.</p> +</div> +<div class="paragraph"> +<p>In addition to specifying a properties file on each command, you can setup a default properties file to be used in the event that no properties file is specified.</p> +</div> +<div class="paragraph"> +<p>The default properties file is specified using the <code>session</code> concept, which persists to the users home directory in a file called <em>.nifi-cli.config</em>.</p> +</div> +<div class="paragraph"> +<p>An example of setting the default property files for NiFi would be the following:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh session set nifi.props /path/to/local-nifi.properties</pre> +</div> +</div> +<div class="paragraph"> +<p>An example for NiFi Registry would be the following:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh session set nifi.reg.props /path/to/local-nifi-registry.properties</pre> +</div> +</div> +<div class="paragraph"> +<p>This will write the above properties into the <em>.nifi-cli.config</em> in the user’s home directory and will allow commands to be executed without specifying a URL or properties file:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/cli.sh registry list-buckets</pre> +</div> +</div> +<div class="paragraph"> +<p>The above command will now use the <code>baseUrl</code> from <em>local-nifi-registry.properties</em>.</p> +</div> +<div class="paragraph"> +<p>The order of resolving an argument is the following:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>A direct argument overrides anything in a properties file or session</p> +</li> +<li> +<p>A properties file argument (<code>-p</code>) overrides the session</p> +</li> +<li> +<p>The session is used when nothing else is specified</p> +</li> +</ul> +</div> +</div> +<div class="sect2"> +<h3 id="security-configuration"><a class="anchor" href="toolkit-guide.html#security-configuration"></a>Security Configuration</h3> +<div class="paragraph"> +<p>If NiFi and NiFi Registry are secured, then commands executed from the CLI will need to make a TLS connection and authenticate as a user with permissions to perform the desired action.</p> +</div> +<div class="paragraph"> +<p>Currently the CLI supports authenticating with a client certificate and an optional proxied-entity. A common scenario would be running the CLI from one of the nodes where NiFi or NiFi Registry is installed, which allows the CLI to use the same keystore and truststore as the NiFi/NiFi Registry instance.</p> +</div> +<div class="paragraph"> +<p>The security configuration can be specified per-command, or in one of the properties files described in the previous section.</p> +</div> +<div class="paragraph"> +<p>The examples below are for NiFi Registry, but the same concept applies for NiFi commands.</p> +</div> +<div class="sect3"> +<h4 id="example-secure-nifi-registry-without-proxied-entity"><a class="anchor" href="toolkit-guide.html#example-secure-nifi-registry-without-proxied-entity"></a>Example - Secure NiFi Registry without Proxied-Entity</h4> +<div class="paragraph"> +<p>Assuming we have a keystore containing the certificate for "CN=user1, OU=NIFI", an example properties file would be the following:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code> baseUrl=https://localhost:18443 + keystore=/path/to/keystore.jks + keystoreType=JKS + keystorePasswd=changeme + keyPasswd=changeme + truststore=/path/to/truststore.jks + truststoreType=JKS + truststorePasswd=changeme</code></pre> +</div> +</div> +<div class="paragraph"> +<p>In this example, commands will be executed as "CN=user1, OU=NIFI". This user would need to be a user in NiFi Registry, and commands accessing buckets would be restricted to buckets this user has access to.</p> +</div> +</div> +<div class="sect3"> +<h4 id="example-secure-nifi-registry-with-proxied-entity"><a class="anchor" href="toolkit-guide.html#example-secure-nifi-registry-with-proxied-entity"></a>Example - Secure NiFi Registry with Proxied-Entity</h4> +<div class="paragraph"> +<p>Assuming we have access to the keystore of NiFi Registry itself, and that NiFi Registry is also configured to allow Kerberos or LDAP authentication, an example properties file would be the following:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code> baseUrl=https://localhost:18443 + keystore=/path/to/keystore.jks + keystoreType=JKS + keystorePasswd=changeme + keyPasswd=changeme + truststore=/path/to/truststore.jks + truststoreType=JKS + truststorePasswd=changeme + [email protected]</code></pre> +</div> +</div> +<div class="paragraph"> +<p>In this example, the certificate in <em>keystore.jks</em> would be for the NiFi Registry server, for example "CN=localhost, OU=NIFI". This identity would need to be defined as a user in NiFi Registry and given permissions to 'Proxy'.</p> +</div> +<div class="paragraph"> +<p>"CN=localhost, OU=NIFI" would be proxying commands to be executed as <a href="mailto:[email protected]";>[email protected]</a>.</p> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="interactive-usage"><a class="anchor" href="toolkit-guide.html#interactive-usage"></a>Interactive Usage</h3> +<div class="paragraph"> +<p>In interactive mode the tab key can be used to perform auto-completion.</p> +</div> +<div class="paragraph"> +<p>For example, typing tab at an empty prompt should display possible commands for the first argument:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>#> +demo exit help nifi registry session</pre> +</div> +</div> +<div class="paragraph"> +<p>Typing "nifi " and then a tab will show the sub-commands for NiFi:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>#> nifi +cluster-summary get-nodes pg-enable-services pg-set-var +connect-node get-reg-client-id pg-get-all-versions pg-start +create-reg-client get-root-id pg-get-services pg-status +current-user list-reg-clients pg-get-vars pg-stop +delete-node offload-node pg-get-version update-reg-client +disconnect-node pg-change-version pg-import +get-node pg-disable-services pg-list</pre> +</div> +</div> +<div class="paragraph"> +<p>Arguments that represent a path to a file, such as <code>-p</code> or when setting a properties file in the session, will auto-complete the path being typed:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>#> session set nifi.props /tmp/ +dir1/ dir2/ dir3/</pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="output"><a class="anchor" href="toolkit-guide.html#output"></a>Output</h3> +<div class="paragraph"> +<p>Most commands support the ability to specify an <code>--outputType</code> argument, or <code>-ot</code> for short.</p> +</div> +<div class="paragraph"> +<p>Currently the output type may be simple or json.</p> +</div> +<div class="paragraph"> +<p>The default output type in interactive mode is simple, and the default output type in standalone mode is json.</p> +</div> +<div class="paragraph"> +<p>Example of simple output for <code>list-buckets</code>:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>#> registry list-buckets -ot simple +My Bucket - 3c7b7467-0012-4d8f-a918-6aa42b6b9d39</pre> +</div> +</div> +<div class="paragraph"> +<p>Example of json output for <code>list-buckets</code>:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>#> registry list-buckets -ot json +[ { + "identifier" : "3c7b7467-0012-4d8f-a918-6aa42b6b9d39", + "name" : "My Bucket", + "createdTimestamp" : 1516718733854, + "permissions" : { + "canRead" : true, + "canWrite" : true, + "canDelete" : true + }, + "link" : { + "params" : { + "rel" : "self" + }, + "href" : "buckets/3c7b7467-0012-4d8f-a918-6aa42b6b9d39" + } +} ]</pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="back-referencing"><a class="anchor" href="toolkit-guide.html#back-referencing"></a>Back-Referencing</h3> +<div class="paragraph"> +<p>When using the interactive CLI, a common scenario will be using an id from a previous result as the input to the next command. Back-referencing provides a shortcut for referencing a result from the previous command via a positional reference.</p> +</div> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +Not every command produces back-references. To determine if a command supports back-referencing, check the usage. +</td> +</tr> +</table> +</div> +<div class="literalblock"> +<div class="content"> +<pre>#> registry list-buckets help +Lists the buckets that the current user has access to. +PRODUCES BACK-REFERENCES</pre> +</div> +</div> +<div class="paragraph"> +<p>A common scenario for utilizing back-references would be the following:</p> +</div> +<div class="olist arabic"> +<ol class="arabic"> +<li> +<p>User starts by exploring the available buckets in a registry instance</p> +<div class="literalblock"> +<div class="content"> +<pre>#> registry list-buckets +# Name Id Description +- ------------ ------------------------------------ ----------- +1 My Bucket 3c7b7467-0012-4d8f-a918-6aa42b6b9d39 (empty) +2 Other Bucket 175fb557-43a2-4abb-871f-81a354f47bc2 (empty)</pre> +</div> +</div> +</li> +<li> +<p>User then views the flows in one of the buckets using a back-reference to the bucket id from the previous result in position 1</p> +<div class="literalblock"> +<div class="content"> +<pre>#> registry list-flows -b &1 +Using a positional back-reference for 'My Bucket' +# Name Id Description +- ------- ------------------------------------ ---------------- +1 My Flow 06acb207-d2f1-447f-85ed-9b8672fe6d30 This is my flow.</pre> +</div> +</div> +</li> +<li> +<p>User then views the version of the flow using a back-reference to the flow id from the previous result in position 1</p> +<div class="literalblock"> +<div class="content"> +<pre>#> registry list-flow-versions -f &1 +Using a positional back-reference for 'My Flow' +Ver Date Author Message +--- -------------------------- ------------------------ ------------------------------------- +1 Tue, Jan 23 2018 09:48 EST anonymous This is the first version of my flow.</pre> +</div> +</div> +</li> +<li> +<p>User deploys version 1 of the flow using back-references to the bucket and flow id from step 2</p> +<div class="literalblock"> +<div class="content"> +<pre>#> nifi pg-import -b &1 -f &1 -fv 1 +Using a positional back-reference for 'My Bucket' +Using a positional back-reference for 'My Flow' +9bd157d4-0161-1000-b946-c1f9b1832efd</pre> +</div> +</div> +</li> +</ol> +</div> +<div class="paragraph"> +<p>The reason step 4 was able to reference the results from step 2, is because the <code>list-flow-versions</code> command in step 3 does not produce back-references, so the results from step 2 are still available.</p> +</div> +</div> +<div class="sect2"> +<h3 id="adding-commands"><a class="anchor" href="toolkit-guide.html#adding-commands"></a>Adding Commands</h3> +<div class="paragraph"> +<p>To add a NiFi command, create a new class that extends <code>AbstractNiFiCommand</code>:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>public class MyCommand extends AbstractNiFiCommand { + + public MyCommand() { + super("my-command"); + } + + @Override + protected void doExecute(NiFiClient client, Properties properties) + throws NiFiClientException, IOException, MissingOptionException, CommandException { + // TODO implement + } + + @Override + public String getDescription() { + return "This is my new command"; + } +}</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Add the new command to <code>NiFiCommandGroup</code>:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code>commands.add(new MyCommand());</code></pre> +</div> +</div> +<div class="paragraph"> +<p>To add a NiFi Registry command, perform the same steps, but extend from <code>AbstractNiFiRegistryCommand</code>, and add the command to <code>NiFiRegistryCommandGroup</code>.</p> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="encrypt_config_tool"><a class="anchor" href="toolkit-guide.html#encrypt_config_tool"></a>Encrypt-Config Tool</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The <code>encrypt-config</code> command line tool (invoked as <code>./bin/encrypt-config.sh</code> or <code>bin\encrypt-config.bat</code>) reads from a <em>nifi.properties</em> file with plaintext sensitive configuration values, prompts for a master password or raw hexadecimal key, and encrypts each value. It replaces the plain values with the protected value in the same file, or writes to a new <em>nifi.properties</em> file if specified.</p> +</div> +<div class="paragraph"> +<p>The default encryption algorithm utilized is AES/GCM 128/256-bit. 128-bit is used if the JCE Unlimited Strength Cryptographic Jurisdiction Policy files are not installed, and 256-bit is used if they are installed.</p> +</div> +<div class="sect2"> +<h3 id="usage-2"><a class="anchor" href="toolkit-guide.html#usage-2"></a>Usage</h3> +<div class="paragraph"> +<p>To show help:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/encrypt-config.sh -h</pre> +</div> +</div> +<div class="paragraph"> +<p>The following are available options:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-h</code>,<code>--help</code> Prints this usage message</p> +</li> +<li> +<p><code>-v</code>,<code>--verbose</code> Sets verbose mode (default false)</p> +</li> +<li> +<p><code>-n</code>,<code>--niFiProperties <arg></code> The <em>nifi.properties</em> file containing unprotected config values (will be overwritten)</p> +</li> +<li> +<p><code>-l</code>,<code>--loginIdentityProviders <arg></code> The <em>login-identity-providers.xml</em> file containing unprotected config values (will be overwritten)</p> +</li> +<li> +<p><code>-a</code>,<code>--authorizers <arg></code> The <em>authorizers.xml</em> file containing unprotected config values (will be overwritten)</p> +</li> +<li> +<p><code>-f</code>,<code>--flowXml <arg></code> The <em>flow.xml.gz</em> file currently protected with old password (will be overwritten)</p> +</li> +<li> +<p><code>-b</code>,<code>--bootstrapConf <arg></code> The <em>bootstrap.conf</em> file to persist master key</p> +</li> +<li> +<p><code>-o</code>,<code>--outputNiFiProperties <arg></code> The destination <em>nifi.properties</em> file containing protected config values (will not modify input <em>nifi.properties</em>)</p> +</li> +<li> +<p><code>-i</code>,<code>--outputLoginIdentityProviders <arg></code> The destination <em>login-identity-providers.xml</em> file containing protected config values (will not modify input <em>login-identity-providers.xml</em>)</p> +</li> +<li> +<p><code>-u</code>,<code>--outputAuthorizers <arg></code> The destination <em>authorizers.xml</em> file containing protected config values (will not modify input <em>authorizers.xml</em>)</p> +</li> +<li> +<p><code>-g</code>,<code>--outputFlowXml <arg></code> The destination <em>flow.xml.gz</em> file containing protected config values (will not modify input <em>flow.xml.gz</em>)</p> +</li> +<li> +<p><code>-k</code>,<code>--key <arg></code> The raw hexadecimal key to use to encrypt the sensitive properties</p> +</li> +<li> +<p><code>-e</code>,<code>--oldKey <arg></code> The old raw hexadecimal key to use during key migration</p> +</li> +<li> +<p><code>-p</code>,<code>--password <arg></code> The password from which to derive the key to use to encrypt the sensitive properties</p> +</li> +<li> +<p><code>-w</code>,<code>--oldPassword <arg></code> The old password from which to derive the key during migration</p> +</li> +<li> +<p><code>-r</code>,<code>--useRawKey</code> If provided, the secure console will prompt for the raw key value in hexadecimal form</p> +</li> +<li> +<p><code>-m</code>,<code>--migrate</code> If provided, the <em>nifi.properties</em> and/or <em>login-identity-providers.xml</em> sensitive properties will be re-encrypted with a new key</p> +</li> +<li> +<p><code>-x</code>,<code>--encryptFlowXmlOnly</code> If provided, the properties in <em>flow.xml.gz</em> will be re-encrypted with a new key but the <em>nifi.properties</em> and/or <em>login-identity-providers.xml</em> files will not be modified</p> +</li> +<li> +<p><code>-s</code>,<code>--propsKey <arg></code> The password or key to use to encrypt the sensitive processor properties in <em>flow.xml.gz</em></p> +</li> +<li> +<p><code>-A</code>,<code>--newFlowAlgorithm <arg></code> The algorithm to use to encrypt the sensitive processor properties in <em>flow.xml.gz</em></p> +</li> +<li> +<p><code>-P</code>,<code>--newFlowProvider <arg></code> The security provider to use to encrypt the sensitive processor properties in <em>flow.xml.gz</em></p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>As an example of how the tool works, assume that you have installed the tool on a machine supporting 256-bit encryption and with the following existing values in the <em>nifi.properties</em> file:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code># security properties # +nifi.sensitive.props.key=thisIsABadSensitiveKeyPassword +nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL +nifi.sensitive.props.provider=BC +nifi.sensitive.props.additional.keys= + +nifi.security.keystore=/path/to/keystore.jks +nifi.security.keystoreType=JKS +nifi.security.keystorePasswd=thisIsABadKeystorePassword +nifi.security.keyPasswd=thisIsABadKeyPassword +nifi.security.truststore= +nifi.security.truststoreType= +nifi.security.truststorePasswd=</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Enter the following arguments when using the tool:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre>encrypt-config.sh +-b bootstrap.conf +-k 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 +-n nifi.properties</pre> +</div> +</div> +<div class="paragraph"> +<p>As a result, the <em>nifi.properties</em> file is overwritten with protected properties and sibling encryption identifiers (<code>aes/gcm/256</code>, the currently supported algorithm):</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code># security properties # +nifi.sensitive.props.key=n2z+tTTbHuZ4V4V2||uWhdasyDXD4ZG2lMAes/vqh6u4vaz4xgL4aEbF4Y/dXevqk3ulRcOwf1vc4RDQ== +nifi.sensitive.props.key.protected=aes/gcm/256 +nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL +nifi.sensitive.props.provider=BC +nifi.sensitive.props.additional.keys= + +nifi.security.keystore=/path/to/keystore.jks +nifi.security.keystoreType=JKS +nifi.security.keystorePasswd=oBjT92hIGRElIGOh||MZ6uYuWNBrOA6usq/Jt3DaD2e4otNirZDytac/w/KFe0HOkrJR03vcbo +nifi.security.keystorePasswd.protected=aes/gcm/256 +nifi.security.keyPasswd=ac/BaE35SL/esLiJ||+ULRvRLYdIDA2VqpE0eQXDEMjaLBMG2kbKOdOwBk/hGebDKlVg== +nifi.security.keyPasswd.protected=aes/gcm/256 +nifi.security.truststore= +nifi.security.truststoreType= +nifi.security.truststorePasswd=</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Additionally, the <em>bootstrap.conf</em> file is updated with the encryption key as follows:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code># Master key in hexadecimal format for encrypted sensitive configuration values +nifi.bootstrap.sensitive.key=0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210</code></pre> +</div> +</div> +<div class="paragraph"> +<p>Sensitive configuration values are encrypted by the tool by default, however you can encrypt any additional properties, if desired. To encrypt additional properties, specify them as comma-separated values in the <code>nifi.sensitive.props.additional.keys</code> property.</p> +</div> +<div class="paragraph"> +<p>If the <em>nifi.properties</em> file already has valid protected values, those property values are not modified by the tool.</p> +</div> +<div class="paragraph"> +<p>When applied to <em>login-identity-providers.xml</em> and <em>authorizers.xml</em>, the property elements are updated with an <code>encryption</code> attribute:</p> +</div> +<div class="paragraph"> +<p>Example of protected <em>login-identity-providers.xml</em>:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code> <!-- LDAP Provider --> + <provider> + <identifier>ldap-provider</identifier> + <class>org.apache.nifi.ldap.LdapProvider</class> + <property name="Authentication Strategy">START_TLS</property> + <property name="Manager DN">someuser</property> + <property name="Manager Password" encryption="aes/gcm/128">q4r7WIgN0MaxdAKM||SGgdCTPGSFEcuH4RraMYEdeyVbOx93abdWTVSWvh1w+klA</property> + <property name="TLS - Keystore"></property> + <property name="TLS - Keystore Password" encryption="aes/gcm/128">Uah59TWX+Ru5GY5p||B44RT/LJtC08QWA5ehQf01JxIpf0qSJUzug25UwkF5a50g</property> + <property name="TLS - Keystore Type"></property> + ... + </provider></code></pre> +</div> +</div> +<div class="paragraph"> +<p>Example of protected <em>authorizers.xml</em>:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code> <!-- LDAP User Group Provider --> + <userGroupProvider> + <identifier>ldap-user-group-provider</identifier> + <class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class> + <property name="Authentication Strategy">START_TLS</property> + <property name="Manager DN">someuser</property> + <property name="Manager Password" encryption="aes/gcm/128">q4r7WIgN0MaxdAKM||SGgdCTPGSFEcuH4RraMYEdeyVbOx93abdWTVSWvh1w+klA</property> + <property name="TLS - Keystore"></property> + <property name="TLS - Keystore Password" encryption="aes/gcm/128">Uah59TWX+Ru5GY5p||B44RT/LJtC08QWA5ehQf01JxIpf0qSJUzug25UwkF5a50g</property> + <property name="TLS - Keystore Type"></property> + ... + </userGroupProvider></code></pre> +</div> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="file-manager"><a class="anchor" href="toolkit-guide.html#file-manager"></a>File Manager</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The File Manager utility (invoked as <code>./bin/file-manager.sh</code> or <code>bin\file-manager.bat</code>) allows system administrators to take a backup of an existing NiFi installation, install a new version of NiFi in a designated location (while migrating any previous configuration settings) or restore an installation from a previous backup. File Manager supports NiFi version 1.0.0 and higher.</p> +</div> +<div class="sect2"> +<h3 id="usage-3"><a class="anchor" href="toolkit-guide.html#usage-3"></a>Usage</h3> +<div class="paragraph"> +<p>To show help:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/file-manager.sh -h</pre> +</div> +</div> +<div class="paragraph"> +<p>The following are available options:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-b</code>,<code>--backupDir <arg></code> Backup NiFi Directory (used with backup or restore operation)</p> +</li> +<li> +<p><code>-c</code>,<code>--nifiCurrentDir <arg></code> Current NiFi Installation Directory (used optionally with install or restore operation)</p> +</li> +<li> +<p><code>-d</code>,<code>--nifiInstallDir <arg></code> NiFi Installation Directory (used with install or restore operation)</p> +</li> +<li> +<p><code>-h</code>,<code>--help</code> Print help info (optional)</p> +</li> +<li> +<p><code>-i</code>,<code>--installFile <arg></code> NiFi Install File (used with install operation)</p> +</li> +<li> +<p><code>-m</code>,<code>--moveRepositories</code> Allow repositories to be moved to new/restored nifi directory from existing installation, if available (used optionally with install or restore operation)</p> +</li> +<li> +<p><code>-o</code>,<code>--operation <arg></code> File operation (install | backup | restore)</p> +</li> +<li> +<p><code>-r</code>,<code>--nifiRollbackDir <arg></code> NiFi Installation Directory (used with install or restore operation)</p> +</li> +<li> +<p><code>-t</code>,<code>--bootstrapConf <arg></code> Current NiFi Bootstrap Configuration File (used optionally)</p> +</li> +<li> +<p><code>-v</code>,<code>--verbose</code> Verbose messaging (optional)</p> +</li> +<li> +<p><code>-x</code>,<code>--overwriteConfigs</code> Overwrite existing configuration directory with upgrade changes (used optionally with install or restore operation)</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>Example usage on Linux:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre># backup NiFi installation +# option -t may be provided to ensure backup of external boostrap.conf file +./file-manager.sh +-o backup +âb /tmp/nifi_bak +âc /usr/nifi_old +-v</pre> +</div> +</div> +<div class="literalblock"> +<div class="content"> +<pre># install NiFi using compressed tar file into /usr/nifi directory (should install as /usr/nifi/nifi-1.3.0). +# migrate existing configurations with location determined by external bootstrap.conf and move over repositories from nifi_old +# options -t and -c should both be provided if migration of configurations, state and repositories are required +./file-manager.sh +-o install +âi nifi-1.3.0.tar.gz +âd /usr/nifi +âc /usr/nifi/nifi_old +-t /usr/nifi/old_conf/bootstrap.conf +-v +-m</pre> +</div> +</div> +<div class="literalblock"> +<div class="content"> +<pre># restore NiFi installation from backup directory and move back repositories +# option -t may be provided to ensure bootstrap.conf is restored to the file path provided, otherwise it is placed in the +# default directory under the rollback path (e.g. /usr/nifi_old/conf) +./file-manager.sh +-o restore +âb /tmp/nifi_bak +âr /usr/nifi_old +âc /usr/nifi +-m +-v</pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="expected-behavior"><a class="anchor" href="toolkit-guide.html#expected-behavior"></a>Expected Behavior</h3> +<div class="sect3"> +<h4 id="backup"><a class="anchor" href="toolkit-guide.html#backup"></a>Backup</h4> +<div class="paragraph"> +<p>During the backup operation a backup directory is created in a designated location for an existing NiFi installation. Backups will capture all critical files (including any internal or external configurations, libraries, scripts and documents) however it excludes backing up repositories and logs due to potential size. If configuration/library files are external from the existing installation folder the backup operation will capture those as well.</p> +</div> +</div> +<div class="sect3"> +<h4 id="install"><a class="anchor" href="toolkit-guide.html#install"></a>Install</h4> +<div class="paragraph"> +<p>During the install operation File Manager will perform installation using the designated NiFi binary file (either tar.gz or zip file) to create a new installation or migrate an existing nifi installation to a new one. Installation can optionally move repositories (if located within the configuration folder of the current installation) to the new installation as well as migrate configuration files to the newer installation.</p> +</div> +</div> +<div class="sect3"> +<h4 id="restore"><a class="anchor" href="toolkit-guide.html#restore"></a>Restore</h4> +<div class="paragraph"> +<p>The restore operation allows an existing installation to revert back to a previous installation. Using an existing backup directory (created from the backup operation) the FileManager utility will restore libraries, scripts and documents as well as revert to previous configurations.</p> +</div> +<div class="admonitionblock note"> +<table> +<tr> +<td class="icon"> +<i class="fa icon-note" title="Note"></i> +</td> +<td class="content"> +If repositories were changed due to the installation of a newer version of NiFi these may no longer be compatible during restore. In that scenario exclude the <code>-m</code> option to ensure new repositories will be created or, if repositories live outside of the NiFi directory, remove them so they can be recreated on startup after restore. +</td> +</tr> +</table> +</div> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="flow-analyzer"><a class="anchor" href="toolkit-guide.html#flow-analyzer"></a>Flow Analyzer</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The <code>flow-analyzer</code> tool (invoked as <code>./bin/flow-analyzer.sh</code> or <code>bin\flow-analyzer.bat</code>) analyzes the <em>flow.xml.gz</em> file and reports:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Total Bytes Utilized by the System</p> +</li> +<li> +<p>Min/Max Back Pressure Size</p> +</li> +<li> +<p>Average Back Pressure Size</p> +</li> +<li> +<p>Min/Max Flowfile Queue Size</p> +</li> +<li> +<p>Average Flowfile Queue Size</p> +</li> +</ul> +</div> +<div class="sect2"> +<h3 id="usage-4"><a class="anchor" href="toolkit-guide.html#usage-4"></a>Usage</h3> +<div class="paragraph"> +<p>To execute the <code>flow-analyzer</code> tool:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>flow-analyzer.sh <path to flow.xml.gz></pre> +</div> +</div> +<div class="paragraph"> +<p>Example:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>$ ./flow-analyzer.sh /Users/nifiuser/nifi-1.8.0/conf/flow.xml.gz +Using flow=/Users/nifiuser/nifi-1.8.0/conf/flow.xml.gz +Total Bytes Utilized by System=1518 GB +Max Back Pressure Size=1 GB +Min Back Pressure Size=1 GB +Average Back Pressure Size=2.504950495 GB +Max Flowfile Queue Size=10000 +Min Flowfile Queue Size=10000 +Avg Flowfile Queue Size=10000.000000000</pre> +</div> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="node-manager"><a class="anchor" href="toolkit-guide.html#node-manager"></a>Node Manager</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Node manager (invoked as <code>./bin/node-manager.sh</code> or <code>bin\node-manager.bat</code>) supports connecting, disconnecting and removing a node when in a cluster (an error message displays if the node is not part of a cluster) as well as obtaining the status of a node. When nodes are disconnected from a cluster and need to be connected or removed, a list of urls of connected nodes should be provided to send the required command to the active cluster. Node Manager supports NiFi version 1.0.0 and higher.</p> +</div> +<div class="sect2"> +<h3 id="usage-5"><a class="anchor" href="toolkit-guide.html#usage-5"></a>Usage</h3> +<div class="paragraph"> +<p>To show help:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/node-manager.sh -h</pre> +</div> +</div> +<div class="paragraph"> +<p>The following are available options:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-b</code>,<code>--bootstrapConf <arg></code> Existing Bootstrap Configuration file (required)</p> +</li> +<li> +<p><code>-d</code>,<code>--nifiInstallDir <arg></code> NiFi Root Folder (required)</p> +</li> +<li> +<p><code>-h</code>,<code>--help</code> Help Text (optional)</p> +</li> +<li> +<p><code>-o</code>, <code>--operation <arg></code> Operations supported: status, connect (cluster), disconnect (cluster), remove (cluster)</p> +</li> +<li> +<p><code>-p</code>,<code>--proxyDN <arg></code> Proxy or User DN (required for secured nodes doing connect, disconnect and remove operations)</p> +</li> +<li> +<p><code>-u</code>,<code>--clusterUrls <arg></code> Comma delimited list of active urls for cluster (optional). Not required for disconnecting a node yet will be needed when connecting or removing from a cluster</p> +</li> +<li> +<p><code>-v</code>,<code>--verbose</code> Verbose messaging (optional)</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>To connect, disconnect, or remove a node from a cluster:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>node-manager.sh -d {$NIFI_HOME} âb { nifi bootstrap file path} +-o {remove|disconnect|connect|status} [-u {url list}] [-p {proxy name}] [-v]</pre> +</div> +</div> +<div class="paragraph"> +<p>Example usage on Linux:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre># disconnect without cluster url list +./node-manager.sh +-d /usr/nifi/nifi_current +-b /usr/nifi/nifi_current/conf/bootstrap.conf +-o disconnect +âp ydavis@nifi +-v</pre> +</div> +</div> +<div class="literalblock"> +<div class="content"> +<pre>#with url list +./node-manager.sh +-d /usr/nifi/nifi_current +-b /usr/nifi/nifi_current/conf/bootstrap.conf +-o connect +-u 'http://nifi-server-1:8080,http://nifi-server-2:8080' +-v</pre> +</div> +</div> +<div class="paragraph"> +<p>Example usage on Windows:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>node-manager.bat +-d "C:\\Program Files\\nifi\\nifi-1.2.0-SNAPSHOT" +-b "C:\\Program Files\\nifi\\nifi-1.2.0-SNAPSHOT\\conf\\bootstrap.conf" +-o disconnect +âv</pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="expected-behavior-2"><a class="anchor" href="toolkit-guide.html#expected-behavior-2"></a>Expected Behavior</h3> +<div class="sect3"> +<h4 id="status"><a class="anchor" href="toolkit-guide.html#status"></a>Status</h4> +<div class="paragraph"> +<p>To obtain information on UI availability of a node, the status operation can be used to determine if the node is running. If the <code>âu (clusterUrls)</code> option is not provided the current node url is checked otherwise the urls provided will be checked.</p> +</div> +</div> +<div class="sect3"> +<h4 id="disconnect"><a class="anchor" href="toolkit-guide.html#disconnect"></a>Disconnect</h4> +<div class="paragraph"> +<p>When a node is disconnected from the cluster, the node itself should appear as disconnected and the cluster should have a bulletin indicating the disconnect request was received. The cluster should also show <em>n-1/n</em> nodes available in the cluster. For example, if 1 node is disconnected from a 3-node cluster, then "2 of 3" nodes should show on the remaining nodes in the cluster. Changes to the flow should not be allowed on the cluster with a disconnected node.</p> +</div> +</div> +<div class="sect3"> +<h4 id="connect"><a class="anchor" href="toolkit-guide.html#connect"></a>Connect</h4> +<div class="paragraph"> +<p>When the connect command is executed to reconnect a node to a cluster, upon completion the node itself should show that it has rejoined the cluster by showing <em>n/n</em> nodes. Previously it would have shown Disconnected. Other nodes in the cluster should receive a bulletin of the connect request and also show <em>n/n</em> nodes allowing for changes to be allowed to the flow.</p> +</div> +</div> +<div class="sect3"> +<h4 id="remove"><a class="anchor" href="toolkit-guide.html#remove"></a>Remove</h4> +<div class="paragraph"> +<p>When the remove command is executed the node should show as disconnected from a cluster. The nodes remaining in the cluster should show <em>n-1/n-1</em> nodes. For example, if 1 node is removed from a 3-node cluster, then the remaining 2 nodes should show "2 of 2" nodes. The cluster should allow a flow to be adjusted. The removed node can rejoin the cluster if restarted and the flow for the cluster has not changed. If the flow was changed, the flow template of the removed node should be deleted before restarting the node to allow it to obtain the cluster flow (otherwise an uninheritable flow file exception may occur).</p> +</div> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="notify"><a class="anchor" href="toolkit-guide.html#notify"></a>Notify</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>Notify (invoked as <code>./bin/notify.sh</code> or <code>bin\notify.bat</code>) allows administrators to send messages as bulletins to NiFi. Notify is supported on NiFi version 1.2.0 and higher.</p> +</div> +<div class="sect2"> +<h3 id="usage-6"><a class="anchor" href="toolkit-guide.html#usage-6"></a>Usage</h3> +<div class="paragraph"> +<p>To show help:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/notify.sh -h</pre> +</div> +</div> +<div class="paragraph"> +<p>The following are available options:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>-b</code>,<code>--bootstrapConf <arg></code> Existing Bootstrap Configuration file (required)</p> +</li> +<li> +<p><code>-d</code>,<code>--nifiInstallDir <arg></code> NiFi Root Folder (required)</p> +</li> +<li> +<p><code>-h</code>,<code>--help</code> Help Text (optional)</p> +</li> +<li> +<p><code>-l</code>,<code>--level <arg></code> Status level of bulletin â <code>INFO</code>, <code>WARN</code>, <code>ERROR</code></p> +</li> +<li> +<p><code>-m</code>,<code>--message <arg></code> Bulletin message (required)</p> +</li> +<li> +<p><code>-p</code>,<code>--proxyDN <arg></code> Proxy or User DN (required for secured nodes)</p> +</li> +<li> +<p><code>-v</code>,<code>--verbose</code> Verbose messaging (optional)</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>To send notifications:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>notify.sh -d {$NIFI_HOME} âb {nifi bootstrap file path} -m {message} [-l {level}] [-v]</pre> +</div> +</div> +<div class="paragraph"> +<p>Example usage on Linux:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./notify.sh -d /usr/nifi/nifi_current -b /usr/nifi/nifi_current/conf/bootstrap.conf -m "Test Message Server 1" -l "WARN" âp âydavis@nifiâ -v</pre> +</div> +</div> +<div class="paragraph"> +<p>Example usage on Windows:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>notify.bat -v -d "C:\\Program Files\\nifi\\nifi-1.2.0-SNAPSHOT" -b "C:\\Program Files\\nifi\\nifi-1.2.0-SNAPSHOT\\conf\\bootstrap.conf" -m "Test Message Server 1" -v</pre> +</div> +</div> +<div class="paragraph"> +<p>Executing the above command line should result in a bulletin appearing in NiFi:</p> +</div> +<div class="imageblock"> +<div class="content"> +<img src="images/nifi-notifications.png" alt="NiFi Notifications"> +</div> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="s2s"><a class="anchor" href="toolkit-guide.html#s2s"></a>S2S</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>S2S is a command line tool (invoked as <code>./bin/s2s.sh</code> or <code>bin\s2s.bat</code>) that can either read a list of DataPackets from stdin to send over site-to-site or write the received DataPackets to stdout.</p> +</div> +<div class="sect2"> +<h3 id="usage-7"><a class="anchor" href="toolkit-guide.html#usage-7"></a>Usage</h3> +<div class="paragraph"> +<p>To show help:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>./bin/s2s.sh -h</pre> +</div> +</div> +<div class="paragraph"> +<p>The following are available options:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p><code>--batchCount <arg></code> Number of flow files in a batch</p> +</li> +<li> +<p><code>--batchDuration <arg></code> Duration of a batch</p> +</li> +<li> +<p><code>--batchSize <arg></code> Size of flow files in a batch</p> +</li> +<li> +<p><code>-c</code>,<code>--compression</code> Use compression</p> +</li> +<li> +<p><code>-d</code>,<code>--direction</code> Direction (valid directions: <code>SEND</code>, <code>RECEIVE</code>) (default: <code>SEND</code>)</p> +</li> +<li> +<p><code>-h</code>,<code>--help</code> Help Text (optional)</p> +</li> +<li> +<p><code>-i</code>,<code>--portIdentifier <arg></code> Port id</p> +</li> +<li> +<p><code>--keystore <arg></code> Keystore</p> +</li> +<li> +<p><code>--keyStorePassword <arg></code> Keystore password</p> +</li> +<li> +<p><code>--keyStoreType <arg></code> Keystore type (default: <code>JKS</code>)</p> +</li> +<li> +<p><code>-n</code>,<code>--portName</code> Port name</p> +</li> +<li> +<p><code>-p</code>,<code>--transportProtocol</code> Site to site transport protocol (default: <code>RAW</code>)</p> +</li> +<li> +<p><code>--peerPersistenceFile <arg></code> File to write peer information to so it can be recovered on restart</p> +</li> +<li> +<p><code>--penalization <arg></code> Penalization period</p> +</li> +<li> +<p><code>--proxyHost <arg></code> Proxy hostname</p> +</li> +<li> +<p><code>--proxyPassword <arg></code> Proxy password</p> +</li> +<li> +<p><code>--proxyPort <arg></code> Proxy port</p> +</li> +<li> +<p><code>--proxyUsername <arg></code> Proxy username</p> +</li> +<li> +<p><code>--timeout <arg></code> Timeout</p> +</li> +<li> +<p><code>--trustStore <arg></code> Truststore</p> +</li> +<li> +<p><code>--trustStorePassword <arg></code> Truststore password</p> +</li> +<li> +<p><code>--trustStoreType <arg></code> Truststore type (default: <code>JKS</code>)</p> +</li> +<li> +<p><code>-u,--url <arg></code> NiFI URL to connect to (default: <code><a href="http://localhost:8080/nifi"; class="bare">http://localhost:8080/nifi</a></code>)</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>The s2s cli input/output format is a JSON list of DataPackets. They can have the following formats:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>[{"attributes":{"key":"value"},"data":"aGVsbG8gbmlmaQ=="}]</pre> +</div> +</div> +<div class="paragraph"> +<p>where data is the base64 encoded value of the FlowFile content (always used for received data) or:</p> +</div> +<div class="literalblock"> +<div class="content"> +<pre>[{"attributes":{"key":"value"},"dataFile":"/Users/pvillard/Documents/GitHub/nifi/nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.9.0-SNAPSHOT-bin/nifi-toolkit-1.9.0-SNAPSHOT/bin/EXAMPLE"}]</pre> +</div> +</div> +<div class="paragraph"> +<p>where dataFile is a file to read the FlowFile content from.</p> +</div>
[... 1127 lines stripped ...]
