Added: nifi/site/trunk/docs/nifi-docs/html/developer-guide.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/docs/nifi-docs/html/developer-guide.html?rev=1811008&view=auto ============================================================================== --- nifi/site/trunk/docs/nifi-docs/html/developer-guide.html (added) +++ nifi/site/trunk/docs/nifi-docs/html/developer-guide.html Tue Oct 3 13:30:16 2017 @@ -0,0 +1,3655 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + --> + <!DOCTYPE html> +<html lang="en"> +<head> +<meta charset="UTF-8"> +<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]--> +<meta name="viewport" content="width=device-width, initial-scale=1.0"> +<meta name="generator" content="Asciidoctor 1.5.2"> +<meta name="author" content="Apache NiFi Team"> +<title>NiFi Developer’s Guide</title> +<style> +/* Asciidoctor default stylesheet | MIT License | http://asciidoctor.org */ +/* Copyright (C) 2012-2015 Dan Allen, Ryan Waldron and the Asciidoctor Project + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. */ +/* Remove the comments around the @import statement below when using this as a custom stylesheet */ +@import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400"; +article,aside,details,figcaption,figure,footer,header,hgroup,main,nav,section,summary{display:block} +audio,canvas,video{display:inline-block} +audio:not([controls]){display:none;height:0} +[hidden],template{display:none} +script{display:none!important} +html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%} +body{margin:0} +a{background:transparent} +a:focus{outline:thin dotted} +a:active,a:hover{outline:0} +h1{font-size:2em;margin:.67em 0} +abbr[title]{border-bottom:1px dotted} +b,strong{font-weight:bold} +dfn{font-style:italic} +hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0} +mark{background:#ff0;color:#000} +code,kbd,pre,samp{font-family:monospace;font-size:1em} +pre{white-space:pre-wrap} +q{quotes:"\201C" "\201D" "\2018" "\2019"} +small{font-size:80%} +sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline} +sup{top:-.5em} +sub{bottom:-.25em} +img{border:0} +svg:not(:root){overflow:hidden} +figure{margin:0} +fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em} +legend{border:0;padding:0} +button,input,select,textarea{font-family:inherit;font-size:100%;margin:0} +button,input{line-height:normal} +button,select{text-transform:none} +button,html input[type="button"],input[type="reset"],input[type="submit"]{-webkit-appearance:button;cursor:pointer} +button[disabled],html input[disabled]{cursor:default} +input[type="checkbox"],input[type="radio"]{box-sizing:border-box;padding:0} +input[type="search"]{-webkit-appearance:textfield;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;box-sizing:content-box} +input[type="search"]::-webkit-search-cancel-button,input[type="search"]::-webkit-search-decoration{-webkit-appearance:none} +button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0} +textarea{overflow:auto;vertical-align:top} +table{border-collapse:collapse;border-spacing:0} +*,*:before,*:after{-moz-box-sizing:border-box;-webkit-box-sizing:border-box;box-sizing:border-box} +html,body{font-size:100%} +body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;font-weight:400;font-style:normal;line-height:1;position:relative;cursor:auto} +a:hover{cursor:pointer} +img,object,embed{max-width:100%;height:auto} +object,embed{height:100%} +img{-ms-interpolation-mode:bicubic} +#map_canvas img,#map_canvas embed,#map_canvas object,.map_canvas img,.map_canvas embed,.map_canvas object{max-width:none!important} +.left{float:left!important} +.right{float:right!important} +.text-left{text-align:left!important} +.text-right{text-align:right!important} +.text-center{text-align:center!important} +.text-justify{text-align:justify!important} +.hide{display:none} +.antialiased,body{-webkit-font-smoothing:antialiased} +img{display:inline-block;vertical-align:middle} +textarea{height:auto;min-height:50px} +select{width:100%} +p.lead,.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{font-size:1.21875em;line-height:1.6} +.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em} +div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0;direction:ltr} +a{color:#2156a5;text-decoration:underline;line-height:inherit} +a:hover,a:focus{color:#1d4b8f} +a img{border:none} +p{font-family:inherit;font-weight:400;font-size:1em;line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility} +p aside{font-size:.875em;line-height:1.35;font-style:italic} +h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em} +h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0} +h1{font-size:2.125em} +h2{font-size:1.6875em} +h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em} +h4,h5{font-size:1.125em} +h6{font-size:1em} +hr{border:solid #ddddd8;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em;height:0} +em,i{font-style:italic;line-height:inherit} +strong,b{font-weight:bold;line-height:inherit} +small{font-size:60%;line-height:inherit} +code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9);padding-right: 1px;} +ul,ol,dl{font-size:1em;line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit} +ul,ol,ul.no-bullet,ol.no-bullet{margin-left:1.5em} +ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0;font-size:1em} +ul.square li ul,ul.circle li ul,ul.disc li ul{list-style:inherit} +ul.square{list-style-type:square} +ul.circle{list-style-type:circle} +ul.disc{list-style-type:disc} +ul.no-bullet{list-style:none} +ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0} +dl dt{margin-bottom:.3125em;font-weight:bold} +dl dd{margin-bottom:1.25em} +abbr,acronym{text-transform:uppercase;font-size:90%;color:rgba(0,0,0,.8);border-bottom:1px dotted #ddd;cursor:help} +abbr{text-transform:none} +blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd} +blockquote cite{display:block;font-size:.9375em;color:rgba(0,0,0,.6)} +blockquote cite:before{content:"\2014 \0020"} +blockquote cite a,blockquote cite a:visited{color:rgba(0,0,0,.6)} +blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)} +@media only screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2} +h1{font-size:2.75em} +h2{font-size:2.3125em} +h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em} +h4{font-size:1.4375em}}table{background:#fff;margin-bottom:1.25em;border:solid 1px #dedede} +table thead,table tfoot{background:#f7f8f7;font-weight:bold} +table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left} +table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)} +table tr.even,table tr.alt,table tr:nth-of-type(even){background:#f8f8f7} +table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{display:table-cell;line-height:1.6} +h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em} +h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400} +.clearfix:before,.clearfix:after,.float-group:before,.float-group:after{content:" ";display:table} +.clearfix:after,.float-group:after{clear:both} +*:not(pre)>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;word-spacing:-.15em;background-color:#f7f7f8;-webkit-border-radius:4px;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed} +pre,pre>code{line-height:1.45;color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;text-rendering:optimizeSpeed} +.keyseq{color:rgba(51,51,51,.8)} +kbd{display:inline-block;color:rgba(0,0,0,.8);font-size:.75em;line-height:1.4;background-color:#f7f7f7;border:1px solid #ccc;-webkit-border-radius:3px;border-radius:3px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em white inset;box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em #fff inset;margin:-.15em .15em 0 .15em;padding:.2em .6em .2em .5em;vertical-align:middle;white-space:nowrap} +.keyseq kbd:first-child{margin-left:0} +.keyseq kbd:last-child{margin-right:0} +.menuseq,.menu{color:rgba(0,0,0,.8)} +b.button:before,b.button:after{position:relative;top:-1px;font-weight:400} +b.button:before{content:"[";padding:0 3px 0 2px} +b.button:after{content:"]";padding:0 2px 0 3px} +p a>code:hover{color:rgba(0,0,0,.9)} +#header,#content,#footnotes,#footer{width:100%;margin-left:auto;margin-right:auto;margin-top:0;margin-bottom:0;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em} +#header:before,#header:after,#content:before,#content:after,#footnotes:before,#footnotes:after,#footer:before,#footer:after{content:" ";display:table} +#header:after,#content:after,#footnotes:after,#footer:after{clear:both} +#content{margin-top:1.25em} +#content:before{content:none} +#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0} +#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #ddddd8} +#header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #ddddd8;padding-bottom:8px} +#header .details{border-bottom:1px solid #ddddd8;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:-ms-flexbox;display:-webkit-flex;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap} +#header .details span:first-child{margin-left:-.125em} +#header .details span.email a{color:rgba(0,0,0,.85)} +#header .details br{display:none} +#header .details br+span:before{content:"\00a0\2013\00a0"} +#header .details br+span.author:before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)} +#header .details br+span#revremark:before{content:"\00a0|\00a0"} +#header #revnumber{text-transform:capitalize} +#header #revnumber:after{content:"\00a0"} +#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #ddddd8;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem} +#toc{border-bottom:1px solid #efefed;padding-bottom:.5em} +#toc>ul{margin-left:.125em} +#toc ul.sectlevel0>li>a{font-style:italic} +#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0} +#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none} +#toc a{text-decoration:none} +#toc a:active{text-decoration:underline} +#toctitle{color:#7a2518;font-size:1.2em} +@media only screen and (min-width:768px){#toctitle{font-size:1.375em} +body.toc2{padding-left:15em;padding-right:0} +#toc.toc2{margin-top:0!important;background-color:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #efefed;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto} +#toc.toc2 #toctitle{margin-top:0;font-size:1.2em} +#toc.toc2>ul{font-size:.9em;margin-bottom:0} +#toc.toc2 ul ul{margin-left:0;padding-left:1em} +#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em} +body.toc2.toc-right{padding-left:0;padding-right:15em} +body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #efefed;left:auto;right:0}}@media only screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0} +#toc.toc2{width:20em} +#toc.toc2 #toctitle{font-size:1.375em} +#toc.toc2>ul{font-size:.95em} +#toc.toc2 ul ul{padding-left:1.25em} +body.toc2.toc-right{padding-left:0;padding-right:20em}}#content #toc{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px} +#content #toc>:first-child{margin-top:0} +#content #toc>:last-child{margin-bottom:0} +#footer{max-width:100%;background-color:rgba(0,0,0,.8);padding:1.25em} +#footer-text{color:rgba(255,255,255,.8);line-height:1.44} +.sect1{padding-bottom:.625em} +@media only screen and (min-width:768px){.sect1{padding-bottom:1.25em}}.sect1+.sect1{border-top:1px solid #efefed} +#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400} +#content h1>a.anchor:before,h2>a.anchor:before,h3>a.anchor:before,#toctitle>a.anchor:before,.sidebarblock>.content>.title>a.anchor:before,h4>a.anchor:before,h5>a.anchor:before,h6>a.anchor:before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em} +#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible} +#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none} +#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221} +.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em} +.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic} +table.tableblock>caption.title{white-space:nowrap;overflow:visible;max-width:0} +.paragraph.lead>p,#preamble>.sectionbody>.paragraph:first-of-type p{color:rgba(0,0,0,.85)} +table.tableblock #preamble>.sectionbody>.paragraph:first-of-type p{font-size:inherit} +.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%} +.admonitionblock>table td.icon{text-align:center;width:80px} +.admonitionblock>table td.icon img{max-width:none} +.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase} +.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #ddddd8;color:rgba(0,0,0,.6)} +.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0} +.exampleblock>.content{border-style:solid;border-width:1px;border-color:#e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;-webkit-border-radius:4px;border-radius:4px} +.exampleblock>.content>:first-child{margin-top:0} +.exampleblock>.content>:last-child{margin-bottom:0} +.sidebarblock{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px} +.sidebarblock>:first-child{margin-top:0} +.sidebarblock>:last-child{margin-bottom:0} +.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center} +.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0} +.literalblock pre,.listingblock pre:not(.highlight),.listingblock pre[class="highlight"],.listingblock pre[class^="highlight "],.listingblock pre.CodeRay,.listingblock pre.prettyprint{background:#f7f7f8} +.sidebarblock .literalblock pre,.sidebarblock .listingblock pre:not(.highlight),.sidebarblock .listingblock pre[class="highlight"],.sidebarblock .listingblock pre[class^="highlight "],.sidebarblock .listingblock pre.CodeRay,.sidebarblock .listingblock pre.prettyprint{background:#f2f1f1} +.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{-webkit-border-radius:4px;border-radius:4px;word-wrap:break-word;padding:1em;font-size:.8125em} +.literalblock pre.nowrap,.literalblock pre[class].nowrap,.listingblock pre.nowrap,.listingblock pre[class].nowrap{overflow-x:auto;white-space:pre;word-wrap:normal} +@media only screen and (min-width:768px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:.90625em}}@media only screen and (min-width:1280px){.literalblock pre,.literalblock pre[class],.listingblock pre,.listingblock pre[class]{font-size:1em}}.literalblock.output pre{color:#f7f7f8;background-color:rgba(0,0,0,.9)} +.listingblock pre.highlightjs{padding:0} +.listingblock pre.highlightjs>code{padding:1em;-webkit-border-radius:4px;border-radius:4px} +.listingblock pre.prettyprint{border-width:0} +.listingblock>.content{position:relative} +.listingblock code[data-lang]:before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:#999} +.listingblock:hover code[data-lang]:before{display:block} +.listingblock.terminal pre .command:before{content:attr(data-prompt);padding-right:.5em;color:#999} +.listingblock.terminal pre .command:not([data-prompt]):before{content:"$"} +table.pyhltable{border-collapse:separate;border:0;margin-bottom:0;background:none} +table.pyhltable td{vertical-align:top;padding-top:0;padding-bottom:0} +table.pyhltable td.code{padding-left:.75em;padding-right:0} +pre.pygments .lineno,table.pyhltable td:not(.code){color:#999;padding-left:0;padding-right:.5em;border-right:1px solid #ddddd8} +pre.pygments .lineno{display:inline-block;margin-right:.25em} +table.pyhltable .linenodiv{background:none!important;padding-right:0!important} +.quoteblock{margin:0 1em 1.25em 1.5em;display:table} +.quoteblock>.title{margin-left:-1.5em;margin-bottom:.75em} +.quoteblock blockquote,.quoteblock blockquote p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify} +.quoteblock blockquote{margin:0;padding:0;border:0} +.quoteblock blockquote:before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)} +.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0} +.quoteblock .attribution{margin-top:.5em;margin-right:.5ex;text-align:right} +.quoteblock .quoteblock{margin-left:0;margin-right:0;padding:.5em 0;border-left:3px solid rgba(0,0,0,.6)} +.quoteblock .quoteblock blockquote{padding:0 0 0 .75em} +.quoteblock .quoteblock blockquote:before{display:none} +.verseblock{margin:0 1em 1.25em 1em} +.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility} +.verseblock pre strong{font-weight:400} +.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex} +.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic} +.quoteblock .attribution br,.verseblock .attribution br{display:none} +.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.05em;color:rgba(0,0,0,.6)} +.quoteblock.abstract{margin:0 0 1.25em 0;display:block} +.quoteblock.abstract blockquote,.quoteblock.abstract blockquote p{text-align:left;word-spacing:0} +.quoteblock.abstract blockquote:before,.quoteblock.abstract blockquote p:first-of-type:before{display:none} +table.tableblock{max-width:100%;border-collapse:separate} +table.tableblock td>.paragraph:last-child p>p:last-child,table.tableblock th>p:last-child,table.tableblock td>p:last-child{margin-bottom:0} +table.spread{width:100%} +table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede} +table.grid-all th.tableblock,table.grid-all td.tableblock{border-width:0 1px 1px 0} +table.grid-all tfoot>tr>th.tableblock,table.grid-all tfoot>tr>td.tableblock{border-width:1px 1px 0 0} +table.grid-cols th.tableblock,table.grid-cols td.tableblock{border-width:0 1px 0 0} +table.grid-all *>tr>.tableblock:last-child,table.grid-cols *>tr>.tableblock:last-child{border-right-width:0} +table.grid-rows th.tableblock,table.grid-rows td.tableblock{border-width:0 0 1px 0} +table.grid-all tbody>tr:last-child>th.tableblock,table.grid-all tbody>tr:last-child>td.tableblock,table.grid-all thead:last-child>tr>th.tableblock,table.grid-rows tbody>tr:last-child>th.tableblock,table.grid-rows tbody>tr:last-child>td.tableblock,table.grid-rows thead:last-child>tr>th.tableblock{border-bottom-width:0} +table.grid-rows tfoot>tr>th.tableblock,table.grid-rows tfoot>tr>td.tableblock{border-width:1px 0 0 0} +table.frame-all{border-width:1px} +table.frame-sides{border-width:0 1px} +table.frame-topbot{border-width:1px 0} +th.halign-left,td.halign-left{text-align:left} +th.halign-right,td.halign-right{text-align:right} +th.halign-center,td.halign-center{text-align:center} +th.valign-top,td.valign-top{vertical-align:top} +th.valign-bottom,td.valign-bottom{vertical-align:bottom} +th.valign-middle,td.valign-middle{vertical-align:middle} +table thead th,table tfoot th{font-weight:bold} +tbody tr th{display:table-cell;line-height:1.6;background:#f7f8f7} +tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold} +p.tableblock>code:only-child{background:none;padding:0} +p.tableblock{font-size:1em} +td>div.verse{white-space:pre} +ol{margin-left:1.75em} +ul li ol{margin-left:1.5em} +dl dd{margin-left:1.125em} +dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0} +ol>li p,ul>li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em} +ul.unstyled,ol.unnumbered,ul.checklist,ul.none{list-style-type:none} +ul.unstyled,ol.unnumbered,ul.checklist{margin-left:.625em} +ul.checklist li>p:first-child>.fa-square-o:first-child,ul.checklist li>p:first-child>.fa-check-square-o:first-child{width:1em;font-size:.85em} +ul.checklist li>p:first-child>input[type="checkbox"]:first-child{width:1em;position:relative;top:1px} +ul.inline{margin:0 auto .625em auto;margin-left:-1.375em;margin-right:0;padding:0;list-style:none;overflow:hidden} +ul.inline>li{list-style:none;float:left;margin-left:1.375em;display:block} +ul.inline>li>*{display:block} +.unstyled dl dt{font-weight:400;font-style:normal} +ol.arabic{list-style-type:decimal} +ol.decimal{list-style-type:decimal-leading-zero} +ol.loweralpha{list-style-type:lower-alpha} +ol.upperalpha{list-style-type:upper-alpha} +ol.lowerroman{list-style-type:lower-roman} +ol.upperroman{list-style-type:upper-roman} +ol.lowergreek{list-style-type:lower-greek} +.hdlist>table,.colist>table{border:0;background:none} +.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none} +td.hdlist1{padding-right:.75em;font-weight:bold} +td.hdlist1,td.hdlist2{vertical-align:top} +.literalblock+.colist,.listingblock+.colist{margin-top:-.5em} +.colist>table tr>td:first-of-type{padding:0 .75em;line-height:1} +.colist>table tr>td:last-of-type{padding:.25em 0} +.thumb,.th{line-height:0;display:inline-block;border:solid 4px #fff;-webkit-box-shadow:0 0 0 1px #ddd;box-shadow:0 0 0 1px #ddd} +.imageblock.left,.imageblock[style*="float: left"]{margin:.25em .625em 1.25em 0} +.imageblock.right,.imageblock[style*="float: right"]{margin:.25em 0 1.25em .625em} +.imageblock>.title{margin-bottom:0} +.imageblock.thumb,.imageblock.th{border-width:6px} +.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em} +.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0} +.image.left{margin-right:.625em} +.image.right{margin-left:.625em} +a.image{text-decoration:none} +span.footnote,span.footnoteref{vertical-align:super;font-size:.875em} +span.footnote a,span.footnoteref a{text-decoration:none} +span.footnote a:active,span.footnoteref a:active{text-decoration:underline} +#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em} +#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em 0;border-width:1px 0 0 0} +#footnotes .footnote{padding:0 .375em;line-height:1.3;font-size:.875em;margin-left:1.2em;text-indent:-1.2em;margin-bottom:.2em} +#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none} +#footnotes .footnote:last-of-type{margin-bottom:0} +#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0} +.gist .file-data>table{border:0;background:#fff;width:100%;margin-bottom:0} +.gist .file-data>table td.line-data{width:99%} +div.unbreakable{page-break-inside:avoid} +.big{font-size:larger} +.small{font-size:smaller} +.underline{text-decoration:underline} +.overline{text-decoration:overline} +.line-through{text-decoration:line-through} +.aqua{color:#00bfbf} +.aqua-background{background-color:#00fafa} +.black{color:#000} +.black-background{background-color:#000} +.blue{color:#0000bf} +.blue-background{background-color:#0000fa} +.fuchsia{color:#bf00bf} +.fuchsia-background{background-color:#fa00fa} +.gray{color:#606060} +.gray-background{background-color:#7d7d7d} +.green{color:#006000} +.green-background{background-color:#007d00} +.lime{color:#00bf00} +.lime-background{background-color:#00fa00} +.maroon{color:#600000} +.maroon-background{background-color:#7d0000} +.navy{color:#000060} +.navy-background{background-color:#00007d} +.olive{color:#606000} +.olive-background{background-color:#7d7d00} +.purple{color:#600060} +.purple-background{background-color:#7d007d} +.red{color:#bf0000} +.red-background{background-color:#fa0000} +.silver{color:#909090} +.silver-background{background-color:#bcbcbc} +.teal{color:#006060} +.teal-background{background-color:#007d7d} +.white{color:#bfbfbf} +.white-background{background-color:#fafafa} +.yellow{color:#bfbf00} +.yellow-background{background-color:#fafa00} +span.icon>.fa{cursor:default} +.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default} +.admonitionblock td.icon .icon-note:before{content:"\f05a";color:#19407c} +.admonitionblock td.icon .icon-tip:before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111} +.admonitionblock td.icon .icon-warning:before{content:"\f071";color:#bf6900} +.admonitionblock td.icon .icon-caution:before{content:"\f06d";color:#bf3400} +.admonitionblock td.icon .icon-important:before{content:"\f06a";color:#bf0000} +.conum[data-value]{display:inline-block;color:#fff!important;background-color:rgba(0,0,0,.8);-webkit-border-radius:100px;border-radius:100px;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold} +.conum[data-value] *{color:#fff!important} +.conum[data-value]+b{display:none} +.conum[data-value]:after{content:attr(data-value)} +pre .conum[data-value]{position:relative;top:-.125em} +b.conum *{color:inherit!important} +.conum:not([data-value]):empty{display:none} +h1,h2{letter-spacing:-.01em} +dt,th.tableblock,td.content{text-rendering:optimizeLegibility} +p,td.content{letter-spacing:-.01em} +p strong,td.content strong{letter-spacing:-.005em} +p,blockquote,dt,td.content{font-size:1.0625rem} +p{margin-bottom:1.25rem} +.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em} +.exampleblock>.content{background-color:#fffef7;border-color:#e0e0dc;-webkit-box-shadow:0 1px 4px #e0e0dc;box-shadow:0 1px 4px #e0e0dc} +.print-only{display:none!important} +@media print{@page{margin:1.25cm .75cm} +*{-webkit-box-shadow:none!important;box-shadow:none!important;text-shadow:none!important} +a{color:inherit!important;text-decoration:underline!important} +a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important} +a[href^="http:"]:not(.bare):after,a[href^="https:"]:not(.bare):after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em} +abbr[title]:after{content:" (" attr(title) ")"} +pre,blockquote,tr,img{page-break-inside:avoid} +thead{display:table-header-group} +img{max-width:100%!important} +p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3} +h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid} +#toc,.sidebarblock,.exampleblock>.content{background:none!important} +#toc{border-bottom:1px solid #ddddd8!important;padding-bottom:0!important} +.sect1{padding-bottom:0!important} +.sect1+.sect1{border:0!important} +#header>h1:first-child{margin-top:1.25rem} +body.book #header{text-align:center} +body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em 0} +body.book #header .details{border:0!important;display:block;padding:0!important} +body.book #header .details span:first-child{margin-left:0!important} +body.book #header .details br{display:block} +body.book #header .details br+span:before{content:none!important} +body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important} +body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always} +.listingblock code[data-lang]:before{display:block} +#footer{background:none!important;padding:0 .9375em} +#footer-text{color:rgba(0,0,0,.6)!important;font-size:.9em} +.hide-on-print{display:none!important} +.print-only{display:block!important} +.hide-for-print{display:none!important} +.show-for-print{display:inherit!important}} +</style> +<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.2.0/css/font-awesome.min.css"> +</head> +<body class="article"> +<div id="header"> +<h1>NiFi Developer’s Guide</h1> +<div class="details"> +<span id="author" class="author">Apache NiFi Team</span><br> +<span id="email" class="email"><a href="mailto:d...@nifi.apache.org">d...@nifi.apache.org</a></span><br> +</div> +<div id="toc" class="toc"> +<div id="toctitle">Table of Contents</div> +<ul class="sectlevel1"> +<li><a href="developer-guide.html#introduction">Introduction</a></li> +<li><a href="developer-guide.html#components">NiFi Components</a></li> +<li><a href="developer-guide.html#processor_api">Processor API</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#supporting_api">Supporting API</a></li> +<li><a href="developer-guide.html#AbstractProcessor">AbstractProcessor API</a></li> +<li><a href="developer-guide.html#component-lifecycle">Component Lifecycle</a></li> +<li><a href="developer-guide.html#component-notification">Component Notification</a></li> +<li><a href="developer-guide.html#restricted">Restricted</a></li> +<li><a href="developer-guide.html#state_manager">State Manager</a></li> +<li><a href="developer-guide.html#reporting-processor-activity">Reporting Processor Activity</a></li> +</ul> +</li> +<li><a href="developer-guide.html#documenting-a-component">Documenting a Component</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#documenting-properties">Documenting Properties</a></li> +<li><a href="developer-guide.html#documenting-relationships">Documenting Relationships</a></li> +<li><a href="developer-guide.html#documenting-capability-and-keywords">Documenting Capability and Keywords</a></li> +<li><a href="developer-guide.html#documenting-flowfile-attribute-interaction">Documenting FlowFile Attribute Interaction</a></li> +<li><a href="developer-guide.html#documenting-related-components">Documenting Related Components</a></li> +<li><a href="developer-guide.html#advanced-documentation">Advanced Documentation</a></li> +</ul> +</li> +<li><a href="developer-guide.html#provenance_events">Provenance Events</a></li> +<li><a href="developer-guide.html#common-processor-patterns">Common Processor Patterns</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#ingress">Data Ingress</a></li> +<li><a href="developer-guide.html#data-egress">Data Egress</a></li> +<li><a href="developer-guide.html#route-based-on-content-one-to-one">Route Based on Content (One-to-One)</a></li> +<li><a href="developer-guide.html#route-based-on-content-one-to-many">Route Based on Content (One-to-Many)</a></li> +<li><a href="developer-guide.html#route-streams-based-on-content-one-to-many">Route Streams Based on Content (One-to-Many)</a></li> +<li><a href="developer-guide.html#route-based-on-attributes">Route Based on Attributes</a></li> +<li><a href="developer-guide.html#split-content-one-to-many">Split Content (One-to-Many)</a></li> +<li><a href="developer-guide.html#update-attributes-based-on-content">Update Attributes Based on Content</a></li> +<li><a href="developer-guide.html#enrich-modify-content">Enrich/Modify Content</a></li> +</ul> +</li> +<li><a href="developer-guide.html#error-handling">Error Handling</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#exceptions-within-the-processor">Exceptions within the Processor</a></li> +<li><a href="developer-guide.html#exceptions-within-a-callback-ioexception-runtimeexception">Exceptions within a callback: IOException, RuntimeException</a></li> +<li><a href="developer-guide.html#penalization-vs-yielding">Penalization vs. Yielding</a></li> +<li><a href="developer-guide.html#session-rollback">Session Rollback</a></li> +</ul> +</li> +<li><a href="developer-guide.html#general-design-considerations">General Design Considerations</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#consider-the-user">Consider the User</a></li> +<li><a href="developer-guide.html#cohesion-and-reusability">Cohesion and Reusability</a></li> +<li><a href="developer-guide.html#naming-convensions">Naming Conventions</a></li> +<li><a href="developer-guide.html#processor-behavior-annotations">Processor Behavior Annotations</a></li> +<li><a href="developer-guide.html#data-buffering">Data Buffering</a></li> +</ul> +</li> +<li><a href="developer-guide.html#controller-services">Controller Services</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#developing-controller-service">Developing a ControllerService</a></li> +<li><a href="developer-guide.html#interacting-with-controller-service">Interacting with a ControllerService</a></li> +</ul> +</li> +<li><a href="developer-guide.html#reporting-tasks">Reporting Tasks</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#developing-a-reporting-task">Developing a Reporting Task</a></li> +</ul> +</li> +<li><a href="developer-guide.html#ui-extensions">UI Extensions</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#custom-processor-uis">Custom Processor UIs</a></li> +<li><a href="developer-guide.html#content-viewers">Content Viewers</a></li> +</ul> +</li> +<li><a href="developer-guide.html#command-line-tools">Command Line Tools</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#tls-toolkit">tls-toolkit</a></li> +</ul> +</li> +<li><a href="developer-guide.html#testing">Testing</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#instantiate-testrunner">Instantiate TestRunner</a></li> +<li><a href="developer-guide.html#add-controllerservices">Add ControllerServices</a></li> +<li><a href="developer-guide.html#set-property-values">Set Property Values</a></li> +<li><a href="developer-guide.html#enqueue-flowfiles">Enqueue FlowFiles</a></li> +<li><a href="developer-guide.html#run-the-processor">Run the Processor</a></li> +<li><a href="developer-guide.html#validate-output">Validate Output</a></li> +<li><a href="developer-guide.html#mocking-external-resources">Mocking External Resources</a></li> +<li><a href="developer-guide.html#additional-testing-capabilities">Additional Testing Capabilities</a></li> +</ul> +</li> +<li><a href="developer-guide.html#nars">NiFi Archives (NARs)</a></li> +<li><a href="developer-guide.html#per-instance-classloading">Per-Instance ClassLoading</a></li> +<li><a href="developer-guide.html#deprecation">Deprecating a Component</a></li> +<li><a href="developer-guide.html#how-to-contribute-to-apache-nifi">How to contribute to Apache NiFi</a> +<ul class="sectlevel2"> +<li><a href="developer-guide.html#technologies">Technologies</a></li> +<li><a href="developer-guide.html#where-to-start">Where to Start?</a></li> +<li><a href="developer-guide.html#supplying-a-contribution">Supplying a contribution</a></li> +<li><a href="developer-guide.html#contact-us">Contact Us</a></li> +</ul> +</li> +</ul> +</div> +</div> +<div id="content"> +<div class="sect1"> +<h2 id="introduction"><a class="anchor" href="developer-guide.html#introduction"></a>Introduction</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The intent of this Developer Guide is to provide the reader with the information needed to understand how Apache NiFi +extensions are developed and help to explain the thought process behind developing the components. It provides an introduction to +and explanation of the API that is used to develop extensions. It does not, however, go into great detail about each +of the methods in the API, as this guide is intended to supplement the JavaDocs of the API rather than replace them. +This guide also assumes that the reader is familiar with Java 7 and Apache Maven.</p> +</div> +<div class="paragraph"> +<p>This guide is written by developers for developers. It is expected that before reading this +guide, you have a basic understanding of NiFi and the concepts of dataflow. If not, please see the <a href="overview.html">NiFi Overview</a> +and the <a href="user-guide.html">NiFi User Guide</a> to familiarize yourself with the concepts of NiFi.</p> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="components"><a class="anchor" href="developer-guide.html#components"></a>NiFi Components</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>NiFi provides several extension points to provide developers the +ability to add functionality to the application to meet their needs. The following list provides a +high-level description of the most common extension points:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Processor</p> +<div class="ulist"> +<ul> +<li> +<p>The Processor interface is the mechanism through which NiFi exposes access to +<a href="developer-guide.html#flowfile">FlowFile</a>s, their attributes, and their content. The Processor is the basic building +block used to comprise a NiFi dataflow. This interface is used to accomplish +all of the following tasks:</p> +<div class="ulist"> +<ul> +<li> +<p>Create FlowFiles</p> +</li> +<li> +<p>Read FlowFile content</p> +</li> +<li> +<p>Write FlowFile content</p> +</li> +<li> +<p>Read FlowFile attributes</p> +</li> +<li> +<p>Update FlowFile attributes</p> +</li> +<li> +<p>Ingest data</p> +</li> +<li> +<p>Egress data</p> +</li> +<li> +<p>Route data</p> +</li> +<li> +<p>Extract data</p> +</li> +<li> +<p>Modify data</p> +</li> +</ul> +</div> +</li> +</ul> +</div> +</li> +<li> +<p>ReportingTask</p> +<div class="ulist"> +<ul> +<li> +<p>The ReportingTask interface is a mechanism that NiFi exposes to allow metrics, +monitoring information, and internal NiFi state to be published to external +endpoints, such as log files, e-mail, and remote web services.</p> +</li> +</ul> +</div> +</li> +<li> +<p>ControllerService</p> +<div class="ulist"> +<ul> +<li> +<p>A ControllerService provides shared state and functionality across Processors, other ControllerServices, +and ReportingTasks within a single JVM. An example use case may include loading a very +large dataset into memory. By performing this work in a ControllerService, the data +can be loaded once and be exposed to all Processors via this service, rather than requiring +many different Processors to load the dataset themselves.</p> +</li> +</ul> +</div> +</li> +<li> +<p>FlowFilePrioritizer</p> +<div class="ulist"> +<ul> +<li> +<p>The FlowFilePrioritizer interface provides a mechanism by which <a href="developer-guide.html#flowfile">FlowFile</a>s +in a queue can be prioritized, or sorted, so that the FlowFiles can be processed in an order +that is most effective for a particular use case.</p> +</li> +</ul> +</div> +</li> +<li> +<p>AuthorityProvider</p> +<div class="ulist"> +<ul> +<li> +<p>An AuthorityProvide is responsible for determining which privileges and roles, if any, +a given user should be granted.</p> +</li> +</ul> +</div> +</li> +</ul> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="processor_api"><a class="anchor" href="developer-guide.html#processor_api"></a>Processor API</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>The Processor is the most widely used Component available in NiFi. +Processors are the only Component +to which access is given to create, remove, modify, or inspect +FlowFiles (data and attributes).</p> +</div> +<div class="paragraph"> +<p>All Processors are loaded and instantiated using Java’s ServiceLoader +mechanism. This means that all +Processors must adhere to the following rules:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>The Processor must have a default constructor.</p> +</li> +<li> +<p>The Processor’s JAR file must contain an entry in the META-INF/services directory named +<code>org.apache.nifi.processor.Processor</code>. This is a text file where each line contains the +fully-qualified class name of a Processor.</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>While <code>Processor</code> is an interface that can be implemented directly, it +will be extremely rare to do so, as +the <code>org.apache.nifi.processor.AbstractProcessor</code> is the base class +for almost all Processor implementations. The <code>AbstractProcessor</code> class provides a significant +amount of functionality, which makes the task of developing a Processor much easier and more convenient. +For the scope of this document, we will focus primarily on the <code>AbstractProcessor</code> class when dealing +with the Processor API.</p> +</div> +<div class="paragraph"> +<div class="title">Concurrency Note</div> +<p>NiFi is a highly concurrent framework. This means that all extensions +must be thread-safe. If unfamiliar with writing concurrent software in Java, it is highly +recommended that you familiarize yourself with the principles of Java concurrency.</p> +</div> +<div class="sect2"> +<h3 id="supporting_api"><a class="anchor" href="developer-guide.html#supporting_api"></a>Supporting API</h3> +<div class="paragraph"> +<p>In order to understand the Processor API, we must first understand - +at least at a high level - several supporting classes and interfaces, which are discussed below.</p> +</div> +<div class="sect3"> +<h4 id="flowfile"><a class="anchor" href="developer-guide.html#flowfile"></a>FlowFile</h4> +<div class="paragraph"> +<p>A FlowFile is a logical notion that correlates a piece of data with a +set of Attributes about that data. +Such attributes include a FlowFile’s unique identifier, as well as its +name, size, and any number of other +flow-specific values. While the contents and attributes of a FlowFile +can change, the FlowFile object is +immutable. Modifications to a FlowFile are made possible by the ProcessSession.</p> +</div> +<div class="paragraph"> +<p>The core attributes for FlowFiles are defined in the <code>org.apache.nifi.flowfile.attributes.CoreAttributes</code> enum. +The most common attributes you’ll see are filename, path and uuid. The string in quotes is the value of the +attribute within the <code>CoreAttributes</code> enum.</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>Filename ("filename"): The filename of the FlowFile. The filename should not contain any directory structure.</p> +</li> +<li> +<p>UUID ("uuid"): A unique universally unique identifier (UUID) assigned to this FlowFile.</p> +</li> +<li> +<p>Path ("path"): The FlowFile’s path indicates the relative directory to which a FlowFile belongs and does not contain the filename.</p> +</li> +<li> +<p>Absolute Path ("absolute.path"): The FlowFile’s absolute path indicates the absolute directory to which a FlowFile belongs and does not contain the filename.</p> +</li> +<li> +<p>Priority ("priority"): A numeric value indicating the FlowFile priority.</p> +</li> +<li> +<p>MIME Type ("mime.type"): The MIME Type of this FlowFile.</p> +</li> +<li> +<p>Discard Reason ("discard.reason"): Specifies the reason that a FlowFile is being discarded.</p> +</li> +<li> +<p>Alternative Identifier ("alternate.identifier"): Indicates an identifier other than the FlowFile’s UUID that is known to refer to this FlowFile.</p> +</li> +</ul> +</div> +</div> +<div class="sect3"> +<h4 id="process_session"><a class="anchor" href="developer-guide.html#process_session"></a>ProcessSession</h4> +<div class="paragraph"> +<p>The ProcessSession, often referred to as simply a "session," provides +a mechanism by which FlowFiles can be created, destroyed, examined, cloned, and transferred to other +Processors. Additionally, a ProcessSession provides mechanism for creating modified versions of +FlowFiles, by adding or removing attributes, or by modifying the FlowFile’s content. The ProcessSession +also exposes a mechanism for emitting <a href="developer-guide.html#provenance_events">Provenance Events</a> that provide for the ability to track the +lineage and history of a FlowFile. After operations are performed on one or more FlowFiles, a +ProcessSession can be either committed or rolled back.</p> +</div> +</div> +<div class="sect3"> +<h4 id="process_context"><a class="anchor" href="developer-guide.html#process_context"></a>ProcessContext</h4> +<div class="paragraph"> +<p>The ProcessContext provides a bridge between a Processor and the framework. It provides information +about how the Processor is currently configured and allows the Processor to perform +Framework-specific tasks, such as yielding its resources so that the framework will schedule other +Processors to run without consuming resources unnecessarily.</p> +</div> +</div> +<div class="sect3"> +<h4 id="property_descriptor"><a class="anchor" href="developer-guide.html#property_descriptor"></a>PropertyDescriptor</h4> +<div class="paragraph"> +<p>PropertyDescriptor defines a property that is to be used by a +Processor, ReportingTask, or ControllerService. +The definition of a property includes its name, a description of the +property, an optional default value, +validation logic, and an indicator as to whether or not the property +is required in order for the Processor +to be valid. PropertyDescriptors are created by instantiating an +instance of the <code>PropertyDescriptor.Builder</code> +class, calling the appropriate methods to fill in the details about +the property, and finally calling +the <code>build</code> method.</p> +</div> +</div> +<div class="sect3"> +<h4 id="validator"><a class="anchor" href="developer-guide.html#validator"></a>Validator</h4> +<div class="paragraph"> +<p>A PropertyDescriptor MUST specify one or more Validators that can be +used to ensure that the user-entered value +for a property is valid. If a Validator indicates that a property +value is invalid, the Component will not be +able to be run or used until the property becomes valid. If a +Validator is not specified, the Component will be assumed invalid and +NiFi will report that the property is not supported.</p> +</div> +</div> +<div class="sect3"> +<h4 id="validation_context"><a class="anchor" href="developer-guide.html#validation_context"></a>ValidationContext</h4> +<div class="paragraph"> +<p>When validating property values, a ValidationContext can be used to +obtain ControllerServices, +create PropertyValue objects, and compile and evaluate property values +using the Expression Language.</p> +</div> +</div> +<div class="sect3"> +<h4 id="property_value"><a class="anchor" href="developer-guide.html#property_value"></a>PropertyValue</h4> +<div class="paragraph"> +<p>All property values returned to a Processor are returned in the form +of a PropertyValue object. This +object has convenience methods for converting the value from a String +to other forms, such as numbers +and time periods, as well as providing an API for evaluating the +Expression Language.</p> +</div> +</div> +<div class="sect3"> +<h4 id="relationship"><a class="anchor" href="developer-guide.html#relationship"></a>Relationship</h4> +<div class="paragraph"> +<p>Relationships define the routes to which a FlowFile may be transfered +from a Processor. Relationships +are created by instantiating an instance of the <code>Relationship.Builder</code> +class, calling the appropriate methods +to fill in the details of the Relationship, and finally calling the +<code>build</code> method.</p> +</div> +</div> +<div class="sect3"> +<h4 id="supporting_api_state_manager"><a class="anchor" href="developer-guide.html#supporting_api_state_manager"></a>StateManager</h4> +<div class="paragraph"> +<p>The StateManager provides Processors, Reporting Tasks, and Controller Services a mechanism +for easily storing and retrieving state. The API is similar to that of ConcurrentHashMap +but requires a Scope for each operation. The Scope indicates whether the state is to be +retrieved/stored locally or in a cluster-wide manner. For more information, see the +<a href="developer-guide.html#state_manager">State Manager</a> section.</p> +</div> +</div> +<div class="sect3"> +<h4 id="processor_initialization_context"><a class="anchor" href="developer-guide.html#processor_initialization_context"></a>ProcessorInitializationContext</h4> +<div class="paragraph"> +<p>After a Processor is created, its <code>initialize</code> method will be called +with an <code>InitializationContext</code> object. +This object exposes configuration to the Processor that will not +change throughout the life of the Processor, +such as the unique identifier of the Processor.</p> +</div> +</div> +<div class="sect3"> +<h4 id="ComponentLog"><a class="anchor" href="developer-guide.html#ComponentLog"></a>ComponentLog</h4> +<div class="paragraph"> +<p>Processors are encouraged to perform their logging via the +<code>ComponentLog</code> interface, rather than obtaining +a direct instance of a third-party logger. This is because logging via +the ComponentLog allows the framework +to render log messages that exceeds a configurable severity level to +the User Interface, allowing those who +monitor the dataflow to be notified when important events occur. +Additionally, it provides a consistent logging +format for all Processors by logging stack traces when in DEBUG mode +and providing the Processor’s unique +identifier in log messages.</p> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="AbstractProcessor"><a class="anchor" href="developer-guide.html#AbstractProcessor"></a>AbstractProcessor API</h3> +<div class="paragraph"> +<p>Since the vast majority of Processors will be created by extending the +AbstractProcessor, it is the +abstract class that we will examine in this section. The +AbstractProcessor provides several methods that +will be of interest to Processor developers.</p> +</div> +<div class="sect3"> +<h4 id="processor-initialization"><a class="anchor" href="developer-guide.html#processor-initialization"></a>Processor Initialization</h4> +<div class="paragraph"> +<p>When a Processor is created, before any other methods are invoked, the +<code>init</code> method of the +AbstractProcessor will be invoked. The method takes a single argument, +which is of type +<code>ProcessorInitializationContext</code>. The context object supplies the +Processor with a ComponentLog, +the Processor’s unique identifier, and a ControllerServiceLookup that +can be used to interact with the +configured ControllerServices. Each of these objects is stored by the +AbstractProcessor and may be obtained by +subclasses via the <code>getLogger</code>, <code>getIdentifier</code>, and +<code>getControllerServiceLookup</code> methods, respectively.</p> +</div> +</div> +<div class="sect3"> +<h4 id="exposing-processor-s-relationships"><a class="anchor" href="developer-guide.html#exposing-processor-s-relationships"></a>Exposing Processor’s Relationships</h4> +<div class="paragraph"> +<p>In order for a Processor to transfer a FlowFile to a new destination +for follow-on processing, the +Processor must first be able to expose to the Framework all of the +Relationships that it currently supports. +This allows users of the application to connect Processors to one +another by creating +Connections between Processors and assigning the appropriate +Relationships to those Connections.</p> +</div> +<div class="paragraph"> +<p>A Processor exposes the valid set of Relationships by overriding the +<code>getRelationships</code> method. +This method takes no arguments and returns a <code>Set</code> of <code>Relationship</code> +objects. For most Processors, this Set +will be static, but other Processors will generate the Set +dynamically, based on user configuration. +For those Processors for which the Set is static, it is advisable to +create an immutable Set in the Processor’s +constructor or init method and return that value, rather than +dynamically generating the Set. This +pattern lends itself to cleaner code and better performance.</p> +</div> +</div> +<div class="sect3"> +<h4 id="exposing-processor-properties"><a class="anchor" href="developer-guide.html#exposing-processor-properties"></a>Exposing Processor Properties</h4> +<div class="paragraph"> +<p>Most Processors will require some amount of user configuration before +they are able to be used. The properties +that a Processor supports are exposed to the Framework via the +<code>getSupportedPropertyDescriptors</code> method. +This method takes no arguments and returns a <code>List</code> of +<code>PropertyDescriptor</code> objects. The order of the objects in the +List is important in that it dictates the order in which the +properties will be rendered in the User Interface.</p> +</div> +<div class="paragraph"> +<p>A <code>PropertyDescriptor</code> object is constructed by creating a new +instance of the <code>PropertyDescriptor.Builder</code> object, +calling the appropriate methods on the builder, and finally calling +the <code>build</code> method.</p> +</div> +<div class="paragraph"> +<p>While this method covers most of the use cases, it is sometimes +desirable to allow users to configure +additional properties whose name are not known. This can be achieved +by overriding the +<code>getSupportedDynamicPropertyDescriptor</code> method. This method takes a +<code>String</code> as its only argument, which +indicates the name of the property. The method returns a +<code>PropertyDescriptor</code> object that can be used to validate +both the name of the property, as well as the value. Any +PropertyDescriptor that is returned from this method +should be built setting the value of <code>isDynamic</code> to true in the +<code>PropertyDescriptor.Builder</code> class. The default +behavior of AbstractProcessor is to not allow any dynamically created +properties.</p> +</div> +</div> +<div class="sect3"> +<h4 id="validating-processor-properties"><a class="anchor" href="developer-guide.html#validating-processor-properties"></a>Validating Processor Properties</h4> +<div class="paragraph"> +<p>A Processor is not able to be started if its configuration is not +valid. Validation of a Processor property can +be achieved by setting a Validator on a PropertyDescriptor or by +restricting the allowable values for a +property via the PropertyDescriptor.Builder’s <code>allowableValues</code> method +or <code>identifiesControllerService</code> method.</p> +</div> +<div class="paragraph"> +<p>There are times, though, when validating a Processor’s properties +individually is not sufficient. For this purpose, +the AbstractProcessor exposes a <code>customValidate</code> method. The method +takes a single argument of type <code>ValidationContext</code>. +The return value of this method is a <code>Collection</code> of +<code>ValidationResult</code> objects that describe any problems that were +found during validation. Only those ValidationResult objects whose +<code>isValid</code> method returns <code>false</code> should be returned. +This method will be invoked only if all properties are valid according +to their associated Validators and Allowable Values. +I.e., this method will be called only if all properties are valid +in-and-of themselves, and this method allows for +validation of a Processor’s configuration as a whole.</p> +</div> +</div> +<div class="sect3"> +<h4 id="responding-to-changes-in-configuration"><a class="anchor" href="developer-guide.html#responding-to-changes-in-configuration"></a>Responding to Changes in Configuration</h4> +<div class="paragraph"> +<p>It is sometimes desirable to have a Processor eagerly react when its +properties are changed. The <code>onPropertyModified</code> +method allows a Processor to do just that. When a user changes the +property values for a Processor, the +<code>onPropertyModified</code> method will be called for each modified property. +The method takes three arguments: the PropertyDescriptor that +indicates which property was modified, +the old value, and the new value. If the property had no previous +value, the second argument will be <code>null</code>. If the property +was removed, the third argument will be <code>null</code>. It is important to +note that this method will be called regardless of whether +or not the values are valid. This method will be called only when a +value is actually modified, rather than being +called when a user updates a Processor without changing its value. At +the point that this method is invoked, it is guaranteed +that the thread invoking this method is the only thread currently +executing code in the Processor, unless the Processor itself +creates its own threads.</p> +</div> +</div> +<div class="sect3"> +<h4 id="performing-the-work"><a class="anchor" href="developer-guide.html#performing-the-work"></a>Performing the Work</h4> +<div class="paragraph"> +<p>When a Processor has work to do, it is scheduled to do so by having +its <code>onTrigger</code> method called by the framework. +The method takes two arguments: a <code>ProcessContext</code> and a +<code>ProcessSession</code>. The first step in the <code>onTrigger</code> method +is often to obtain a FlowFile on which the work is to be performed by +calling one of the <code>get</code> methods on the ProcessSession. +For Processors that ingest data into NiFi from external sources, this +step is skipped. The Processor is then free to examine +FlowFile attributes; add, remove, or modify attributes; read or modify +FlowFile content; and transfer FlowFiles to the appropriate +Relationships.</p> +</div> +</div> +<div class="sect3"> +<h4 id="when-processors-are-triggered"><a class="anchor" href="developer-guide.html#when-processors-are-triggered"></a>When Processors are Triggered</h4> +<div class="paragraph"> +<p>A Processor’s <code>onTrigger</code> method will be called only when it is +scheduled to run and when work exists for the Processor. +Work is said to exist for a Processor if any of the following conditions is met:</p> +</div> +<div class="ulist"> +<ul> +<li> +<p>A Connection whose destination is the Processor has at least one +FlowFile in its queue</p> +</li> +<li> +<p>The Processors has no incoming Connections</p> +</li> +<li> +<p>The Processor is annotated with the @TriggerWhenEmpty annotation</p> +</li> +</ul> +</div> +<div class="paragraph"> +<p>Several factors exist that will contribute to when a Processor’s +<code>onTrigger</code> method is invoked. First, the Processor will not +be triggered unless a user has configured the Processor to run. If a +Processor is scheduled to run, the Framework periodically +(the period is configured by users in the User Interface) checks if +there is work for the Processor to do, as described above. +If so, the Framework will check downstream destinations of the +Processor. If any of the Processor’s outbound Connections is full, +by default, the Processor will not be scheduled to run.</p> +</div> +<div class="paragraph"> +<p>However, the <code>@TriggerWhenAnyDestinationAvailable</code> annotation may be +added to the Processor’s class. In this case, the requirement +is changed so that only one downstream destination must be "available" +(a destination is considered "available" if the Connection’s +queue is not full), rather than requiring that all downstream +destinations be available.</p> +</div> +<div class="paragraph"> +<p>Also related to Processor scheduling is the <code>@TriggerSerially</code> +annotation. Processors that use this Annotation will never have more +than one thread running the <code>onTrigger</code> method simultaneously. It is +crucial to note, though, that the thread executing the code +may change from invocation to invocation. Therefore, care must still +be taken to ensure that the Processor is thread-safe!</p> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="component-lifecycle"><a class="anchor" href="developer-guide.html#component-lifecycle"></a>Component Lifecycle</h3> +<div class="paragraph"> +<p>The NiFi API provides lifecycle support through use of Java +Annotations. The <code>org.apache.nifi.annotations.lifecycle</code> package +contains +several annotations for lifecycle management. The following +Annotations may be applied to Java methods in a NiFi component to +indicate to +the framework when the methods should be called. For the discussion of +Component Lifecycle, we will define a NiFi component as a +Processor, ControllerServices, or ReportingTask.</p> +</div> +<div class="sect3"> +<h4 id="onadded"><a class="anchor" href="developer-guide.html#onadded"></a>@OnAdded</h4> +<div class="paragraph"> +<p>The <code>@OnAdded</code> annotation causes a method to be invoked as soon as a +component is created. The +component’s <code>initialize</code> method (or <code>init</code> method, if subclasses +<code>AbstractProcessor</code>) will be invoked after the component is +constructed, +followed by methods that are annotated with <code>@OnAdded</code>. If any method +annotated with <code>@OnAdded</code> throws an Exception, an error will +be returned to the user, and that component will not be added to the +flow. Furthermore, other methods with this +Annotation will not be invoked. This method will be called only once +for the lifetime of a component. +Methods with this Annotation must take zero arguments.</p> +</div> +</div> +<div class="sect3"> +<h4 id="onenabled"><a class="anchor" href="developer-guide.html#onenabled"></a>@OnEnabled</h4> +<div class="paragraph"> +<p>The <code>@OnEnabled</code> annotation can be used to indicate a method should be called +whenever the Controller Service is enabled. Any method that has this annotation will be +called every time a user enables the service. Additionally, each time that NiFi +is restarted, if NiFi is configured to "auto-resume state" and the service +is enabled, the method will be invoked.</p> +</div> +<div class="paragraph"> +<p>If a method with this annotation throws a Throwable, a log message and +bulletin will be issued for the component. In this event, the service will +remain in an <em>ENABLING</em> state and will not be usable. All methods with this +annotation will then be called again after a delay. The service will not be +made available for use until all methods with this annotation have returned +without throwing anything.</p> +</div> +<div class="paragraph"> +<p>Methods using this annotation must take either 0 arguments or a single argument +of type <code>org.apache.nifi.controller.ConfigurationContext</code>.</p> +</div> +<div class="paragraph"> +<p>Note that this annotation will be ignored if applied to a ReportingTask or +Processor. For a Controller Service, enabling and disabling are considered +lifecycle events, as the action makes them usable or unusable by other +components. However, for a Processor and a Reporting Task, these are not +lifecycle events but rather a mechanism to allow a component to be excluded +when starting or stopping a group of components.</p> +</div> +</div> +<div class="sect3"> +<h4 id="onremoved"><a class="anchor" href="developer-guide.html#onremoved"></a>@OnRemoved</h4> +<div class="paragraph"> +<p>The <code>@OnRemoved</code> annotation causes a method to be invoked before a +component is removed from the flow. +This allows resources to be cleaned up before removing a component. +Methods with this annotation must take zero arguments. +If a method with this annotation throws an Exception, the component +will still be removed.</p> +</div> +</div> +<div class="sect3"> +<h4 id="onscheduled"><a class="anchor" href="developer-guide.html#onscheduled"></a>@OnScheduled</h4> +<div class="paragraph"> +<p>This annotation indicates that a method should be called every time +the component is scheduled to run. Because ControllerServices +are not scheduled, using this annotation on a ControllerService does +not make sense and will not be honored. It should be +used only for Processors and Reporting Tasks. If any method with this +annotation throws an Exception, other methods with this +annotation will not be invoked, and a notification will be presented +to the user. In this case, methods annotated with +<code>@OnUnscheduled</code> are then triggered, followed by methods with the +<code>@OnStopped</code> annotation (during this state, if any of these +methods throws an Exception, those Exceptions are ignored). The +component will then yield its execution for some period of time, +referred to as the "Administrative Yield Duration," which is a value +that is configured in the <code>nifi.properties</code> file. Finally, the +process will start again, until all of the methods annotated with +<code>@OnScheduled</code> have returned without throwing any Exception. +Methods with this annotation may take zero arguments or may take a +single argument. If the single argument variation is used, +the argument must be of type <code>ProcessContext</code> if the component is a +Processor or <code>ConfigurationContext</code> if the component +is a ReportingTask.</p> +</div> +</div> +<div class="sect3"> +<h4 id="onunscheduled"><a class="anchor" href="developer-guide.html#onunscheduled"></a>@OnUnscheduled</h4> +<div class="paragraph"> +<p>Methods with this annotation will be called whenever a Processor or +ReportingTask is no longer scheduled to run. At that time, many threads +may still be active in the Processor’s <code>onTrigger</code> method. If such a method +throws an Exception, a log message will be generated, and the +Exception will be otherwise +ignored and other methods with this annotation will still be invoked. +Methods with this annotation may take zero arguments or may take a +single argument. +If the single argument variation is used, the argument must be of type +<code>ProcessContext</code> if the component is a Processor or +<code>ConfigurationContext</code> if the +component is a ReportingTask.</p> +</div> +</div> +<div class="sect3"> +<h4 id="onstopped"><a class="anchor" href="developer-guide.html#onstopped"></a>@OnStopped</h4> +<div class="paragraph"> +<p>Methods with this annotation will be called when a Processor or +ReportingTask is no longer scheduled to run +and all threads have returned from the <code>onTrigger</code> method. If such a +method throws an Exception, +a log message will be generated, and the Exception will otherwise be +ignored; other methods with +this annotation will still be invoked. +Methods with this annotation are permitted to take either 0 or 1 argument. If +an argument is used, it must be of type ConfigurationContext if the +component is a ReportingTask or of type ProcessContext if the +component is a Processor.</p> +</div> +</div> +<div class="sect3"> +<h4 id="onshutdown"><a class="anchor" href="developer-guide.html#onshutdown"></a>@OnShutdown</h4> +<div class="paragraph"> +<p>Any method that is annotated with the <code>@OnShutdown</code> annotation will be +called when NiFi is successfully +shut down. If such a method throws an Exception, a log message will be +generated, and the +Exception will be otherwise ignored and other methods with this +annotation will still be invoked. +Methods with this annotation must take zero arguments. Note: while +NiFi will attempt to invoke methods +with this annotation on all components that use it, this is not always +possible. For example, the process +may be killed unexpectedly, in which case it does not have a chance to +invoke these methods. Therefore, +while methods using this annotation can be used to clean up resources, +for instance, they should not be +relied upon for critical business logic.</p> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="component-notification"><a class="anchor" href="developer-guide.html#component-notification"></a>Component Notification</h3> +<div class="paragraph"> +<p>The NiFi API provides notification support through use of Java +Annotations. The <code>org.apache.nifi.annotations.notification</code> package +contains several annotations for notification management. The following +annotations may be applied to Java methods in a NiFi component to +indicate to the framework when the methods should be called. For the +discussion of Component Notification, we will define a NiFi component +as a <strong>Processor</strong>, <strong>Controller Service</strong>, or <strong>Reporting Task</strong>.</p> +</div> +<div class="sect3"> +<h4 id="onprimarynodestatechange"><a class="anchor" href="developer-guide.html#onprimarynodestatechange"></a>@OnPrimaryNodeStateChange</h4> +<div class="paragraph"> +<p>The <code>@OnPrimaryNodeStateChange</code> annotation causes a method to be invoked +as soon as the state of the Primary Node in a cluster has changed. +Methods with this annotation should take either no arguments or one +argument of type <code>PrimaryNodeState</code>. The <code>PrimaryNodeState</code> provides +context about what changed so that the component can take appropriate +action. The <code>PrimaryNodeState</code> enumerator has two possible values: +<code>ELECTED_PRIMARY_NODE</code> (the node receiving this +state has been elected the Primary Node of the NiFi cluster), or +<code>PRIMARY_NODE_REVOKED</code> (the node receiving this state was the Primary +Node but has now had its Primary Node role revoked).</p> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="restricted"><a class="anchor" href="developer-guide.html#restricted"></a>Restricted</h3> +<div class="paragraph"> +<p>A Restricted component is one that can be used to execute arbitrary unsanitized code provided by the operator +through the NiFi REST API/UI or can be used to obtain or alter data on the NiFi host system using the NiFi OS +credentials. These components could be used by an otherwise authorized NiFi user to go beyond the intended use of +the application, escalate privilege, or could expose data about the internals of the NiFi process or the host +system. All of these capabilities should be considered privileged, and admins should be aware of these +capabilities and explicitly enable them for a subset of trusted users.</p> +</div> +<div class="paragraph"> +<p>A Processor, Controller Service, or Reporting Task can be marked with the @Restricted annotation. This +will result in the component being treated as restricted and will require a user to be explicitly added to the +list of users who can access restricted components. Once a user is permitted to access restricted components, +they will be allowed to create and modify those components assuming all other permissions are permitted. +Without access to restricted components, a user will be still be aware these types of components exist but will +be unable to create or modify them even with otherwise sufficient permissions.</p> +</div> +</div> +<div class="sect2"> +<h3 id="state_manager"><a class="anchor" href="developer-guide.html#state_manager"></a>State Manager</h3> +<div class="paragraph"> +<p>From the ProcessContext, ReportingContext, and ControllerServiceInitializationContext, components are +able to call the <code>getStateManager()</code> method. This State Manager is responsible for providing a simple API +for storing and retrieving state. This mechanism is intended to provide developers with the ability to +very easily store a set of key/value pairs, retrieve those values, and update them atomically. The state +can be stored local to the node or across all nodes in a cluster. It is important to note, however, that +this mechanism is intended only to provide a mechanism for storing very <em>simple</em> state. As such, the API +simply allows a <code>Map<String, String></code> to be stored and retrieved and for the entire Map to be atomically +replaced. Moreover, the only implementation that is currently supported for storing cluster-wide state is +backed by ZooKeeper. As such, the entire State Map must be less than 1 MB in size, after being serialized. +Attempting to store more than this will result in an Exception being thrown. If the interactions required +by the Processor for managing state are more complex than this (e.g., large amounts of data must be stored +and retrieved, or individual keys must be stored and fetched individually) than a different mechanism should +be used (e.g., communicating with an external database).</p> +</div> +<div class="sect3"> +<h4 id="state_scope"><a class="anchor" href="developer-guide.html#state_scope"></a>Scope</h4> +<div class="paragraph"> +<p>When communicating with the State Manager, all method calls require that a Scope be provided. This Scope will +either be <code>Scope.LOCAL</code> or <code>Scope.CLUSTER</code>. If NiFi is run in a cluster, this Scope provides important information +to the framework about how the operation should occur.</p> +</div> +<div class="paragraph"> +<p>If state as stored using <code>Scope.CLUSTER</code>, then all nodes in the cluster will be communicating with the same +state storage mechanism. If state is stored and retrieved using <code>Scope.LOCAL</code>, then each node will see a different +representation of the state.</p> +</div> +<div class="paragraph"> +<p>It is also worth noting that if NiFi is configured to run as a standalone instance, rather than running in a cluster, +a scope of <code>Scope.LOCAL</code> is always used. This is done in order to allow the developer of a NiFi component to write the code +in one consistent way, without worrying about whether or not the NiFi instance is clustered. The developer should instead assume +that the instance is clustered and write the code accordingly.</p> +</div> +</div> +<div class="sect3"> +<h4 id="storing-and-retrieving-state"><a class="anchor" href="developer-guide.html#storing-and-retrieving-state"></a>Storing and Retrieving State</h4> +<div class="paragraph"> +<p>State is stored using the StateManager’s <code>getState</code>, <code>setState</code>, <code>replace</code>, and <code>clear</code> methods. All of these methods +require that a Scope be provided. It should be noted that the state that is stored with the Local scope is entirely different +than state stored with a Cluster scope. If a Processor stores a value with the key of <em>My Key</em> using the <code>Scope.CLUSTER</code> scope, +and then attempts to retrieve the value using the <code>Scope.LOCAL</code> scope, the value retrieved will be <code>null</code> (unless a value was +also stored with the same key using the <code>Scope.CLUSTER</code> scope). Each Processor’s state, is stored in isolation from other +Processors' state.</p> +</div> +<div class="paragraph"> +<p>It follows, then, that two Processors cannot share the same state. There are, however, some circumstances in which it is very +necessary to share state between two Processors of different types, or two Processors of the same type. This can be accomplished +by using a Controller Service. By storing and retrieving state from a Controller Service, multiple Processors can use the same +Controller Service and the state can be exposed via the Controller Service’s API.</p> +</div> +</div> +<div class="sect3"> +<h4 id="unit-tests"><a class="anchor" href="developer-guide.html#unit-tests"></a>Unit Tests</h4> +<div class="paragraph"> +<p>NiFi’s Mock Framework provides an extensive collection of tools to perform unit testing of Processors. Processor unit tests typically +begin with the <code>TestRunner</code> class. As a result, the <code>TestRunner</code> class contains a <code>getStateManager</code> method of its own. The StateManager +that is returned, however, is of a specific type: <code>MockStateManager</code>. This implementation provides several methods in addition to those +defined by the <code>StateManager</code> interface, that help developers to more easily develop unit tests.</p> +</div> +<div class="paragraph"> +<p>First, the <code>MockStateManager</code> implements the <code>StateManager</code> interface, so all of the state can be examined from within a unit test. +Additionally, the <code>MockStateManager</code> exposes a handful of <code>assert*</code> methods to perform assertions that the State is set as expected. +The <code>MockStateManager</code> also provides the ability to indicate that the unit test should immediately fail if state is updated for a particular +<code>Scope</code>.</p> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="reporting-processor-activity"><a class="anchor" href="developer-guide.html#reporting-processor-activity"></a>Reporting Processor Activity</h3> +<div class="paragraph"> +<p>Processors are responsible for reporting their activity so that users +are able to understand what happens +to their data. Processors should log events via the ComponentLog, +which is accessible via the InitializationContext +or by calling the <code>getLogger</code> method of <code>AbstractProcessor</code>.</p> +</div> +<div class="paragraph"> +<p>Additionally, Processors should use the <code>ProvenanceReporter</code> +interface, obtained via the ProcessSession’s +<code>getProvenanceReporter</code> method. The ProvenanceReporter should be used +to indicate any time that content is +received from an external source or sent to an external location. The +ProvenanceReporter also has methods for +reporting when a FlowFile is cloned, forked, or modified, and when +multiple FlowFiles are merged into a single FlowFile +as well as associating a FlowFile with some other identifier. However, +these functions are less critical to report, as +the framework is able to detect these things and emit appropriate +events on the Processor’s behalf. Yet, it is a best practice +for the Processor developer to emit these events, as it becomes +explicit in the code that these events are being emitted, and +the developer is able to provide additional details to the events, +such as the amount of time that the action took or +pertinent information about the action that was taken. If the +Processor emits an event, the framework will not emit a duplicate +event. Instead, it always assumes that the Processor developer knows +what is happening in the context of the Processor +better than the framework does. The framework may, however, emit a +different event. For example, if a Processor modifies both the +content of a FlowFile and its attributes and then emits only an +ATTRIBUTES_MODIFIED event, the framework will emit a CONTENT_MODIFIED +event. The framework will not emit an ATTRIBUTES_MODIFIED event if any +other event is emitted for that FlowFile (either by the +Processor or the framework). This is due to the fact that all +<a href="developer-guide.html#provenance_events">Provenance Events</a> know about the attributes of the FlowFile before the +event occurred as well as those attributes that occurred as a result +of the processing of that FlowFile, and as a result the +ATTRIBUTES_MODIFIED is generally considered redundant and would result +in a rendering of the FlowFile lineage being very verbose. +It is, however, acceptable for a Processor to emit this event along +with others, if the event is considered pertinent from the +perspective of the Processor.</p> +</div> +</div> +</div> +</div> +<div class="sect1"> +<h2 id="documenting-a-component"><a class="anchor" href="developer-guide.html#documenting-a-component"></a>Documenting a Component</h2> +<div class="sectionbody"> +<div class="paragraph"> +<p>NiFi attempts to make the user experience as simple and convenient as +possible by providing significant amount of documentation +to the user from within the NiFi application itself via the User +Interface. In order for this to happen, of course, Processor +developers must provide that documentation to the framework. NiFi +exposes a few different mechanisms for supplying documentation to +the framework.</p> +</div> +<div class="sect2"> +<h3 id="documenting-properties"><a class="anchor" href="developer-guide.html#documenting-properties"></a>Documenting Properties</h3> +<div class="paragraph"> +<p>Individual properties can be documented by calling the <code>description</code> +method of a PropertyDescriptor’s builder as such:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code class="language-java" data-lang="java">public static final PropertyDescriptor MY_PROPERTY = new PropertyDescriptor.Builder() + .name("My Property") + .description("Description of the Property") + ... + .build();</code></pre> +</div> +</div> +<div class="paragraph"> +<p>If the property is to provide a set of allowable values, those values +are presented to the user in a drop-down field in the UI. +Each of those values can also be given a description:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code class="language-java" data-lang="java">public static final AllowableValue EXTENSIVE = new AllowableValue("Extensive", "Extensive", + "Everything will be logged - use with caution!"); +public static final AllowableValue VERBOSE = new AllowableValue("Verbose", "Verbose", + "Quite a bit of logging will occur"); +public static final AllowableValue REGULAR = new AllowableValue("Regular", "Regular", + "Typical logging will occur"); + +public static final PropertyDescriptor LOG_LEVEL = new PropertyDescriptor.Builder() + .name("Amount to Log") + .description("How much the Processor should log") + .allowableValues(REGULAR, VERBOSE, EXTENSIVE) + .defaultValue(REGULAR.getValue()) + ... + .build();</code></pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="documenting-relationships"><a class="anchor" href="developer-guide.html#documenting-relationships"></a>Documenting Relationships</h3> +<div class="paragraph"> +<p>Processor Relationships are documented in much the same way that +properties are - by calling the <code>description</code> method of a +Relationship’s builder:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code class="language-java" data-lang="java">public static final Relationship MY_RELATIONSHIP = new Relationship.Builder() + .name("My Relationship") + .description("This relationship is used only if the Processor fails to process the data.") + .build();</code></pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="documenting-capability-and-keywords"><a class="anchor" href="developer-guide.html#documenting-capability-and-keywords"></a>Documenting Capability and Keywords</h3> +<div class="paragraph"> +<p>The <code>org.apache.nifi.annotations.documentation</code> package provides Java +annotations that can be used to document components. The +CapabilityDescription +annotation can be added to a Processor, Reporting Task, or Controller +Service and is intended to provide a brief description of the +functionality +provided by the component. The Tags annotation has a <code>value</code> variable +that is defined to be an Array of Strings. As such, it is used +by providing multiple values as a comma-separated list of <code>String</code>s +with curly braces. These values are then incorporated into the UI by +allowing +users to filter the components based on a tag (i.e., a keyword). +Additionally, the UI provides a tag cloud that allows users to select +the tags that +they want to filter by. The tags that are largest in the cloud are +those tags that exist the most on the components in that instance of +NiFi. An +example of using these annotations is provided below:</p> +</div> +<div class="listingblock"> +<div class="content"> +<pre class="highlight"><code class="language-java" data-lang="java">@Tags({"example", "documentation", "developer guide", "processor", "tags"}) +@CapabilityDescription("Example Processor that provides no real functionality but is provided" + + " for an example in the Developer Guide") +public static final ExampleProcessor extends Processor { + ... +}</code></pre> +</div> +</div> +</div> +<div class="sect2"> +<h3 id="documenting-flowfile-attribute-interaction"><a class="anchor" href="developer-guide.html#documenting-flowfile-attribute-interaction"></a>Documenting FlowFile Attribute Interaction</h3> +<div class="paragraph"> +<p>Many times a processor will expect certain FlowFile attributes be set on in-bound FlowFiles in order +for the processor to function properly. In other cases a processor may update or +create FlowFile attributes on the out-bound FlowFile. Processor developers may document both of these +behaviors using the <code>ReadsAttribute</code> and <code>WritesAttribute</code> documentation annotations. These attributes are used to generate documentation +that gives users a better understanding of how a processor will interact with the flow.</p> +</div> +<div class="paragraph"> +<p>Note: Because Java 7 does not support +repeated annotations on a type, you may need to use <code>ReadsAttributes</code> and <code>WritesAttributes</code> to indicate +that a processor reads or writes multiple FlowFile attributes. This annotation can only be applied to Processors. An example is listed below:</p> +</div>
[... 2130 lines stripped ...]