Added: falcon/site/0.9/Recipes.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/Recipes.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/Recipes.html (added) +++ falcon/site/0.9/Recipes.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,175 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Falcon Recipes</title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Falcon Recipes</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h2>Falcon Recipes<a name="Falcon_Recipes"></a></h2></div> +<div class="section"> +<h3>Overview<a name="Overview"></a></h3> +<p>A Falcon recipe is a static process template with parameterized workflow to realize a specific use case. Recipes are defined in user space. Recipes will not have support for update or lifecycle management.</p> +<p>For example:</p> +<p></p> +<ul> +<li>Replicating directories from one HDFS cluster to another (not timed partitions)</li> +<li>Replicating hive metadata (database, table, views, etc.)</li> +<li>Replicating between HDFS and Hive - either way</li> +<li>Data masking etc.</li></ul></div> +<div class="section"> +<h3>Proposal<a name="Proposal"></a></h3> +<p>Falcon provides a Process abstraction that encapsulates the configuration for a user workflow with scheduling controls. All recipes can be modeled as a Process with in Falcon which executes the user workflow periodically. The process and its associated workflow are parameterized. The user will provide a properties file with name value pairs that are substituted by falcon before scheduling it. Falcon translates these recipes as a process entity by replacing the parameters in the workflow definition.</p></div> +<div class="section"> +<h3>Falcon CLI recipe support<a name="Falcon_CLI_recipe_support"></a></h3> +<p><a href="./FalconCLI.html">Recipe command usage is defined here.</a></p> +<p>CLI accepts recipe option with a recipe name and optional tool and does the following:</p> +<ul> +<li>Validates the options; name option is mandatory and tool is optional and should be provided if user wants to override the base recipe tool</li> +<li>Looks for <name>-workflow.xml, <name>-template.xml and <name>.properties file in the path specified by falcon.recipe.path in client.properties. If files cannot be found then Falcon CLI will fail</li> +<li>Invokes a Tool to substitute the properties in the templated process for the recipe. By default invokes base tool if tool option is not passed. Tool is responsible for generating process entity at the path specified by FalconCLI</li> +<li>Validates the generated entity</li> +<li>Submit and schedule this entity</li> +<li>Generated process entity files are stored in tmp directory</li></ul></div> +<div class="section"> +<h3>Base Recipe tool<a name="Base_Recipe_tool"></a></h3> +<p>Falcon provides a base tool that recipes can override. Base Recipe tool does the following:</p> +<ul> +<li>Expects recipe template file path, recipe properties file path and path where process entity to be submitted should be generated. Validates these arguments</li> +<li>Validates the artifacts i.e. workflow and/or lib files specified in the recipe template exists on local filesystem or HDFS at the specified path else returns error</li> +<li>Copies if the artifacts exists on local filesystem +<ul> +<li>If workflow is on local FS then falcon.recipe.workflow.path in recipe property file is mandatory for it to be copied to HDFS. If templated process requires custom libs falcon.recipe.workflow.lib.path property is mandatory for them to be copied from Local FS to HDFS. Recipe tool will copy the local artifacts only if these properties are set in properties file</li></ul></li> +<li>Looks for the patten ##[A-Za-z0-9_.]*## in the templated process and substitutes it with the properties. Process entity generated after the substitution is written to the empty file passed by FalconCLI</li></ul></div> +<div class="section"> +<h3>Recipe template file format<a name="Recipe_template_file_format"></a></h3> +<p></p> +<ul> +<li>Any templatized string should be in the format ##[A-Za-z0-9_.]*##.</li> +<li>There should be a corresponding entry in the recipe properties file "falcon.recipe.<templatized-string> = <value to be substituted>"</li></ul> +<div class="source"> +<pre> +Example: If the entry in recipe template is <workflow name="##workflow.name##"> there should be a corresponding entry in the recipe properties file falcon.recipe.workflow.name=hdfs-dr-workflow + +</pre></div></div> +<div class="section"> +<h3>Recipe properties file format<a name="Recipe_properties_file_format"></a></h3> +<p></p> +<ul> +<li>Regular key value pair properties file</li> +<li>Property key should be prefixed by "falcon.recipe."</li></ul> +<div class="source"> +<pre> +Example: falcon.recipe.workflow.name=hdfs-dr-workflow +Recipe template will have <workflow name="##workflow.name##">. Recipe tool will look for the patten ##workflow.name## +and replace it with the property value "hdfs-dr-workflow". Substituted template will have <workflow name="hdfs-dr-workflow"> + +</pre></div></div> +<div class="section"> +<h3>Metrics<a name="Metrics"></a></h3> +<p>HDFS DR and Hive DR recipes will capture the replication metrics like TIMETAKEN, BYTESCOPIED, COPY (number of files copied) for an instance and populate to the GraphDB.</p></div> +<div class="section"> +<h3>Managing the scheduled recipe process<a name="Managing_the_scheduled_recipe_process"></a></h3> +<p></p> +<ul> +<li>Scheduled recipe process is similar to regular process +<ul> +<li>List : falcon entity -type process -name <recipe-process-name> -list</li> +<li>Status : falcon entity -type process -name <recipe-process-name> -status</li> +<li>Delete : falcon entity -type process -name <recipe-process-name> -delete</li></ul></li></ul></div> +<div class="section"> +<h3>Sample recipes<a name="Sample_recipes"></a></h3> +<p></p> +<ul> +<li>Sample recipes are published in addons/recipes</li></ul></div> +<div class="section"> +<h3>Types of recipes<a name="Types_of_recipes"></a></h3> +<p></p> +<ul> +<li><a href="./HDFSDR.html">HDFS Recipe</a></li> +<li><a href="./HiveDR.html">HiveDR Recipe</a></li></ul></div> +<div class="section"> +<h3>Packaging<a name="Packaging"></a></h3> +<p></p> +<ul> +<li>There is no packaging for recipes at this time but will be added soon.</li></ul></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html>
Added: falcon/site/0.9/Security.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/Security.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/Security.html (added) +++ falcon/site/0.9/Security.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,510 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Securing Falcon</title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Securing Falcon</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h2>Securing Falcon<a name="Securing_Falcon"></a></h2></div> +<div class="section"> +<h3>Overview<a name="Overview"></a></h3> +<p>Apache Falcon enforces authentication and authorization which are detailed below. Falcon also provides transport level security ensuring data confidentiality and integrity.</p></div> +<div class="section"> +<h3>Authentication (User Identity)<a name="Authentication_User_Identity"></a></h3> +<p>Apache Falcon enforces authentication on protected resources. Once authentication has been established it sets a signed HTTP Cookie that contains an authentication token with the user name, user principal, authentication type and expiration time.</p> +<p>It does so by using <a class="externalLink" href="./Http://hadoop.apache.org/docs/current/hadoop-auth/index.html.html">Hadoop Auth</a>. Hadoop Auth is a Java library consisting of a client and a server components to enable Kerberos SPNEGO authentication for HTTP. Hadoop Auth also supports additional authentication mechanisms on the client and the server side via 2 simple interfaces.</p></div> +<div class="section"> +<h4>Authentication Methods<a name="Authentication_Methods"></a></h4> +<p>It supports 2 authentication methods, simple and kerberos out of the box.</p></div> +<div class="section"> +<h5>Pseudo/Simple Authentication<a name="PseudoSimple_Authentication"></a></h5> +<p>Falcon authenticates the user by simply trusting the value of the query string parameter 'user.name'. This is the default mode Falcon is configured with.</p></div> +<div class="section"> +<h5>Kerberos Authentication<a name="Kerberos_Authentication"></a></h5> +<p>Falcon uses HTTP Kerberos SPNEGO to authenticate the user.</p></div> +<div class="section"> +<h3>Authorization<a name="Authorization"></a></h3> +<p>Falcon also enforces authorization on Entities using ACLs (Access Control Lists). ACLs are useful for implementing permission requirements and provide a way to set different permissions for specific users or named groups.</p> +<p>By default, support for authorization is disabled and can be enabled in startup.properties.</p></div> +<div class="section"> +<h4>ACLs in Entity<a name="ACLs_in_Entity"></a></h4> +<p>All Entities now have ACL which needs to be present if authorization is enabled. Only owners who own or created the entity will be allowed to update or delete their entities.</p> +<p>An entity has ACLs (Access Control Lists) that are useful for implementing permission requirements and provide a way to set different permissions for specific users or named groups.</p> +<div class="source"> +<pre> + <ACL owner="test-user" group="test-group" permission="*"/> + +</pre></div> +<p>ACL indicates the Access control list for this cluster. owner is the Owner of this entity. group is the one which has access to read. permission indicates the rwx is not enforced at this time.</p></div> +<div class="section"> +<h4>Super-User<a name="Super-User"></a></h4> +<p>The super-user is the user with the same identity as falcon process itself. Loosely, if you started the falcon, then you are the super-user. The super-user can do anything in that permissions checks never fail for the super-user. There is no persistent notion of who was the super-user; when the falcon is started the process identity determines who is the super-user for now. The Falcon super-user does not have to be the super-user of the falcon host, nor is it necessary that all clusters have the same super-user. Also, an experimenter running Falcon on a personal workstation, conveniently becomes that installation's super-user without any configuration.</p> +<p>Falcon also allows users to configure a super user group and allows users belonging to this group to be a super user.</p> +<p>ACL owner and group must be valid even if the authenticated user is a super-user.</p></div> +<div class="section"> +<h4>Group Memberships<a name="Group_Memberships"></a></h4> +<p>Once a user has been authenticated and a username has been determined, the list of groups is determined by a group mapping service, configured by the hadoop.security.group.mapping property in Hadoop. The default implementation, org.apache.hadoop.security.ShellBasedUnixGroupsMapping, will shell out to the Unix bash -c groups command to resolve a list of groups for a user.</p> +<p>Note that Falcon stores the user and group of an Entity as strings; there is no conversion from user and group identity numbers as is conventional in Unix.</p> +<p>The only limitation is that a user cannot add a group in ACL that he does not belong to.</p></div> +<div class="section"> +<h4>Authorization Provider<a name="Authorization_Provider"></a></h4> +<p>Falcon provides a plugin-able provider interface for Authorization. It also ships with a default implementation that enforces the following authorization policy.</p></div> +<div class="section"> +<h5>Entity and Instance Management Operations Policy<a name="Entity_and_Instance_Management_Operations_Policy"></a></h5> +<p></p> +<ul> +<li>All Entity and Instance operations are authorized for users who created them, Owners and users with group memberships</li> +<li>Reference to entities with in a feed or process is allowed with out enforcing permissions</li></ul> +<p>Any Feed or Process can refer to a Cluster entity not owned by the Feed or Process owner. Any Process can refer to a Feed entity not owned by the Process owner</p> +<p>The authorization is enforced in the following way:</p> +<p></p> +<ul> +<li>if admin resource, +<ul> +<li>If authenticated user name matches the admin users configuration</li> +<li>Else if groups of the authenticated user matches the admin groups configuration</li> +<li>Else authorization exception is thrown</li></ul></li> +<li>Else if entities or instance resource +<ul> +<li>If the authenticated user matches the owner in ACL for the entity</li> +<li>Else if the groups of the authenticated user matches the group in ACL for the entity</li> +<li>Else authorization exception is thrown</li></ul></li> +<li>Else if lineage resource +<ul> +<li>All have read-only permissions, reason being folks should be able to examine the dependency and allow reuse</li></ul></li></ul> +<p>To authenticate user for REST api calls, user should append "user.name=<username>" to the query.</p> +<p><b>operations on Entity Resource</b></p> +<p></p> +<table border="0" class="table table-striped"> +<tr class="a"> +<th>Resource</th> +<th>Description</th> +<th>Authorization</th></tr> +<tr class="b"> +<td><a href="./Restapi/EntityValidate.html">api/entities/validate/:entity-type</a></td> +<td>Validate the entity</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/EntitySubmit.html">api/entities/submit/:entity-type</a></td> +<td>Submit the entity</td> +<td>Owner/Group</td></tr> +<tr class="b"> +<td><a href="./Restapi/EntityUpdate.html">api/entities/update/:entity-type/:entity-name</a></td> +<td>Update the entity</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/EntitySubmitAndSchedule.html">api/entities/submitAndSchedule/:entity-type</a></td> +<td>Submit & Schedule the entity</td> +<td>Owner/Group</td></tr> +<tr class="b"> +<td><a href="./Restapi/EntitySchedule.html">api/entities/schedule/:entity-type/:entity-name</a></td> +<td>Schedule the entity</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/EntitySuspend.html">api/entities/suspend/:entity-type/:entity-name</a></td> +<td>Suspend the entity</td> +<td>Owner/Group</td></tr> +<tr class="b"> +<td><a href="./Restapi/EntityResume.html">api/entities/resume/:entity-type/:entity-name</a></td> +<td>Resume the entity</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/EntityDelete.html">api/entities/delete/:entity-type/:entity-name</a></td> +<td>Delete the entity</td> +<td>Owner/Group</td></tr> +<tr class="b"> +<td><a href="./Restapi/EntityStatus.html">api/entities/status/:entity-type/:entity-name</a></td> +<td>Get the status of the entity</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/EntityDefinition.html">api/entities/definition/:entity-type/:entity-name</a></td> +<td>Get the definition of the entity</td> +<td>Owner/Group</td></tr> +<tr class="b"> +<td><a href="./Restapi/EntityList.html">api/entities/list/:entity-type?fields=:fields</a></td> +<td>Get the list of entities</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/EntityDependencies.html">api/entities/dependencies/:entity-type/:entity-name</a></td> +<td>Get the dependencies of the entity</td> +<td>Owner/Group</td></tr></table><b>REST Call on Feed and Process Instances</b> +<p></p> +<table border="0" class="table table-striped"> +<tr class="a"> +<th>Resource</th> +<th>Description</th> +<th>Authorization</th></tr> +<tr class="b"> +<td><a href="./Restapi/InstanceRunning.html">api/instance/running/:entity-type/:entity-name</a></td> +<td>List of running instances.</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/InstanceStatus.html">api/instance/status/:entity-type/:entity-name</a></td> +<td>Status of a given instance</td> +<td>Owner/Group</td></tr> +<tr class="b"> +<td><a href="./Restapi/InstanceKill.html">api/instance/kill/:entity-type/:entity-name</a></td> +<td>Kill a given instance</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/InstanceSuspend.html">api/instance/suspend/:entity-type/:entity-name</a></td> +<td>Suspend a running instance</td> +<td>Owner/Group</td></tr> +<tr class="b"> +<td><a href="./Restapi/InstanceResume.html">api/instance/resume/:entity-type/:entity-name</a></td> +<td>Resume a given instance</td> +<td>Owner/Group</td></tr> +<tr class="a"> +<td><a href="./Restapi/InstanceRerun.html">api/instance/rerun/:entity-type/:entity-name</a></td> +<td>Rerun a given instance</td> +<td>Owner/Group</td></tr> +<tr class="b"> +<td><a href="./InstanceLogs.html">api/instance/logs/:entity-type/:entity-name</a></td> +<td>Get logs of a given instance</td> +<td>Owner/Group</td></tr></table></div> +<div class="section"> +<h5>Admin Resources Policy<a name="Admin_Resources_Policy"></a></h5> +<p>Only users belonging to admin users or groups have access to this resource. Admin membership is determined by a static configuration parameter.</p> +<p></p> +<table border="0" class="table table-striped"> +<tr class="a"> +<th>Resource</th> +<th>Description</th> +<th>Authorization</th></tr> +<tr class="b"> +<td><a href="./Restapi/AdminVersion.html">api/admin/version</a></td> +<td>Get version of the server</td> +<td>No restriction</td></tr> +<tr class="a"> +<td><a href="./Restapi/AdminStack.html">api/admin/stack</a></td> +<td>Get stack of the server</td> +<td>Admin User/Group</td></tr> +<tr class="b"> +<td><a href="./Restapi/AdminConfig.html">api/admin/config/:config-type</a></td> +<td>Get configuration information of the server</td> +<td>Admin User/Group</td></tr></table></div> +<div class="section"> +<h5>Lineage Resource Policy<a name="Lineage_Resource_Policy"></a></h5> +<p>Lineage is read-only and hence all users can look at lineage for their respective entities. <b>Note:</b> This gap will be fixed in a later release.</p></div> +<div class="section"> +<h3>Authentication Configuration<a name="Authentication_Configuration"></a></h3> +<p>Following is the Server Side Configuration Setup for Authentication.</p></div> +<div class="section"> +<h4>Common Configuration Parameters<a name="Common_Configuration_Parameters"></a></h4> +<div class="source"> +<pre> +# Authentication type must be specified: simple|kerberos +*.falcon.authentication.type=kerberos + +</pre></div></div> +<div class="section"> +<h4>Kerberos Configuration<a name="Kerberos_Configuration"></a></h4> +<div class="source"> +<pre> +##### Service Configuration + +# Indicates the Kerberos principal to be used in Falcon Service. +*.falcon.service.authentication.kerberos.principal=falcon/[email protected] + +# Location of the keytab file with the credentials for the Service principal. +*.falcon.service.authentication.kerberos.keytab=/etc/security/keytabs/falcon.service.keytab + +# name node principal to talk to config store +*.dfs.namenode.kerberos.principal=nn/[email protected] + +# Indicates how long (in seconds) falcon authentication token is valid before it has to be renewed. +*.falcon.service.authentication.token.validity=86400 + +##### SPNEGO Configuration + +# Authentication type must be specified: simple|kerberos|<class> +# org.apache.falcon.security.RemoteUserInHeaderBasedAuthenticationHandler can be used for backwards compatibility +*.falcon.http.authentication.type=kerberos + +# Indicates how long (in seconds) an authentication token is valid before it has to be renewed. +*.falcon.http.authentication.token.validity=36000 + +# The signature secret for signing the authentication tokens. +*.falcon.http.authentication.signature.secret=falcon + +# The domain to use for the HTTP cookie that stores the authentication token. +*.falcon.http.authentication.cookie.domain= + +# Indicates if anonymous requests are allowed when using 'simple' authentication. +*.falcon.http.authentication.simple.anonymous.allowed=true + +# Indicates the Kerberos principal to be used for HTTP endpoint. +# The principal MUST start with 'HTTP/' as per Kerberos HTTP SPNEGO specification. +*.falcon.http.authentication.kerberos.principal=HTTP/[email protected] + +# Location of the keytab file with the credentials for the HTTP principal. +*.falcon.http.authentication.kerberos.keytab=/etc/security/keytabs/spnego.service.keytab + +# The kerberos names rules is to resolve kerberos principal names, refer to Hadoop's KerberosName for more details. +*.falcon.http.authentication.kerberos.name.rules=DEFAULT + +# Comma separated list of black listed users +*.falcon.http.authentication.blacklisted.users= + +# Increase Jetty request buffer size to accommodate the generated Kerberos token +*.falcon.jetty.request.buffer.size=16192 + +</pre></div></div> +<div class="section"> +<h4>Pseudo/Simple Configuration<a name="PseudoSimple_Configuration"></a></h4> +<div class="source"> +<pre> +##### SPNEGO Configuration + +# Authentication type must be specified: simple|kerberos|<class> +# org.apache.falcon.security.RemoteUserInHeaderBasedAuthenticationHandler can be used for backwards compatibility +*.falcon.http.authentication.type=simple + +# Indicates how long (in seconds) an authentication token is valid before it has to be renewed. +*.falcon.http.authentication.token.validity=36000 + +# The signature secret for signing the authentication tokens. +*.falcon.http.authentication.signature.secret=falcon + +# The domain to use for the HTTP cookie that stores the authentication token. +*.falcon.http.authentication.cookie.domain= + +# Indicates if anonymous requests are allowed when using 'simple' authentication. +*.falcon.http.authentication.simple.anonymous.allowed=true + +# Comma separated list of black listed users +*.falcon.http.authentication.blacklisted.users= + +</pre></div></div> +<div class="section"> +<h3>Authorization Configuration<a name="Authorization_Configuration"></a></h3></div> +<div class="section"> +<h4>Enabling Authorization<a name="Enabling_Authorization"></a></h4> +<p>By default, support for authorization is disabled and specifying ACLs in entities are optional. To enable support for authorization, set falcon.security.authorization.enabled to true in the startup configuration.</p> +<div class="source"> +<pre> +# Authorization Enabled flag: false|true +*.falcon.security.authorization.enabled=true + +</pre></div></div> +<div class="section"> +<h4>Authorization Provider<a name="Authorization_Provider"></a></h4> +<p>Falcon provides a basic implementation for Authorization bundled, org.apache.falcon.security .DefaultFalconAuthorizationProvider. This can be overridden by custom implementations in the startup configuration.</p> +<div class="source"> +<pre> +# Authorization Provider Fully Qualified Class Name +*.falcon.security.authorization.provider=org.apache.falcon.security.DefaultAuthorizationProvider + +</pre></div></div> +<div class="section"> +<h4>Super User Group<a name="Super_User_Group"></a></h4> +<p>Super user group is determined by the configuration:</p> +<div class="source"> +<pre> +# The name of the group of super-users +*.falcon.security.authorization.superusergroup=falcon + +</pre></div></div> +<div class="section"> +<h4>Admin Membership<a name="Admin_Membership"></a></h4> +<p>Administrative users are determined by the configuration:</p> +<div class="source"> +<pre> +# Admin Users, comma separated users +*.falcon.security.authorization.admin.users=falcon,ambari-qa,seetharam + +</pre></div> +<p>Administrative groups are determined by the configuration:</p> +<div class="source"> +<pre> +# Admin Group Membership, comma separated users +*.falcon.security.authorization.admin.groups=falcon,testgroup,staff + +</pre></div></div> +<div class="section"> +<h3>SSL<a name="SSL"></a></h3> +<p>Falcon provides transport level security ensuring data confidentiality and integrity. This is enabled by default for communicating over HTTP between the client and the server.</p></div> +<div class="section"> +<h4>SSL Configuration<a name="SSL_Configuration"></a></h4> +<div class="source"> +<pre> +*.falcon.enableTLS=true +*.keystore.file=/path/to/keystore/file +*.keystore.password=password + +</pre></div></div> +<div class="section"> +<h4>Distributed Falcon Setup<a name="Distributed_Falcon_Setup"></a></h4> +<p>Falcon should be configured to communicate with Prism over TLS in secure mode. Its not enabled by default.</p></div> +<div class="section"> +<h3>Changes to ownership and permissions of directories managed by Falcon<a name="Changes_to_ownership_and_permissions_of_directories_managed_by_Falcon"></a></h3> +<p></p> +<table border="0" class="table table-striped"> +<tr class="a"> +<th>Directory</th> +<th>Location</th> +<th>Owner</th> +<th>Permissions</th></tr> +<tr class="b"> +<td>Configuration Store</td> +<td>${config.store.uri}</td> +<td>falcon</td> +<td>700</td></tr> +<tr class="a"> +<td>Cluster Staging Location</td> +<td>${cluster.staging-location}</td> +<td>falcon</td> +<td>777</td></tr> +<tr class="b"> +<td>Cluster Working Location</td> +<td>${cluster.working-location}</td> +<td>falcon</td> +<td>755</td></tr> +<tr class="a"> +<td>Shared libs</td> +<td>{cluster.working}/{lib,libext}</td> +<td>falcon</td> +<td>755</td></tr> +<tr class="b"> +<td>Oozie coord/bundle XMLs</td> +<td>${cluster.staging-location}/workflows/{entity}/{entity-name}</td> +<td>$user</td> +<td>cluster umask</td></tr> +<tr class="a"> +<td>App logs</td> +<td>${cluster.staging-location}/workflows/{entity}/{entity-name}/logs</td> +<td>$user</td> +<td>cluster umask</td></tr></table><b>Note:</b> Please note that the cluster staging and working locations MUST be created prior to submitting a cluster entity to Falcon. Also, note that the the parent dirs must have execute permissions.</div> +<div class="section"> +<h3>Backwards compatibility<a name="Backwards_compatibility"></a></h3></div> +<div class="section"> +<h4>Scheduled Entities<a name="Scheduled_Entities"></a></h4> +<p>Entities already scheduled with an earlier version of Falcon are not compatible with this version</p></div> +<div class="section"> +<h4>Falcon Clients<a name="Falcon_Clients"></a></h4> +<p>Older Falcon clients are backwards compatible wrt Authentication and user information sent as part of the HTTP header, Remote-User is still honoured when the authentication type is configured as below:</p> +<div class="source"> +<pre> +*.falcon.http.authentication.type=org.apache.falcon.security.RemoteUserInHeaderBasedAuthenticationHandler + +</pre></div></div> +<div class="section"> +<h4>Blacklisted super users for authentication<a name="Blacklisted_super_users_for_authentication"></a></h4> +<p>The blacklist users used to have the following super users: hdfs, mapreduce, oozie, and falcon. The list is externalized from code into Startup.properties file and is empty now and needs to be configured specifically in the file.</p></div> +<div class="section"> +<h4>Falcon Dashboard<a name="Falcon_Dashboard"></a></h4> +<p>To initialize the current user for dashboard, user should append query param "user.name=<username>" to the REST api call.</p> +<p>If dashboard user wishes to change the current user, they should do the following.</p> +<ul> +<li>delete the hadoop.auth cookie from browser cache.</li> +<li>append query param "user.name=<new_user>" to the next REST API call.</li></ul> +<p>In Kerberos method, the browser must support HTTP Kerberos SPNEGO.</p></div> +<div class="section"> +<h3>Known Limitations<a name="Known_Limitations"></a></h3> +<p></p> +<ul> +<li>ActiveMQ topics are not secure but will be in the near future</li> +<li>Entities already scheduled with an earlier version of Falcon are not compatible with this version as new</li></ul>workflow parameters are being passed back into Falcon such as the user are required +<ul> +<li>Use of hftp as the scheme for read only interface in cluster entity <a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-10215";>will not work in Oozie</a></li></ul>The alternative is to use webhdfs scheme instead and its been tested with <a href="./DistCp.html">DistCp</a>.</div> +<div class="section"> +<h3>Examples<a name="Examples"></a></h3></div> +<div class="section"> +<h4>Accessing the server using Falcon CLI (Java client)<a name="Accessing_the_server_using_Falcon_CLI_Java_client"></a></h4> +<p>There is no change in the way the CLI is used. The CLI has been changed to work with the configured authentication method.</p></div> +<div class="section"> +<h4>Accessing the server using curl<a name="Accessing_the_server_using_curl"></a></h4> +<p>Try accessing protected resources using curl. The protected resources are:</p> +<div class="source"> +<pre> +$ kinit +Please enter the password for venkatesh@LOCALHOST: + +$ curl http://localhost:15000/api/admin/version + +$ curl http://localhost:15000/api/admin/version?user.name=venkatesh + +$ curl --negotiate -u foo -b ~/cookiejar.txt -c ~/cookiejar.txt curl http://localhost:15000/api/admin/version + +</pre></div></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/css/apache-maven-fluido-1.3.0.min.css URL: http://svn.apache.org/viewvc/falcon/site/0.9/css/apache-maven-fluido-1.3.0.min.css?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/css/apache-maven-fluido-1.3.0.min.css (added) +++ falcon/site/0.9/css/apache-maven-fluido-1.3.0.min.css Mon Feb 15 05:36:56 2016 @@ -0,0 +1,9 @@ +/*! + * Bootstrap v2.1.0 + * + * Copyright 2012 Twitter, Inc + * Licensed under the Apache License v2.0 + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Designed and built with all the love in the world @twitter by @mdo and @fat. [... 3 lines stripped ...] Added: falcon/site/0.9/css/print.css URL: http://svn.apache.org/viewvc/falcon/site/0.9/css/print.css?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/css/print.css (added) +++ falcon/site/0.9/css/print.css Mon Feb 15 05:36:56 2016 @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/* $Id: print.css 1201871 2011-11-14 20:18:24Z simonetripodi $ */ + +#banner, #footer, #leftcol, #breadcrumbs, .docs #toc, .docs .courtesylinks, #leftColumn, #navColumn {display: none !important;} +#bodyColumn, body.docs div.docs {margin: 0 !important;border: none !important} Added: falcon/site/0.9/css/site.css URL: http://svn.apache.org/viewvc/falcon/site/0.9/css/site.css?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/css/site.css (added) +++ falcon/site/0.9/css/site.css Mon Feb 15 05:36:56 2016 @@ -0,0 +1 @@ +/* You can override this file with your own styles */ \ No newline at end of file Added: falcon/site/0.9/dependency-convergence.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/dependency-convergence.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/dependency-convergence.html (added) +++ falcon/site/0.9/dependency-convergence.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,121 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Reactor Dependency Convergence</title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Reactor Dependency Convergence</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h2>Reactor Dependency Convergence<a name="Reactor_Dependency_Convergence"></a></h2> +<table border="0" class="table table-striped"><caption> + <b>Legend:</b> +</caption> +<tr class="a"> +<td><img alt="error" src="images/icon_error_sml.gif" /></td> +<td>At least one dependency has a differing version of the dependency or has SNAPSHOT dependencies.</td></tr></table><br /> +<table border="0" class="table table-striped"><caption> + <b>Statistics:</b> +</caption> +<tr class="b"> +<th>Number of modules:</th> +<td>9</td></tr> +<tr class="a"> +<th>Number of dependencies (NOD):</th> +<td>0</td></tr> +<tr class="b"> +<th>Number of unique artifacts (NOA):</th> +<td>0</td></tr> +<tr class="a"> +<th>Number of version-conflicting artifacts (NOC):</th> +<td>0</td></tr> +<tr class="b"> +<th>Number of SNAPSHOT artifacts (NOS):</th> +<td>0</td></tr> +<tr class="a"> +<th>Convergence (NOD/NOA):</th> +<td><img alt="error" src="images/icon_error_sml.gif" /> <b>0 %</b></td></tr> +<tr class="b"> +<th>Ready for release (100 % convergence and no SNAPSHOTS):</th> +<td><img alt="error" src="images/icon_error_sml.gif" /> <b>Error</b><br />You do not have 100 % convergence.</td></tr></table></div> +<div class="section"> +<h3>Dependencies used in modules<a name="Dependencies_used_in_modules"></a></h3></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/dependency-info.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/dependency-info.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/dependency-info.html (added) +++ falcon/site/0.9/dependency-info.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,126 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Dependency Information</title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Dependency Information</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h2>Dependency Information<a name="Dependency_Information"></a></h2><a name="Dependency_Information"></a> +<div class="section"> +<h3>Apache Maven<a name="Apache_Maven"></a></h3><a name="Apache_Maven"></a> +<div class="source"> +<pre><dependency> + <groupId>org.apache.falcon</groupId> + <artifactId>falcon-website-0.9</artifactId> + <version>0.9</version> + <type>pom</type> +</dependency></pre></div></div> +<div class="section"> +<h3>Apache Buildr<a name="Apache_Buildr"></a></h3><a name="Apache_Buildr"></a> +<div class="source"> +<pre>'org.apache.falcon:falcon-website-0.9:pom:0.9'</pre></div></div> +<div class="section"> +<h3>Apache Ivy<a name="Apache_Ivy"></a></h3><a name="Apache_Ivy"></a> +<div class="source"> +<pre><dependency org="org.apache.falcon" name="falcon-website-0.9" rev="0.9"> + <artifact name="falcon-website-0.9" type="pom" /> +</dependency></pre></div></div> +<div class="section"> +<h3>Groovy Grape<a name="Groovy_Grape"></a></h3><a name="Groovy_Grape"></a> +<div class="source"> +<pre>@Grapes( +@Grab(group='org.apache.falcon', module='falcon-website-0.9', version='0.9') +)</pre></div></div> +<div class="section"> +<h3>Grails<a name="Grails"></a></h3><a name="Grails"></a> +<div class="source"> +<pre>compile 'org.apache.falcon:falcon-website-0.9:0.9'</pre></div></div> +<div class="section"> +<h3>Leiningen<a name="Leiningen"></a></h3><a name="Leiningen"></a> +<div class="source"> +<pre>[org.apache.falcon/falcon-website-0.9 "0.9"]</pre></div></div> +<div class="section"> +<h3>SBT<a name="SBT"></a></h3><a name="SBT"></a> +<div class="source"> +<pre>libraryDependencies += "org.apache.falcon" % "falcon-website-0.9" % "0.9"</pre></div></div></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/distribution-management.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/distribution-management.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/distribution-management.html (added) +++ falcon/site/0.9/distribution-management.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,93 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Project Distribution Management</title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Project Distribution Management</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h2>Overview<a name="Overview"></a></h2><a name="Overview"></a> +<p>The following is the distribution management information used by this project.</p> +<div class="section"> +<h3>Site - apache-falcon-website<a name="Site_-_apache-falcon-website"></a></h3><a name="Site_-_apache-falcon-website"></a> +<p>scpexe://people.apache.org/www/falcon.apache.org/falcon-website-releases/falcon-website-0.9</p></div></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/falconcli/CommonCLI.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/falconcli/CommonCLI.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/falconcli/CommonCLI.html (added) +++ falcon/site/0.9/falconcli/CommonCLI.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,100 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Common CLI Options</title> + <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="../css/site.css" /> + <link rel="stylesheet" href="../css/print.css" media="print" /> + + + <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="../images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="../index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Common CLI Options</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h3>Common CLI Options<a name="Common_CLI_Options"></a></h3></div> +<div class="section"> +<h4>Falcon URL<a name="Falcon_URL"></a></h4> +<p>Optional -url option indicating the URL of the Falcon system to run the command against can be provided. If not mentioned it will be picked from the system environment variable FALCON_URL. If FALCON_URL is not set then it will be picked from client.properties file. If the option is not provided and also not set in client.properties, Falcon CLI will fail.</p></div> +<div class="section"> +<h4>Proxy user support<a name="Proxy_user_support"></a></h4> +<p>The -doAs option allows the current user to impersonate other users when interacting with the Falcon system. The current user must be configured as a proxyuser in the Falcon system. The proxyuser configuration may restrict from which hosts a user may impersonate users, as well as users of which groups can be impersonated.</p> +<p><a href="../FalconDocumentation.html#Proxyuser_support">Proxyuser support described here.</a></p></div> +<div class="section"> +<h4>Debug Mode<a name="Debug_Mode"></a></h4> +<p>If you export FALCON_DEBUG=true then the Falcon CLI will output the Web Services API details used by any commands you execute. This is useful for debugging purposes to or see how the Falcon CLI works with the WS API. Alternately, you can specify '-debug' through the CLI arguments to get the debug statements.</p> +<p>Example: $FALCON_HOME/bin/falcon entity -submit -type cluster -file /cluster/definition.xml -debug</p></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/falconcli/ContinueInstance.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/falconcli/ContinueInstance.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/falconcli/ContinueInstance.html (added) +++ falcon/site/0.9/falconcli/ContinueInstance.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,92 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Continue</title> + <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="../css/site.css" /> + <link rel="stylesheet" href="../css/print.css" media="print" /> + + + <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="../images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="../index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Continue</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h4>Continue<a name="Continue"></a></h4> +<p><a href="./CommonCLI.html">Common CLI Options</a></p> +<p>Continue option is used to continue the failed workflow instance. This option is valid only for process instances in terminal state, i.e. KILLED or FAILED.</p> +<p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -continue -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'"</p></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/falconcli/Definition.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/falconcli/Definition.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/falconcli/Definition.html (added) +++ falcon/site/0.9/falconcli/Definition.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,92 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Definition</title> + <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="../css/site.css" /> + <link rel="stylesheet" href="../css/print.css" media="print" /> + + + <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="../images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="../index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Definition</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h4>Definition<a name="Definition"></a></h4> +<p><a href="./CommonCLI.html">Common CLI Options</a></p> +<p>Definition option returns the entity definition submitted earlier during submit step.</p> +<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -definition</p></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/falconcli/DeleteEntity.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/falconcli/DeleteEntity.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/falconcli/DeleteEntity.html (added) +++ falcon/site/0.9/falconcli/DeleteEntity.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,92 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Delete</title> + <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="../css/site.css" /> + <link rel="stylesheet" href="../css/print.css" media="print" /> + + + <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="../images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="../index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Delete</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h4>Delete<a name="Delete"></a></h4> +<p><a href="./CommonCLI.html">Common CLI Options</a></p> +<p>Delete removes the submitted entity definition for the specified entity and put it into the archive.</p> +<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -delete</p></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/falconcli/DependencyEntity.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/falconcli/DependencyEntity.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/falconcli/DependencyEntity.html (added) +++ falcon/site/0.9/falconcli/DependencyEntity.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,92 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Dependency</title> + <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="../css/site.css" /> + <link rel="stylesheet" href="../css/print.css" media="print" /> + + + <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="../images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="../index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Dependency</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h4>Dependency<a name="Dependency"></a></h4> +<p><a href="./CommonCLI.html">Common CLI Options</a></p> +<p>With the use of dependency option, we can list all the entities on which the specified entity is dependent. For example for a feed, dependency return the cluster name and for process it returns all the input feeds, output feeds and cluster names.</p> +<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -dependency</p></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/falconcli/DependencyInstance.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/falconcli/DependencyInstance.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/falconcli/DependencyInstance.html (added) +++ falcon/site/0.9/falconcli/DependencyInstance.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,98 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Dependency</title> + <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="../css/site.css" /> + <link rel="stylesheet" href="../css/print.css" media="print" /> + + + <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="../images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="../index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Dependency</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h4>Dependency<a name="Dependency"></a></h4> +<p>Display the dependent instances which are dependent on the given instance. For example for a given process instance it will list all the input feed instances(if any) and the output feed instances(if any).</p> +<p>An example use case of this command is as follows: Suppose you find out that the data in a feed instance was incorrect and you need to figure out which all process instances consumed this feed instance so that you can reprocess them after correcting the feed instance. You can give the feed instance and it will tell you which process instance produced this feed and which all process instances consumed this feed.</p> +<p>NOTE: 1. instanceTime must be a valid instanceTime e.g. instanceTime of a feed should be in it's validity range on applicable clusters, and it should be in the range of instances produced by the producer process(if any)</p> +<p>2. For processes with inputs like latest() which vary with time the results are not guaranteed to be correct.</p> +<p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -dependency -instanceTime "yyyy-MM-dd'T'HH:mm'Z'"</p> +<p>For example: $FALCON_HOME/bin/falcon instance -dependency -type feed -name out -instanceTime 2014-12-15T00:00Z name: producer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:00Z, tags: Output name: consumer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:03Z, tags: Input name: consumer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:04Z, tags: Input name: consumer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:02Z, tags: Input name: consumer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:05Z, tags: Input</p> +<p>Response: default/Success!</p> +<p>Request Id: default/1125035965@qtp-503156953-7 - 447be0ad-1d38-4dce-b438-20f3de69b172</p> +<p><a href="../Restapi/InstanceDependencies.html">Optional params described here.</a></p></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html> Added: falcon/site/0.9/falconcli/EdgeMetadata.html URL: http://svn.apache.org/viewvc/falcon/site/0.9/falconcli/EdgeMetadata.html?rev=1730448&view=auto ============================================================================== --- falcon/site/0.9/falconcli/EdgeMetadata.html (added) +++ falcon/site/0.9/falconcli/EdgeMetadata.html Mon Feb 15 05:36:56 2016 @@ -0,0 +1,93 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Edge</title> + <link rel="stylesheet" href="../css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="../css/site.css" /> + <link rel="stylesheet" href="../css/print.css" media="print" /> + + + <script type="text/javascript" src="../js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + + <div class="container"> + <div id="banner"> + <div class="pull-left"> + <div id="bannerLeft"> + <img src="../images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> + </div> + </div> + <div class="pull-right"> </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="../index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Edge</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> <li class="divider pull-right">|</li> + <li id="projectVersion" class="pull-right">Version: 0.9</li> + + </ul> + </div> + + + + <div id="bodyColumn" > + + <div class="section"> +<h4>Edge<a name="Edge"></a></h4> +<p><a href="./CommonCLI.html">Common CLI Options</a></p> +<p>Get the edge with the specified id.</p> +<p>Usage: $FALCON_HOME/bin/falcon metadata -edge -id <<id>></p> +<p>Example: $FALCON_HOME/bin/falcon metadata -edge -id Q9n-Q-5g</p></div> + </div> + </div> + + <hr/> + + <footer> + <div class="container"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + <p id="poweredBy" class="pull-right"> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="../images/logos/maven-feather.png" /> + </a> + </p> + + </div> + </footer> + </body> +</html>
