Modified: falcon/site/FalconCLI.html URL: http://svn.apache.org/viewvc/falcon/site/FalconCLI.html?rev=1730450&r1=1730449&r2=1730450&view=diff ============================================================================== --- falcon/site/FalconCLI.html (original) +++ falcon/site/FalconCLI.html Mon Feb 15 06:02:08 2016 @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia at 2015-12-08 + | Generated by Apache Maven Doxia at 2016-02-09 | Rendered using Apache Maven Fluido Skin 1.3.0 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20151208" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> <meta http-equiv="Content-Language" content="en" /> <title>Falcon - FalconCLI</title> <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> @@ -58,7 +58,7 @@ - <li id="publishDate" class="pull-right">Last Published: 2015-12-08</li> + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> </ul> </div> @@ -306,6 +306,13 @@ <li> + <a href="0.9/index.html" title="0.9 (Current)"> + <i class="none"></i> + 0.9 (Current)</a> + </li> + + <li> + <a href="0.8/index.html" title="0.8"> <i class="none"></i> 0.8</a> @@ -405,16 +412,30 @@ <h2>FalconCLI<a name="FalconCLI"></a></h2> <p>FalconCLI is a interface between user and Falcon. It is a command line utility provided by Falcon. FalconCLI supports Entity Management, Instance Management and Admin operations.There is a set of web services that are used by FalconCLI to interact with Falcon.</p></div> <div class="section"> +<h3>Common CLI Options<a name="Common_CLI_Options"></a></h3></div> +<div class="section"> +<h4>Falcon URL<a name="Falcon_URL"></a></h4> +<p>Optional -url option indicating the URL of the Falcon system to run the command against can be provided. If not mentioned it will be picked from the system environment variable FALCON_URL. If FALCON_URL is not set then it will be picked from client.properties file. If the option is not provided and also not set in client.properties, Falcon CLI will fail.</p></div> +<div class="section"> +<h4>Proxy user support<a name="Proxy_user_support"></a></h4> +<p>The -doAs option allows the current user to impersonate other users when interacting with the Falcon system. The current user must be configured as a proxyuser in the Falcon system. The proxyuser configuration may restrict from which hosts a user may impersonate users, as well as users of which groups can be impersonated.</p> +<p><a href="./FalconDocumentation.html#Proxyuser_support">Proxyuser support described here.</a></p></div> +<div class="section"> +<h4>Debug Mode<a name="Debug_Mode"></a></h4> +<p>If you export FALCON_DEBUG=true then the Falcon CLI will output the Web Services API details used by any commands you execute. This is useful for debugging purposes to or see how the Falcon CLI works with the WS API. Alternately, you can specify '-debug' through the CLI arguments to get the debug statements. Example: $FALCON_HOME/bin/falcon entity -submit -type cluster -file /cluster/definition.xml -debug</p></div> +<div class="section"> <h3>Entity Management Operations<a name="Entity_Management_Operations"></a></h3></div> <div class="section"> <h4>Submit<a name="Submit"></a></h4> <p>Submit option is used to set up entity definition.</p> +<p>Usage: $FALCON_HOME/bin/falcon entity -submit -type [cluster|datasource|feed|process] -file <entity-definition.xml></p> <p>Example: $FALCON_HOME/bin/falcon entity -submit -type cluster -file /cluster/definition.xml</p> <p>Note: The url option in the above and all subsequent commands is optional. If not mentioned it will be picked from client.properties file. If the option is not provided and also not set in client.properties, Falcon CLI will fail.</p></div> <div class="section"> <h4>Schedule<a name="Schedule"></a></h4> <p>Once submitted, an entity can be scheduled using schedule option. Process and feed can only be scheduled.</p> <p>Usage: $FALCON_HOME/bin/falcon entity -type [process|feed] -name <<name>> -schedule</p> +<p>Optional Arg : -skipDryRun. When this argument is specified, Falcon skips oozie dryrun.</p> <p>Example: $FALCON_HOME/bin/falcon entity -type process -name sampleProcess -schedule</p></div> <div class="section"> <h4>Suspend<a name="Suspend"></a></h4> @@ -427,46 +448,70 @@ <div class="section"> <h4>Delete<a name="Delete"></a></h4> <p>Delete removes the submitted entity definition for the specified entity and put it into the archive.</p> -<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|feed|process] -name <<name>> -delete</p></div> +<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -delete</p></div> <div class="section"> <h4>List<a name="List"></a></h4> <p>Entities of a particular type can be listed with list sub-command.</p> -<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|feed|process] -list</p> -<p>Optional Args : -fields <<field1,field2>> -filterBy <<field1:value1,field2:value2>> -tags <<tagkey=tagvalue,tagkey=tagvalue>> -nameseq <<namesubsequence>> -orderBy <<field>> -sortOrder <<sortOrder>> -offset 0 -numResults 10</p> -<p><a href="./restapi/EntityList.html">Optional params described here.</a></p></div> +<p>Usage: $FALCON_HOME/bin/falcon entity -list</p> +<p>Optional Args : -fields <<field1,field2>> -type <<[cluster|datasource|feed|process],[cluster|datasource|feed|process]>> -nameseq <<namesubsequence>> -tagkeys <<tagkeyword1,tagkeyword2>> -filterBy <<field1:value1,field2:value2>> -tags <<tagkey=tagvalue,tagkey=tagvalue>> -orderBy <<field>> -sortOrder <<sortOrder>> -offset 0 -numResults 10</p> +<p><a href="./Restapi/EntityList.html">Optional params described here.</a></p></div> <div class="section"> <h4>Summary<a name="Summary"></a></h4> <p>Summary of entities of a particular type and a cluster will be listed. Entity summary has N most recent instances of entity.</p> <p>Usage: $FALCON_HOME/bin/falcon entity -type [feed|process] -summary</p> <p>Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -fields <<field1,field2>> -filterBy <<field1:value1,field2:value2>> -tags <<tagkey=tagvalue,tagkey=tagvalue>> -orderBy <<field>> -sortOrder <<sortOrder>> -offset 0 -numResults 10 -numInstances 7</p> -<p><a href="./restapi/EntitySummary.html">Optional params described here.</a></p></div> +<p><a href="./Restapi/EntitySummary.html">Optional params described here.</a></p></div> <div class="section"> <h4>Update<a name="Update"></a></h4> -<p>Update operation allows an already submitted/scheduled entity to be updated. Cluster update is currently not allowed.</p> +<p>Update operation allows an already submitted/scheduled entity to be updated. Cluster and datasource updates are currently not allowed.</p> <p>Usage: $FALCON_HOME/bin/falcon entity -type [feed|process] -name <<name>> -update -file <<path_to_file>></p> +<p>Optional Arg : -skipDryRun. When this argument is specified, Falcon skips oozie dryrun.</p> <p>Example: $FALCON_HOME/bin/falcon entity -type process -name <a href="./HourlyReportsGenerator.html">HourlyReportsGenerator</a> -update -file /process/definition.xml</p></div> <div class="section"> <h4>Touch<a name="Touch"></a></h4> <p>Force Update operation allows an already submitted/scheduled entity to be updated.</p> -<p>Usage: $FALCON_HOME/bin/falcon entity -type [feed|process] -name <<name>> -touch</p></div> +<p>Usage: $FALCON_HOME/bin/falcon entity -type [feed|process] -name <<name>> -touch</p> +<p>Optional Arg : -skipDryRun. When this argument is specified, Falcon skips oozie dryrun.</p></div> <div class="section"> <h4>Status<a name="Status"></a></h4> <p>Status returns the current status of the entity.</p> -<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|feed|process] -name <<name>> -status</p></div> +<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -status</p></div> <div class="section"> <h4>Dependency<a name="Dependency"></a></h4> <p>With the use of dependency option, we can list all the entities on which the specified entity is dependent. For example for a feed, dependency return the cluster name and for process it returns all the input feeds, output feeds and cluster names.</p> -<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|feed|process] -name <<name>> -dependency</p></div> +<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -dependency</p></div> <div class="section"> <h4>Definition<a name="Definition"></a></h4> <p>Definition option returns the entity definition submitted earlier during submit step.</p> -<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|feed|process] -name <<name>> -definition</p></div> +<p>Usage: $FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -definition</p></div> <div class="section"> <h4>Lookup<a name="Lookup"></a></h4> <p>Lookup option tells you which feed does a given path belong to. This can be useful in several scenarios e.g. generally you would want to have a single definition for common feeds like metadata with same location otherwise it can result in a problem (different retention durations can result in surprises for one team) If you want to check if there are multiple definitions of same metadata then you can pick an instance of that and run through the lookup command like below.</p> <p>Usage: $FALCON_HOME/bin/falcon entity -type feed -lookup -path /data/projects/my-hourly/2014/10/10/23/</p> <p>If you have multiple feeds with location as /data/projects/my-hourly/${YEAR}/${MONTH}/${DAY}/${HOUR} then this command will return all of them.</p></div> <div class="section"> +<h4>SLAAlert<a name="SLAAlert"></a></h4> +<div class="source"> +<pre> +Since: 0.8 + +</pre></div> +<p>This command lists all the feed instances which have missed sla and are still not available. If a feed instance missed sla but is now available, then it will not be reported in results. The purpose of this API is alerting and hence it doesn't return feed instances which missed SLA but are available as they don't require any action.</p> +<p>* Currently sla monitoring is supported only for feeds.</p> +<p>* Option end is optional and will default to current time if missing.</p> +<p>* Option name is optional, if provided only instances of that feed will be considered.</p> +<p>Usage:</p> +<p><b>Example 1</b></p> +<p><b>$FALCON_HOME/bin/falcon entity -type feed -start 2014-09-05T00:00Z -slaAlert -end 2016-05-03T00:00Z -colo local</b></p> +<p>name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T11:59Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:00Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:01Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:02Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:03Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:04Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:05Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:06Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:07Z, tags: Missed SLA High name: out, type: FEED, cluster: local, instanceTime: 2015-09-26T12:08Z, tags: Missed SLA Low</p> +<p>Response: default/Success!</p> +<p>Request Id: default/216978070@qtp-830047511-4 - f5a6c129-ab42-4feb-a2bf-c3baed356248</p> +<p><b>Example 2</b></p> +<p><b>$FALCON_HOME/bin/falcon entity -type feed -start 2014-09-05T00:00Z -slaAlert -end 2016-05-03T00:00Z -colo local -name in</b></p> +<p>name: in, type: FEED, cluster: local, instanceTime: 2015-09-26T06:00Z, tags: Missed SLA High</p> +<p>Response: default/Success!</p> +<p>Request Id: default/1580107885@qtp-830047511-7 - f16cbc51-5070-4551-ad25-28f75e5e4cf2</p></div> +<div class="section"> <h3>Instance Management Options<a name="Instance_Management_Options"></a></h3></div> <div class="section"> <h4>Kill<a name="Kill"></a></h4> @@ -484,7 +529,7 @@ <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -continue -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'"</p></div> <div class="section"> <h4>Rerun<a name="Rerun"></a></h4> -<p>Rerun option is used to rerun instances of a given process. On issuing a rerun, by default the execution resumes from the last failed node in the workflow. This option is valid only for process instances in terminal state, i.e. SUCCEEDED, KILLED or FAILED. If one wants to forcefully rerun the entire workflow, -force should be passed along with -rerun Additionally, you can also specify properties to override via a properties file.</p> +<p>Rerun option is used to rerun instances of a given process. On issuing a rerun, by default the execution resumes from the last failed node in the workflow. This option is valid only for process instances in terminal state, i.e. SUCCEEDED, KILLED or FAILED. If one wants to forcefully rerun the entire workflow, -force should be passed along with -rerun Additionally, you can also specify properties to override via a properties file and this will be prioritized over force option in case of contradiction.</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -rerun -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" [-force] [-file <<properties file>>]</p></div> <div class="section"> <h4>Resume<a name="Resume"></a></h4> @@ -497,50 +542,65 @@ <p>{"status":"SUCCEEDED","message":"getStatus is successful","instances":[{"instance":"2012-05-07T05:02Z","status":"SUCCEEDED","logFile":"http://oozie-dashboard-url"},{"instance":"2012-05-07T05:07Z","status":"RUNNING","logFile":"http://oozie-dashboard-url"}, {"instance":"2010-01-02T11:05Z","status":"WAITING"}]</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -status</p> <p>Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -colo <<colo>> -filterBy <<field1:value1,field2:value2>> -lifecycle <<lifecycles>> -orderBy field -sortOrder <<sortOrder>> -offset 0 -numResults 10</p> -<p><a href="./restapi/InstanceStatus.html"> Optional params described here.</a></p></div> +<p><a href="./Restapi/InstanceStatus.html"> Optional params described here.</a></p></div> <div class="section"> <h4>List<a name="List"></a></h4> <p>List option via CLI can be used to get single or multiple instances. If the instance is not yet materialized but is within the process validity range, WAITING is returned as the state. Instance time is also returned. Log location gives the oozie workflow url If the instance is in WAITING state, missing dependencies are listed</p> <p>Example : Suppose a process has 3 instance, one has succeeded,one is in running state and other one is waiting, the expected output is:</p> -<p>{"status":"SUCCEEDED","message":"getStatus is successful","instances":[{"instance":"2012-05-07T05:02Z","status":"SUCCEEDED","logFile":"http://oozie-dashboard-url"},{"instance":"2012-05-07T05:07Z","status":"RUNNING","logFile":"http://oozie-dashboard-url"}, {"instance":"2010-01-02T11:05Z","status":"WAITING"}]</p> +<p>{"status":"SUCCEEDED","message":"getStatus is successful","instances":[{"instance":"2012-05-07T05:02Z","status":"SUCCEEDED","logFile":"http://oozie-dashboard-url"},{"instance":"2012-05-07T05:07Z","status":"RUNNING","logFile":"http://oozie-dashboard-url"}, {"instance":"2010-01-02T11:05Z","status":"WAITING"}]}</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -list</p> <p>Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -colo <<colo>> -lifecycle <<lifecycles>> -filterBy <<field1:value1,field2:value2>> -orderBy field -sortOrder <<sortOrder>> -offset 0 -numResults 10</p> -<p><a href="./restapi/InstanceList.html">Optional params described here.</a></p></div> +<p><a href="./Restapi/InstanceList.html">Optional params described here.</a></p></div> <div class="section"> <h4>Summary<a name="Summary"></a></h4> <p>Summary option via CLI can be used to get the consolidated status of the instances between the specified time period. Each status along with the corresponding instance count are listed for each of the applicable colos. The unscheduled instances between the specified time period are included as UNSCHEDULED in the output to provide more clarity.</p> <p>Example : Suppose a process has 3 instance, one has succeeded,one is in running state and other one is waiting, the expected output is:</p> -<p>{"status":"SUCCEEDED","message":"getSummary is successful", "cluster": <<name>> [{"SUCCEEDED":"1"}, {"WAITING":"1"}, {"RUNNING":"1"}]}</p> +<p>{"status":"SUCCEEDED","message":"getSummary is successful", instancesSummary:[{"cluster": <<name>> "map":[{"SUCCEEDED":"1"}, {"WAITING":"1"}, {"RUNNING":"1"}]}]}</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -summary</p> -<p>Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -colo <<colo>> -lifecycle <<lifecycles>></p> -<p><a href="./restapi/InstanceSummary.html">Optional params described here.</a></p></div> +<p>Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -colo <<colo>> -filterBy <<field1:value1,field2:value2>> -lifecycle <<lifecycles>> -orderBy field -sortOrder <<sortOrder>></p> +<p><a href="./Restapi/InstanceSummary.html">Optional params described here.</a></p></div> <div class="section"> <h4>Running<a name="Running"></a></h4> <p>Running option provides all the running instances of the mentioned process.</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -running</p> <p>Optional Args : -colo <<colo>> -lifecycle <<lifecycles>> -filterBy <<field1:value1,field2:value2>> -orderBy <<field>> -sortOrder <<sortOrder>> -offset 0 -numResults 10</p> -<p><a href="./restapi/InstanceRunning.html">Optional params described here.</a></p></div> +<p><a href="./Restapi/InstanceRunning.html">Optional params described here.</a></p></div> <div class="section"> <h4>FeedInstanceListing<a name="FeedInstanceListing"></a></h4> <p>Get falcon feed instance availability.</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type feed -name <<name>> -listing</p> <p>Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -colo <<colo>></p> -<p><a href="./restapi/FeedInstanceListing.html">Optional params described here.</a></p></div> +<p><a href="./Restapi/FeedInstanceListing.html">Optional params described here.</a></p></div> <div class="section"> <h4>Logs<a name="Logs"></a></h4> <p>Get logs for instance actions</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -logs</p> <p>Optional Args : -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'" -runid <<runid>> -colo <<colo>> -lifecycle <<lifecycles>> -filterBy <<field1:value1,field2:value2>> -orderBy field -sortOrder <<sortOrder>> -offset 0 -numResults 10</p> -<p><a href="./restapi/InstanceLogs.html">Optional params described here.</a></p></div> +<p><a href="./Restapi/InstanceLogs.html">Optional params described here.</a></p></div> <div class="section"> <h4>LifeCycle<a name="LifeCycle"></a></h4> <p>Describes list of life cycles of a entity , for feed it can be replication/retention and for process it can be execution. This can be used with instance management options. Default values are replication for feed and execution for process.</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -status -lifecycle <<lifecycletype>> -start "yyyy-MM-dd'T'HH:mm'Z'" -end "yyyy-MM-dd'T'HH:mm'Z'"</p></div> <div class="section"> +<h4>Triage<a name="Triage"></a></h4> +<p>Given a feed/process instance this command traces it's ancestors to find what all ancestors have failed. It's useful if lot of instances are failing in a pipeline as it then finds out the root cause of the pipeline being stuck.</p> +<p>Usage: $FALCON_HOME/bin/falcon instance -triage -type <<feed/process>> -name <<name>> -start "yyyy-MM-dd'T'HH:mm'Z'"</p></div> +<div class="section"> <h4>Params<a name="Params"></a></h4> <p>Displays the workflow params of a given instance. Where start time is considered as nominal time of that instance and end time won't be considered.</p> <p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -params -start "yyyy-MM-dd'T'HH:mm'Z'"</p></div> <div class="section"> +<h4>Dependency<a name="Dependency"></a></h4> +<p>Display the dependent instances which are dependent on the given instance. For example for a given process instance it will list all the input feed instances(if any) and the output feed instances(if any).</p> +<p>An example use case of this command is as follows: Suppose you find out that the data in a feed instance was incorrect and you need to figure out which all process instances consumed this feed instance so that you can reprocess them after correcting the feed instance. You can give the feed instance and it will tell you which process instance produced this feed and which all process instances consumed this feed.</p> +<p>NOTE: 1. instanceTime must be a valid instanceTime e.g. instanceTime of a feed should be in it's validity range on applicable clusters, and it should be in the range of instances produced by the producer process(if any)</p> +<p>2. For processes with inputs like latest() which vary with time the results are not guaranteed to be correct.</p> +<p>Usage: $FALCON_HOME/bin/falcon instance -type <<feed/process>> -name <<name>> -params -instanceTime "yyyy-MM-dd'T'HH:mm'Z'"</p> +<p>For example: $FALCON_HOME/bin/falcon instance -dependency -type feed -name out -instanceTime 2014-12-15T00:00Z name: producer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:00Z, tags: Output name: consumer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:03Z, tags: Input name: consumer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:04Z, tags: Input name: consumer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:02Z, tags: Input name: consumer, type: PROCESS, cluster: local, instanceTime: 2014-12-15T00:05Z, tags: Input</p> +<p>Response: default/Success!</p> +<p>Request Id: default/1125035965@qtp-503156953-7 - 447be0ad-1d38-4dce-b438-20f3de69b172</p> +<p><a href="./Restapi/InstanceDependency.html">Optional params described here.</a></p></div> +<div class="section"> <h3>Metadata Lineage Options<a name="Metadata_Lineage_Options"></a></h3></div> <div class="section"> <h4>Lineage<a name="Lineage"></a></h4> @@ -572,9 +632,11 @@ <h3>Metadata Discovery Options<a name="Metadata_Discovery_Options"></a></h3></div> <div class="section"> <h4>List<a name="List"></a></h4> -<p>Lists of all dimensions of given type. If the user provides optional param cluster, only the dimensions related to the cluster are listed. Usage: $FALCON_HOME/bin/falcon metadata -list -type [cluster_entity|feed_entity|process_entity|user|colo|tags|groups|pipelines]</p> +<p>Lists of all dimensions of given type. If the user provides optional param cluster, only the dimensions related to the cluster are listed. Usage: $FALCON_HOME/bin/falcon metadata -list -type [cluster_entity|datasource_entity|feed_entity|process_entity|user|colo|tags|groups|pipelines|replication_metrics]</p> <p>Optional Args : -cluster <<cluster name>></p> -<p>Example: $FALCON_HOME/bin/falcon metadata -list -type process_entity -cluster primary-cluster $FALCON_HOME/bin/falcon metadata -list -type tags</p></div> +<p>Example: $FALCON_HOME/bin/falcon metadata -list -type process_entity -cluster primary-cluster $FALCON_HOME/bin/falcon metadata -list -type tags</p> +<p>To display replication metrics from recipe based replication process and from feed replication. Usage: $FALCON_HOME/bin/falcon metadata -list -type replication_metrics -process/-feed <entity name> Optional Args : -numResults <<value>></p> +<p>Example: $FALCON_HOME/bin/falcon metadata -list -type replication_metrics -process hdfs-replication $FALCON_HOME/bin/falcon metadata -list -type replication_metrics -feed fs-replication</p></div> <div class="section"> <h4>Relations<a name="Relations"></a></h4> <p>List all dimensions related to specified Dimension identified by dimension-type and dimension-name. Usage: $FALCON_HOME/bin/falcon metadata -relations -type [cluster_entity|feed_entity|process_entity|user|colo|tags|groups|pipelines] -name <<Dimension Name>></p> @@ -606,7 +668,7 @@ <footer> <div class="container-fluid"> - <div class="row span12">Copyright © 2013-2015 + <div class="row span12">Copyright © 2013-2016 <a href="http://www.apache.org";>Apache Software Foundation</a>. All Rights Reserved.
Modified: falcon/site/FalconDocumentation.html URL: http://svn.apache.org/viewvc/falcon/site/FalconDocumentation.html?rev=1730450&r1=1730449&r2=1730450&view=diff ============================================================================== --- falcon/site/FalconDocumentation.html (original) +++ falcon/site/FalconDocumentation.html Mon Feb 15 06:02:08 2016 @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia at 2015-12-08 + | Generated by Apache Maven Doxia at 2016-02-09 | Rendered using Apache Maven Fluido Skin 1.3.0 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20151208" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> <meta http-equiv="Content-Language" content="en" /> <title>Falcon - Contents</title> <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> @@ -58,7 +58,7 @@ - <li id="publishDate" class="pull-right">Last Published: 2015-12-08</li> + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> </ul> </div> @@ -306,6 +306,13 @@ <li> + <a href="0.9/index.html" title="0.9 (Current)"> + <i class="none"></i> + 0.9 (Current)</a> + </li> + + <li> + <a href="0.8/index.html" title="0.8"> <i class="none"></i> 0.8</a> @@ -421,7 +428,10 @@ <li><a href="#Security">Security</a></li> <li><a href="#Recipes">Recipes</a></li> <li><a href="#Monitoring">Monitoring</a></li> -<li><a href="#Backwards_Compatibility">Backwards Compatibility Instructions</a></li></ul></div> +<li><a href="#Email_Notification">Email Notification</a></li> +<li><a href="#Backwards_Compatibility">Backwards Compatibility Instructions</a></li> +<li><a href="#Proxyuser_support">Proxyuser support</a></li> +<li><a href="#ImportExport">Data Import and Export</a></li></ul></div> <div class="section"> <h3>Architecture<a name="Architecture"></a></h3></div> <div class="section"> @@ -432,7 +442,8 @@ <p><img src="Architecture.png" alt="" /></p></div> <div class="section"> <h4>Scheduler<a name="Scheduler"></a></h4> -<p>Falcon system has picked Oozie as the default scheduler. However the system is open for integration with other schedulers. Lot of the data processing in hadoop requires scheduling to be based on both data availability as well as time. Oozie currently supports these capabilities off the shelf and hence the choice.</p></div> +<p>Falcon system has picked Oozie as the default scheduler. However the system is open for integration with other schedulers. Lot of the data processing in hadoop requires scheduling to be based on both data availability as well as time. Oozie currently supports these capabilities off the shelf and hence the choice.</p> +<p>While the use of Oozie works reasonably well, there are scenarios where Oozie scheduling is proving to be a limiting factor. In its current form, Falcon relies on Oozie for both scheduling and for workflow execution, due to which the scheduling is limited to time based/cron based scheduling with additional gating conditions on data availability. Also, this imposes restrictions on datasets being periodic/cyclic in nature. In order to offer better scheduling capabilities, Falcon comes with its own native scheduler. Refer to <a href="./FalconNativeScheduler.html">Falcon Native Scheduler</a> for details.</p></div> <div class="section"> <h4>Control flow<a name="Control_flow"></a></h4> <p>Though the actual responsibility of the workflow is with the scheduler (Oozie), Falcon remains in the execution path, by subscribing to messages that each of the workflow may generate. When Falcon generates a workflow in Oozie, it does so, after instrumenting the workflow with additional steps which includes messaging via JMS. Falcon system itself subscribes to these control messages and can perform actions such as retries, handling late input arrival etc.</p></div> @@ -507,7 +518,7 @@ catalog:$database-name:$table-name#parti </pre></div></div> <div class="section"> <h3>Entity Management actions<a name="Entity_Management_actions"></a></h3> -<p>All the following operation can also be done using <a href="././restapi/ResourceList.html">Falcon's RESTful API</a>.</p></div> +<p>All the following operation can also be done using <a href="./Restapi/ResourceList.html">Falcon's RESTful API</a>.</p></div> <div class="section"> <h4>Submit<a name="Submit"></a></h4> <p>Entity submit action allows a new cluster/feed/process to be setup within Falcon. Submitted entity is not scheduled, meaning it would simply be in the configuration store within Falcon. Besides validating against the schema for the corresponding entity being added, the Falcon system would also perform inter-field validations within the configuration file and validations across dependent entities.</p></div> @@ -584,11 +595,11 @@ catalog:$database-name:$table-name#parti </clusters> </pre></div> -<p>The 'limit' attribute can be specified in units of minutes/hours/days/months, and a corresponding numeric value can be attached to it. It essentially instructs the system to retain data spanning from the current moment to the time specified in the attribute spanning backwards in time. Any data beyond the limit (past/future) is erased from the system.</p> +<p>The 'limit' attribute can be specified in units of minutes/hours/days/months, and a corresponding numeric value can be attached to it. It essentially instructs the system to retain data till the time specified in the attribute spanning backwards in time, from now. Any data older than that is erased from the system. By default, Falcon runs retention jobs up to the cluster validity end time. This causes the instances created within the endTime and "endTime - retentionLimit" to be retained forever. If the users do not want to retain any instances of the feed past the cluster validity end time, user should set property "falcon.retention.keep.instances.beyond.validity" to false in runtime.properties.</p> <p>With the integration of Hive, Falcon also provides retention for tables in Hive catalog.</p></div> <div class="section"> <h4>Example:<a name="Example:"></a></h4> -<p>If retention period is 10 hours, and the policy kicks in at time 't', the data retained by system is essentially the one in range [t-10h, t]. Any data before t-10h and after t is removed from the system.</p> +<p>If retention period is 10 hours, and the policy kicks in at time 't', the data retained by system is essentially the one after or equal to t-10h . Any data before t-10h is removed from the system.</p> <p>The 'action' attribute can attain values of DELETE/ARCHIVE. Based upon the tag value, the data eligible for removal is either deleted/archived.</p></div> <div class="section"> <h4>NOTE: Falcon 0.1/0.2 releases support Delete operation only<a name="NOTE:_Falcon_0.10.2_releases_support_Delete_operation_only"></a></h4></div> @@ -623,7 +634,16 @@ catalog:$database-name:$table-name#parti <p>If more than 1 source cluster is defined, then partition expression is compulsory, a partition can also have a constant. The expression is required to avoid copying data from different source location to the same target location, also only the data in the partition is considered for replication if it is present. The partitions defined in the cluster should be less than or equal to the number of partition declared in the feed definition.</p> <p>Falcon uses pull based replication mechanism, meaning in every target cluster, for a given source cluster, a coordinator is scheduled which pulls the data using distcp from source cluster. So in the above example, 2 coordinators are scheduled in backupCluster, one which pulls the data from sourceCluster1 and another from sourceCluster2. Also, for every feed instance which is replicated Falcon sends a JMS message on success or failure of replication instance.</p> <p>Replication can be scheduled with the past date, the time frame considered for replication is the minimum overlapping window of start and end time of source and target cluster, ex: if s1 and e1 is the start and end time of source cluster respectively, and s2 and e2 of target cluster, then the coordinator is scheduled in target cluster with start time max(s1,s2) and min(e1,e2).</p> -<p>A feed can also optionally specify the delay for replication instance in the cluster tag, the delay governs the replication instance delays. If the frequency of the feed is hours(2) and delay is hours(1), then the replication instance will run every 2 hours and replicates data with an offset of 1 hour, i.e. at 09:00 UTC, feed instance which is eligible for replication is 08:00; and 11:00 UTC, feed instance of 10:00 UTC is eligible and so on.</p></div> +<p>A feed can also optionally specify the delay for replication instance in the cluster tag, the delay governs the replication instance delays. If the frequency of the feed is hours(2) and delay is hours(1), then the replication instance will run every 2 hours and replicates data with an offset of 1 hour, i.e. at 09:00 UTC, feed instance which is eligible for replication is 08:00; and 11:00 UTC, feed instance of 10:00 UTC is eligible and so on.</p> +<p>If it is required to capture the feed replication metrics like TIMETAKEN, COPY, BYTESCOPIED, set the parameter "job.counter" to "true" in feed entity properties section. Captured metrics from instance will be populated to the GraphDB for display on UI.</p> +<p><b>Example:</b></p> +<div class="source"> +<pre> +<properties> + <property name="job.counter" value="true" /> +</properties> + +</pre></div></div> <div class="section"> <h4>Where is the feed path defined for File System Storage?<a name="Where_is_the_feed_path_defined_for_File_System_Storage"></a></h4> <p>It's defined in the feed xml within the location tag.</p> @@ -808,7 +828,7 @@ validity start="2009-01-01T00:00Z&q <div class="section"> <h3>Handling late input data<a name="Handling_late_input_data"></a></h3> <p>Falcon system can handle late arrival of input data and appropriately re-trigger processing for the affected instance. From the perspective of late handling, there are two main configuration parameters late-arrival cut-off and late-inputs section in feed and process entity definition that are central. These configurations govern how and when the late processing happens. In the current implementation (oozie based) the late handling is very simple and basic. The falcon system looks at all dependent input feeds for a process and computes the max late cut-off period. Then it uses a scheduled messaging framework, like the one available in Apache ActiveMQ or Java's DelayQueue to schedule a message with a cut-off period, then after a cut-off period the message is dequeued and Falcon checks for changes in the feed data which is recorded in HDFS in latedata file by falcons "record-size" action, if it detects any changes then the workflow will be rerun with the new set of feed da ta.</p> -<p><b>Example:</b> The late rerun policy can be configured in the process definition. Falcon supports 3 policies, periodic, exp-backoff and final. Delay specifies, how often the feed data should be checked for changes, also one needs to explicitly set the feed names in late-input which needs to be checked for late data.</p> +<p><b>Example:</b> For a process entity, the late rerun policy can be configured in the process definition. Falcon supports 3 policies, periodic, exp-backoff and final. Delay specifies, how often the feed data should be checked for changes, also one needs to explicitly set the feed names in late-input which needs to be checked for late data.</p> <div class="source"> <pre> <late-process policy="exp-backoff" delay="hours(1)"> @@ -817,7 +837,16 @@ validity start="2009-01-01T00:00Z&q </late-process> </pre></div> -<p><b>NOTE:</b> Feeds configured with table storage does not support late input data handling at this point. This will be made available in the near future.</p></div> +<p><b>NOTE:</b> Feeds configured with table storage does not support late input data handling at this point. This will be made available in the near future.</p> +<p>For a feed entity replication job, the default late data handling policy can be configured in the runtime.properties file. Since these properties are runtime.properties, they will take effect for all replication jobs completed subsequent to the change.</p> +<div class="source"> +<pre> + # Default configs to handle replication for late arriving feeds. + *.feed.late.allowed=true + *.feed.late.frequency=hours(3) + *.feed.late.policy=exp-backoff + +</pre></div></div> <div class="section"> <h3>Idempotency<a name="Idempotency"></a></h3> <p>All the operations in Falcon are Idempotent. That is if you make same request to the falcon server / prism again you will get a SUCCESSFUL return if it was SUCCESSFUL in the first attempt. For example, you submit a new process / feed and get SUCCESSFUL message return. Now if you run the same command / api request on same entity you will again get a SUCCESSFUL message. Same is true for other operations like schedule, kill, suspend and resume. Idempotency also by takes care of the condition when request is sent through prism and fails on one or more servers. For example prism is configured to send request to 3 servers. First user sends a request to SUBMIT a process on all 3 of them, and receives a response SUCCESSFUL from all of them. Then due to some issue one of the servers goes down, and user send a request to schedule the submitted process. This time he will receive a response with PARTIAL status and a FAILURE message from the server that has gone down. If the users check he wi ll find the process would have been started and running on the 2 SUCCESSFUL servers. Now the issue with server is figured out and it is brought up. Sending the SCHEDULE request again through prism will result in a SUCCESSFUL response from prism as well as other three servers, but this time PROCESS will be SCHEDULED only on the server which had failed earlier and other two will keep running as before.</p></div> @@ -932,8 +961,30 @@ config value: org.apache.falcon.metadata <h3>Monitoring<a name="Monitoring"></a></h3> <p>Monitoring and Operationalizing Falcon is detailed in <a href="./Operability.html">Operability</a>.</p></div> <div class="section"> +<h3>Email Notification<a name="Email_Notification"></a></h3> +<p>Notification for instance completion in Falcon is defined in <a href="./FalconEmailNotification.html">Falcon Email Notification</a>.</p></div> +<div class="section"> <h3>Backwards Compatibility<a name="Backwards_Compatibility"></a></h3> <p>Backwards compatibility instructions are <a href="./Compatibility.html">detailed here.</a></p></div> +<div class="section"> +<h3>Proxyuser support<a name="Proxyuser_support"></a></h3> +<p>Falcon supports impersonation or proxyuser functionality (identical to Hadoop proxyuser capabilities and conceptually similar to Unix 'sudo').</p> +<p>Proxyuser enables Falcon clients to submit entities on behalf of other users. Falcon will utilize Hadoop core's hadoop-auth module to implement this functionality.</p> +<p>Because proxyuser is a powerful capability, Falcon provides the following restriction capabilities (similar to Hadoop):</p> +<p></p> +<ul> +<li>Proxyuser is an explicit configuration on per proxyuser user basis.</li> +<li>A proxyuser user can be restricted to impersonate other users from a set of hosts.</li> +<li>A proxyuser user can be restricted to impersonate users belonging to a set of groups.</li></ul> +<p>There are 2 configuration properties needed in runtime properties to set up a proxyuser:</p> +<ul> +<li>falcon.service.ProxyUserService.proxyuser.#USER#.hosts: hosts from where the user #USER# can impersonate other users.</li> +<li>falcon.service.ProxyUserService.proxyuser.#USER#.groups: groups the users being impersonated by user #USER# must belong to.</li></ul> +<p>If these configurations are not present, impersonation will not be allowed and connection will fail. If more lax security is preferred, the wildcard value * may be used to allow impersonation from any host or of any user, although this is recommended only for testing/development.</p> +<p>-doAs option via CLI or doAs query parameter can be appended if using API to enable impersonation.</p></div> +<div class="section"> +<h3>ImportExport<a name="ImportExport"></a></h3> +<p>Data Import and Export is detailed in <a href="./ImportExport.html">Data Import and Export</a>.</p></div> </div> </div> </div> @@ -942,7 +993,7 @@ config value: org.apache.falcon.metadata <footer> <div class="container-fluid"> - <div class="row span12">Copyright © 2013-2015 + <div class="row span12">Copyright © 2013-2016 <a href="http://www.apache.org";>Apache Software Foundation</a>. All Rights Reserved. Added: falcon/site/FalconEmailNotification.html URL: http://svn.apache.org/viewvc/falcon/site/FalconEmailNotification.html?rev=1730450&view=auto ============================================================================== --- falcon/site/FalconEmailNotification.html (added) +++ falcon/site/FalconEmailNotification.html Mon Feb 15 06:02:08 2016 @@ -0,0 +1,459 @@ +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2016-02-09 + | Rendered using Apache Maven Fluido Skin 1.3.0 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20160209" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Falcon - Falcon Email Notification</title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> + + + +<script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> + + </head> + <body class="topBarDisabled"> + + + + <div class="container-fluid"> + <div id="banner"> + <div class="pull-left"> + <a href="../index.html" id="bannerLeft"> + <img src="images/falcon-logo.png" alt="Falcon" width="200px" height="45px"/> + </a> + </div> + <div class="pull-right"> <a href="http://www.apache.org"; id="bannerRight"> + <img src="images/apache-feather-tm.gif" alt="Falcon" height="45px"/> + </a> + </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="http://www.apache.org"; class="externalLink" title="Apache"> + Apache</a> + </li> + <li class="divider ">/</li> + <li class=""> + <a href="index.html" title="Falcon"> + Falcon</a> + </li> + <li class="divider ">/</li> + <li class="">Falcon Email Notification</li> + + + + <li id="publishDate" class="pull-right">Last Published: 2016-02-09</li> + + </ul> + </div> + + + <div class="row-fluid"> + <div id="leftColumn" class="span3"> + <div class="well sidebar-nav"> + + + <ul class="nav nav-list"> + <li class="nav-header">Falcon</li> + + <li> + + <a href="index.html" title="About"> + <i class="none"></i> + About</a> + </li> + + <li> + + <a href="slides/falcon-overview.html" title="Overview"> + <i class="none"></i> + Overview</a> + </li> + + <li> + + <a href="slides/falcon-user-guide.html" title="User Guide"> + <i class="none"></i> + User Guide</a> + </li> + + <li> + + <a href="GettingStarted.html" title="Getting Started"> + <i class="none"></i> + Getting Started</a> + </li> + + <li> + + <a href="FalconDocumentation.html" title="Architecture"> + <i class="none"></i> + Architecture</a> + </li> + + <li> + + <a href="InstallationSteps.html" title="Installation"> + <i class="none"></i> + Installation</a> + </li> + + <li> + + <a href="OnBoarding.html" title="On Boarding"> + <i class="none"></i> + On Boarding</a> + </li> + + <li> + + <a href="Operability.html" title="Operability"> + <i class="none"></i> + Operability</a> + </li> + + <li> + + <a href="EntitySpecification.html" title="Entity Specification"> + <i class="none"></i> + Entity Specification</a> + </li> + + <li> + + <a href="FalconCLI.html" title="Client (Falcon CLI)"> + <i class="none"></i> + Client (Falcon CLI)</a> + </li> + + <li> + + <a href="restapi/ResourceList.html" title="Rest API"> + <i class="icon-chevron-right"></i> + Rest API</a> + </li> + + <li> + + <a href="HiveIntegration.html" title="Hive Integration"> + <i class="none"></i> + Hive Integration</a> + </li> + + <li> + + <a href="Security.html" title="Security"> + <i class="none"></i> + Security</a> + </li> + <li class="nav-header">Project Information</li> + + <li> + + <a href="project-info.html" title="Summary"> + <i class="none"></i> + Summary</a> + </li> + + <li> + + <a href="mail-lists.html" title="Mailing Lists"> + <i class="none"></i> + Mailing Lists</a> + </li> + + <li> + + <a href="http://webchat.freenode.net?channels=apachefalcon&uio=d4"; class="externalLink" title="IRC"> + <i class="none"></i> + IRC</a> + </li> + + <li> + + <a href="team-list.html" title="Team"> + <i class="none"></i> + Team</a> + </li> + + <li> + + <a href="issue-tracking.html" title="Issue Tracking"> + <i class="none"></i> + Issue Tracking</a> + </li> + + <li> + + <a href="source-repository.html" title="Source Repository"> + <i class="none"></i> + Source Repository</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/FALCON/Index"; class="externalLink" title="Wiki"> + <i class="none"></i> + Wiki</a> + </li> + + <li> + + <a href="license.html" title="License"> + <i class="none"></i> + License</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/FALCON/News"; class="externalLink" title="News"> + <i class="none"></i> + News</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/FALCON/PoweredBy"; class="externalLink" title="Powered by"> + <i class="none"></i> + Powered by</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/FALCON/Acknowledgements"; class="externalLink" title="Acknowledgements"> + <i class="none"></i> + Acknowledgements</a> + </li> + + <li> + + <a href="http://blogs.apache.org/falcon/"; class="externalLink" title="Blog"> + <i class="none"></i> + Blog</a> + </li> + <li class="nav-header">Releases</li> + + <li> + + <a href="http://www.apache.org/dyn/closer.lua/falcon/0.8"; class="externalLink" title="0.8"> + <i class="none"></i> + 0.8</a> + </li> + + <li> + + <a href="http://www.apache.org/dyn/closer.lua/falcon/0.7"; class="externalLink" title="0.7"> + <i class="none"></i> + 0.7</a> + </li> + + <li> + + <a href="http://archive.apache.org/dist/falcon/0.6.1"; class="externalLink" title="0.6.1"> + <i class="none"></i> + 0.6.1</a> + </li> + + <li> + + <a href="http://archive.apache.org/dist/incubator/falcon/0.6-incubating"; class="externalLink" title="0.6-incubating"> + <i class="none"></i> + 0.6-incubating</a> + </li> + + <li> + + <a href="http://archive.apache.org/dist/incubator/falcon/0.5-incubating"; class="externalLink" title="0.5-incubating"> + <i class="none"></i> + 0.5-incubating</a> + </li> + + <li> + + <a href="http://archive.apache.org/dist/incubator/falcon/0.4-incubating"; class="externalLink" title="0.4-incubating"> + <i class="none"></i> + 0.4-incubating</a> + </li> + + <li> + + <a href="http://archive.apache.org/dist/incubator/falcon/0.3-incubating"; class="externalLink" title="0.3-incubating"> + <i class="none"></i> + 0.3-incubating</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/FALCON/Roadmap"; class="externalLink" title="Coming soon"> + <i class="none"></i> + Coming soon</a> + </li> + <li class="nav-header">Documentation</li> + + <li> + + <a href="0.9/index.html" title="0.9 (Current)"> + <i class="none"></i> + 0.9 (Current)</a> + </li> + + <li> + + <a href="0.8/index.html" title="0.8"> + <i class="none"></i> + 0.8</a> + </li> + + <li> + + <a href="0.7/index.html" title="0.7"> + <i class="none"></i> + 0.7</a> + </li> + + <li> + + <a href="0.6.1/index.html" title="0.6.1"> + <i class="none"></i> + 0.6.1</a> + </li> + + <li> + + <a href="0.6-incubating/index.html" title="0.6-incubating"> + <i class="none"></i> + 0.6-incubating</a> + </li> + + <li> + + <a href="0.5-incubating/index.html" title="0.5-incubating"> + <i class="none"></i> + 0.5-incubating</a> + </li> + + <li> + + <a href="0.4-incubating/index.html" title="0.4-incubating"> + <i class="none"></i> + 0.4-incubating</a> + </li> + + <li> + + <a href="0.3-incubating/index.html" title="0.3-incubating"> + <i class="none"></i> + 0.3-incubating</a> + </li> + <li class="nav-header">ASF</li> + + <li> + + <a href="http://www.apache.org/foundation/how-it-works.html"; class="externalLink" title="How Apache Works"> + <i class="none"></i> + How Apache Works</a> + </li> + + <li> + + <a href="http://www.apache.org/foundation/"; class="externalLink" title="Foundation"> + <i class="none"></i> + Foundation</a> + </li> + + <li> + + <a href="http://www.apache.org/foundation/sponsorship.html"; class="externalLink" title="Sponsoring Apache"> + <i class="none"></i> + Sponsoring Apache</a> + </li> + + <li> + + <a href="http://www.apache.org/foundation/thanks.html"; class="externalLink" title="Thanks"> + <i class="none"></i> + Thanks</a> + </li> + </ul> + + + + <hr class="divider" /> + + <div id="poweredBy"> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </div> + </div> + </div> + + + <div id="bodyColumn" class="span9" > + + <div class="section"> +<h3>Falcon Email Notification<a name="Falcon_Email_Notification"></a></h3> +<p>Falcon Email notification allows sending email notifications when scheduled feed/process instances complete. Email notification in feed/process entity can be defined as follows:</p> +<div class="source"> +<pre> +<process name="[process name]"> + ... + <notification type="email" to="[email protected],[email protected]"/> + ... +</process> + +</pre></div> +<p></p> +<ul> +<li><b>type</b> - specifies about the type of notification. <b>Note:</b> Currently "email" notification type is supported.</li> +<li><b>to</b> - specifies the address to send notifications to; multiple recipients may be provided as a comma-separated list.</li></ul> +<p>Falcon email notification requires some SMTP server configuration to be defined in startup.properties. Following are the values it looks for:</p> +<ul> +<li><b>falcon.email.smtp.host</b> - The host where the email action may find the SMTP server (localhost by default).</li> +<li><b>falcon.email.smtp.port</b> - The port to connect to for the SMTP server (25 by default).</li> +<li><b>falcon.email.from.address</b> - The from address to be used for mailing all emails (falcon@localhost by default).</li> +<li><b>falcon.email.smtp.auth</b> - Boolean property that specifies if authentication is to be done or not. (false by default).</li> +<li><b>falcon.email.smtp.user</b> - If authentication is enabled, the username to login as (empty by default).</li> +<li><b>falcon.email.smtp.password</b> - If authentication is enabled, the username's password (empty by default).</li></ul> +<p>Also ensure that email notification plugin is enabled in startup.properties to send email notifications:</p> +<ul> +<li><b>monitoring.plugins</b> - org.apache.falcon.plugin.EmailNotificationPlugin,org.apache.falcon.plugin.DefaultMonitoringPlugin</li></ul></div> + </div> + </div> + </div> + + <hr/> + + <footer> + <div class="container-fluid"> + <div class="row span12">Copyright © 2013-2016 + <a href="http://www.apache.org";>Apache Software Foundation</a>. + All Rights Reserved. + + </div> + + + + </div> + </footer> + </body> +</html>
