FALCON-1419 Document ability to disable oozie dryRun during entity schedule actions. Contributed by Balu Vellanki.
Project: http://git-wip-us.apache.org/repos/asf/falcon/repo Commit: http://git-wip-us.apache.org/repos/asf/falcon/commit/9d31172f Tree: http://git-wip-us.apache.org/repos/asf/falcon/tree/9d31172f Diff: http://git-wip-us.apache.org/repos/asf/falcon/diff/9d31172f Branch: refs/heads/0.7 Commit: 9d31172f47f3350e878eeacd551672235979f9b6 Parents: fbb4d31 Author: Ajay Yadava <[email protected]> Authored: Fri Aug 28 18:00:46 2015 +0530 Committer: Ajay Yadav <[email protected]> Committed: Tue Sep 1 10:14:27 2015 +0530 ---------------------------------------------------------------------- CHANGES.txt | 2 ++ common/src/main/resources/runtime.properties | 5 ++++- docs/src/site/twiki/FalconCLI.twiki | 8 ++++++-- docs/src/site/twiki/restapi/EntitySchedule.twiki | 4 +++- docs/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki | 3 ++- docs/src/site/twiki/restapi/EntityTouch.twiki | 3 ++- docs/src/site/twiki/restapi/EntityUpdate.twiki | 3 ++- docs/src/site/twiki/restapi/EntityValidate.twiki | 7 ++++--- src/conf/runtime.properties | 3 +++ 9 files changed, 28 insertions(+), 10 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/CHANGES.txt ---------------------------------------------------------------------- diff --git a/CHANGES.txt b/CHANGES.txt index f7e9127..1b60f7f 100755 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -13,6 +13,8 @@ Trunk (Unreleased) FALCON-796 Enable users to triage data processing issues through falcon (Ajay Yadava) IMPROVEMENTS + FALCON-1419 Document ability to disable oozie dryRun during entity schedule actions(Balu Vellanki via Ajay Yadava) + FALCON-1174 Ability to disable oozie dryrun while scheduling or updating the falcon entity(Balu Vellanki via Ajay Yadava) FALCON-1374 Remove the cap on numResults(Pragya Mittal via Ajay Yadava) http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/common/src/main/resources/runtime.properties ---------------------------------------------------------------------- diff --git a/common/src/main/resources/runtime.properties b/common/src/main/resources/runtime.properties index 086bca0..23ecc16 100644 --- a/common/src/main/resources/runtime.properties +++ b/common/src/main/resources/runtime.properties @@ -28,4 +28,7 @@ # Default configs to handle replication for late arriving feeds. *.feed.late.allowed=true *.feed.late.frequency=hours(3) -*.feed.late.policy=exp-backoff \ No newline at end of file +*.feed.late.policy=exp-backoff + +# If true, Falcon skips oozie dryrun while scheduling entities. +*.falcon.skip.dryrun=false \ No newline at end of file http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/docs/src/site/twiki/FalconCLI.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/FalconCLI.twiki b/docs/src/site/twiki/FalconCLI.twiki index 9203699..8bf3155 100644 --- a/docs/src/site/twiki/FalconCLI.twiki +++ b/docs/src/site/twiki/FalconCLI.twiki @@ -20,6 +20,8 @@ Once submitted, an entity can be scheduled using schedule option. Process and fe Usage: $FALCON_HOME/bin/falcon entity -type [process|feed] -name <<name>> -schedule +Optional Arg : -skipDryRun. When this argument is specified, Falcon skips oozie dryrun. + Example: $FALCON_HOME/bin/falcon entity -type process -name sampleProcess -schedule @@ -60,8 +62,6 @@ Optional Args : -fields <<field1,field2>> <a href="./Restapi/EntityList.html">Optional params described here.</a> - - ---+++Summary Summary of entities of a particular type and a cluster will be listed. Entity summary has N most recent instances of entity. @@ -83,6 +83,8 @@ not allowed. Usage: $FALCON_HOME/bin/falcon entity -type [feed|process] -name <<name>> -update -file <<path_to_file>> +Optional Arg : -skipDryRun. When this argument is specified, Falcon skips oozie dryrun. + Example: $FALCON_HOME/bin/falcon entity -type process -name HourlyReportsGenerator -update -file /process/definition.xml @@ -93,6 +95,8 @@ Force Update operation allows an already submitted/scheduled entity to be update Usage: $FALCON_HOME/bin/falcon entity -type [feed|process] -name <<name>> -touch +Optional Arg : -skipDryRun. When this argument is specified, Falcon skips oozie dryrun. + ---+++Status Status returns the current status of the entity. http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/docs/src/site/twiki/restapi/EntitySchedule.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/restapi/EntitySchedule.twiki b/docs/src/site/twiki/restapi/EntitySchedule.twiki index dd97141..1ebdf30 100644 --- a/docs/src/site/twiki/restapi/EntitySchedule.twiki +++ b/docs/src/site/twiki/restapi/EntitySchedule.twiki @@ -1,4 +1,4 @@ ----++ POST /api/entities/schedule/:entity-type/:entity-name +---++ POST /api/entities/schedule/:entity-type/:entity-name?skipDryRun=false * <a href="#Description">Description</a> * <a href="#Parameters">Parameters</a> * <a href="#Results">Results</a> @@ -10,6 +10,8 @@ Schedule an entity. ---++ Parameters * :entity-type can either be a feed or a process. * :entity-name is name of the entity. + * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true. + ---++ Results Result of the schedule command. http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/docs/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki b/docs/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki index 042a5fb..a6516bb 100644 --- a/docs/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki +++ b/docs/src/site/twiki/restapi/EntitySubmitAndSchedule.twiki @@ -1,4 +1,4 @@ ----++ POST /api/entities/submitAndSchedule/:entity-type +---++ POST /api/entities/submitAndSchedule/:entity-type?skipDryRun=false * <a href="#Description">Description</a> * <a href="#Parameters">Parameters</a> * <a href="#Results">Results</a> @@ -9,6 +9,7 @@ Submits and schedules an entity. ---++ Parameters * :entity-type can either be a feed or a process. + * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true. ---++ Results Result of the submit and schedule command. http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/docs/src/site/twiki/restapi/EntityTouch.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/restapi/EntityTouch.twiki b/docs/src/site/twiki/restapi/EntityTouch.twiki index ffc7967..69e6d1b 100644 --- a/docs/src/site/twiki/restapi/EntityTouch.twiki +++ b/docs/src/site/twiki/restapi/EntityTouch.twiki @@ -1,4 +1,4 @@ ----++ POST api/entities/touch/:entity-type/:entity-name +---++ POST api/entities/touch/:entity-type/:entity-name?skipDryRun=true * <a href="#Description">Description</a> * <a href="#Parameters">Parameters</a> * <a href="#Results">Results</a> @@ -10,6 +10,7 @@ Force updates the entity. ---++ Parameters * :entity-type can be feed or process. * :entity-name is name of the feed or process. + * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true. ---++ Results Result of the validation. http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/docs/src/site/twiki/restapi/EntityUpdate.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/restapi/EntityUpdate.twiki b/docs/src/site/twiki/restapi/EntityUpdate.twiki index 0e5f56f..ba618df 100644 --- a/docs/src/site/twiki/restapi/EntityUpdate.twiki +++ b/docs/src/site/twiki/restapi/EntityUpdate.twiki @@ -1,4 +1,4 @@ ----++ POST api/entities/update/:entity-type/:entity-name +---++ POST api/entities/update/:entity-type/:entity-name?skipDryRun=false * <a href="#Description">Description</a> * <a href="#Parameters">Parameters</a> * <a href="#Results">Results</a> @@ -10,6 +10,7 @@ Updates the submitted entity. ---++ Parameters * :entity-type can be feed or process. * :entity-name is name of the feed or process. + * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true. ---++ Results Result of the validation. http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/docs/src/site/twiki/restapi/EntityValidate.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/restapi/EntityValidate.twiki b/docs/src/site/twiki/restapi/EntityValidate.twiki index 3fe0f83..86de630 100644 --- a/docs/src/site/twiki/restapi/EntityValidate.twiki +++ b/docs/src/site/twiki/restapi/EntityValidate.twiki @@ -1,4 +1,4 @@ ----++ POST api/entities/validate/entity-type +---++ POST api/entities/validate/entity-type?skipDryRun=false * <a href="#Description">Description</a> * <a href="#Parameters">Parameters</a> * <a href="#Results">Results</a> @@ -8,7 +8,8 @@ Validates the submitted entity. ---++ Parameters -:entity-type can be cluster, feed or process. + * :entity-type can be cluster, feed or process. + * skipDryRun : Optional query param, Falcon skips oozie dryrun when value is set to true. ---++ Results Result of the validation. @@ -44,7 +45,7 @@ POST http://localhost:15000/api/entities/validate/cluster ---+++ Rest Call <verbatim> -POST http://localhost:15000/api/entities/validate/feed +POST http://localhost:15000/api/entities/validate/feed?skipDryRun=true <?xml version="1.0" encoding="UTF-8"?> <!-- Hourly sample input data --> http://git-wip-us.apache.org/repos/asf/falcon/blob/9d31172f/src/conf/runtime.properties ---------------------------------------------------------------------- diff --git a/src/conf/runtime.properties b/src/conf/runtime.properties index b8ebe39..b31e6a3 100644 --- a/src/conf/runtime.properties +++ b/src/conf/runtime.properties @@ -40,3 +40,6 @@ falcon.current.colo=local *.feed.late.allowed=true *.feed.late.frequency=hours(3) *.feed.late.policy=exp-backoff + +# If true, Falcon skips oozie dryrun while scheduling entities. +*.falcon.skip.dryrun=false
