ZEST-128 - Added overrun detection and counting.
Project: http://git-wip-us.apache.org/repos/asf/zest-java/repo Commit: http://git-wip-us.apache.org/repos/asf/zest-java/commit/d85a93fe Tree: http://git-wip-us.apache.org/repos/asf/zest-java/tree/d85a93fe Diff: http://git-wip-us.apache.org/repos/asf/zest-java/diff/d85a93fe Branch: refs/heads/develop Commit: d85a93fe83f230ed12eccb40ca131d78fe05a198 Parents: 5d51f37 Author: Niclas Hedhman <[email protected]> Authored: Sat Nov 14 12:37:05 2015 +0800 Committer: Niclas Hedhman <[email protected]> Committed: Sat Nov 14 12:37:05 2015 +0800 ---------------------------------------------------------------------- libraries/scheduler/src/docs/scheduler.txt | 51 ++++++++++++++----- .../zest/library/scheduler/Execution.java | 1 - .../scheduler/SchedulerConfiguration.java | 5 -- .../zest/library/scheduler/TaskRunner.java | 52 +++++++++++++------- .../library/scheduler/schedule/Schedule.java | 7 ++- .../zest/library/scheduler/SchedulerTest.java | 3 +- .../scheduler/docsupport/SchedulerDocs.java | 44 ++++++++++++----- 7 files changed, 113 insertions(+), 50 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zest-java/blob/d85a93fe/libraries/scheduler/src/docs/scheduler.txt ---------------------------------------------------------------------- diff --git a/libraries/scheduler/src/docs/scheduler.txt b/libraries/scheduler/src/docs/scheduler.txt index 2a42cbe..a5ab18d 100644 --- a/libraries/scheduler/src/docs/scheduler.txt +++ b/libraries/scheduler/src/docs/scheduler.txt @@ -25,7 +25,7 @@ source=libraries/scheduler/dev-status.xml -------------- -The Scheduler library provides an easy way to schedule tasks using cron expressions if needed. +The Scheduler library provides an easy way to schedule tasks either for one time execution, CRON expression intervals or a custom algorithm. An optional Timeline allows you to browse past and future task runs. @@ -39,7 +39,7 @@ The SLF4J Logger used by this library is named "org.apache.zest.library.schedule Use SchedulerAssembler to add the Scheduler service to your Application. This Assembler provide a fluent api to programmatically configure configuration defaults and activate the -Timeline service assembly that allow to browse in past and future Task runs. +Timeline service assembly that allow browsing of past and future Task runs. Here is a full example: @@ -71,12 +71,12 @@ source=libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/Task. tag=task ---- -Tasks have a mandatory name property and an optional tags property. Theses properties get copied in +Tasks have a mandatory name property and an optional tags property. These properties get copied in each TimelineRecord created when the Timeline feature is activated. The run() method of Tasks is wrapped in a UnitOfWork when called by the Scheduler. Thanks to the UnitOfWork handling in Zest, you can split the work done in your Tasks in -several UnitOfWorks, the one around the Task#run() invocation will then be paused. +several UnitOfWorks. See UnitOfWork strategy below. Here is a simple example: @@ -92,11 +92,11 @@ Tasks are scheduled using the Scheduler service. This creates a Schedule associa the Task that allows you to know if it is running, to change it's cron expression and set it's durability. -By default, a Schedule is not durable. In other words, it do not survive an Application -restart. To make a Schedule durable, set it's durable property to true once its scheduled. +All Schedules are durable. In other words, it will survive an Application restart, and your application should +not schedule it again, as the Schedules are when the SchedulerService is activated after bootstrap. -There are two ways to schedule a Task using the Scheduler service: once or with a cron -expression. +There are three ways to schedule a Task using the Scheduler service: once or with a cron +expression or providing your own Schedule instance. === Scheduling once === @@ -108,7 +108,8 @@ source=libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/docsu tag=2 ----------- -Note that there is no point in making such a Schedule durable because it won't be run repeatedly. +Since all Schedules are durable, this "once" can be far into the future, and still be executed if the application +has been restarted. === Scheduling using a cron expression === @@ -137,11 +138,37 @@ To sum up, cron expressions used here have a precision of one second. The follow - @annualy or @yearly +== Overrun == +If the Schedule is running when it is time to be executed, then the execution will be skipped. This means that +the Task must complete within its period, or executions will be skipped. The sideeffect of that is that this +reduces thread exhaustion. + +When the Task execution is skipped, the overrun() property on the Schedule is incremented by 1. + == Durability == -Schedules can either be ethereal or durable, passed as an argument to the +Scheduler+. If it is a durable -schedule, then the Task must be an Entity Composite. +All Schedules are durable and the Task must be an Entity Composite. It also means that Tasks should be schedule +once and not on each reboot. The SchedulerService will load all Schedules on activation. + +While the Task is running, the Schedule will be held in the UnitOfWork of the TaskRunner. This means that IF the +Schedule is updated, i.e. cancelled or directly manipulating Schedule properties, the UnitOfWork.complete() will fail. +And if the Task is executing within the same UnitOfWork, any changes made will not take place. + +== UnitOfWork strategy == +The TaskRunner creates a UnitOfWork and the Task is excuted within that UnitOfWork. This may be very convenient, but +as noted in Durability above, that UnitOfWork will fail if Schedule properties are updated while the Task is +running. To avoid that the Task's operations suffers from this, OR if the Task wants a Retry/DiscardOn strategy +different from the default one, then the Task can simply declare its own. such as; + +[snippet,java] +----------- +source=libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/docsupport/SchedulerDocs.java +tag=strategy +----------- + +== Custom Schedules == +It is possible to implement Schedule directly. It must be declared as an EntityComposite in the assembly, and be +visible from the SchedulerService. No other considerations should be necessary. -When the == Observing the Timeline == Timeline allow to browse in past and future Task runs. This feature is available only if you activate http://git-wip-us.apache.org/repos/asf/zest-java/blob/d85a93fe/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/Execution.java ---------------------------------------------------------------------- diff --git a/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/Execution.java b/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/Execution.java index 8aecbe5..3bc120c 100644 --- a/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/Execution.java +++ b/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/Execution.java @@ -218,7 +218,6 @@ public interface Execution public void stop() throws Exception { - running = false; synchronized( this ) { http://git-wip-us.apache.org/repos/asf/zest-java/blob/d85a93fe/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/SchedulerConfiguration.java ---------------------------------------------------------------------- diff --git a/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/SchedulerConfiguration.java b/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/SchedulerConfiguration.java index 0ebc81d..66d7769 100644 --- a/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/SchedulerConfiguration.java +++ b/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/SchedulerConfiguration.java @@ -42,10 +42,5 @@ public interface SchedulerConfiguration @Optional @UseDefaults Property<Integer> workQueueSize(); - /** - * @return If the scheduler must stop without waiting for running tasks, optional and defaults to false. - */ - @UseDefaults - Property<Boolean> stopViolently(); // END SNIPPET: configuration } http://git-wip-us.apache.org/repos/asf/zest-java/blob/d85a93fe/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/TaskRunner.java ---------------------------------------------------------------------- diff --git a/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/TaskRunner.java b/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/TaskRunner.java index f506129..9f5d772 100644 --- a/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/TaskRunner.java +++ b/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/TaskRunner.java @@ -25,7 +25,7 @@ import org.apache.zest.api.injection.scope.Structure; import org.apache.zest.api.injection.scope.Uses; import org.apache.zest.api.structure.Module; import org.apache.zest.api.unitofwork.UnitOfWork; -import org.apache.zest.api.unitofwork.concern.UnitOfWorkPropagation; +import org.apache.zest.api.usecase.UsecaseBuilder; import org.apache.zest.library.scheduler.schedule.Schedule; import org.apache.zest.library.scheduler.schedule.ScheduleTime; @@ -39,31 +39,38 @@ public class TaskRunner private ScheduleTime schedule; @Override - @UnitOfWorkPropagation( usecase = "Task Runner" ) public void run() { + // TODO: (niclas) I am NOT happy with this implementation, requiring 3 UnitOfWorks to be created. 15-20 milliseconds on my MacBook. If there is a better way to detect overrun, two of those might not be needed. try { - UnitOfWork uow = module.currentUnitOfWork(); + UnitOfWork uow = module.newUnitOfWork( UsecaseBuilder.newUsecase( "Task Runner initialize" ) ); Schedule schedule = uow.get( Schedule.class, this.schedule.scheduleIdentity() ); - Task task = schedule.task().get(); - try + if( !schedule.running().get() ) // check for overrun. { - schedule.taskStarting(); - task.run(); - schedule.taskCompletedSuccessfully(); - } - catch( RuntimeException ex ) - { - Throwable exception = ex; - while(exception instanceof UndeclaredThrowableException) + try + { + schedule.taskStarting(); + uow.complete(); // This completion is needed to detect overrun + uow = module.newUnitOfWork( UsecaseBuilder.newUsecase( "Task Runner" ) ); + schedule = uow.get( schedule ); // re-attach the entity to the new UnitOfWork + Task task = schedule.task().get(); + task.run(); + uow.complete(); // Need this to avoid ConcurrentModificationException when there has been an overrun. + uow = module.newUnitOfWork( UsecaseBuilder.newUsecase( "Task Runner conclude" ) ); + schedule = uow.get( schedule ); // re-attach the entity to the new UnitOfWork + schedule.taskCompletedSuccessfully(); + } + catch( RuntimeException ex ) { - exception = ((UndeclaredThrowableException) ex).getUndeclaredThrowable(); + processException( schedule, ex ); } - schedule.taskCompletedWithException( exception ); - schedule.exceptionCounter().set( schedule.exceptionCounter().get() + 1 ); + schedule.executionCounter().set( schedule.executionCounter().get() + 1 ); + } + else + { + schedule.overrun().set( schedule.overrun().get() + 1 ); } - schedule.executionCounter().set( schedule.executionCounter().get() + 1 ); uow.complete(); } catch( Exception e ) @@ -71,4 +78,15 @@ public class TaskRunner throw new UndeclaredThrowableException( e ); } } + + private void processException( Schedule schedule, RuntimeException ex ) + { + Throwable exception = ex; + while( exception instanceof UndeclaredThrowableException ) + { + exception = ( (UndeclaredThrowableException) ex ).getUndeclaredThrowable(); + } + schedule.taskCompletedWithException( exception ); + schedule.exceptionCounter().set( schedule.exceptionCounter().get() + 1 ); + } } http://git-wip-us.apache.org/repos/asf/zest-java/blob/d85a93fe/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/schedule/Schedule.java ---------------------------------------------------------------------- diff --git a/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/schedule/Schedule.java b/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/schedule/Schedule.java index 7e9555f..48f2e6f 100644 --- a/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/schedule/Schedule.java +++ b/libraries/scheduler/src/main/java/org/apache/zest/library/scheduler/schedule/Schedule.java @@ -86,6 +86,12 @@ public interface Schedule extends EntityComposite @UseDefaults Property<Boolean> done(); + /** Returns the number of times the Schedule has been skipped, due to the Task was still running. + * + * @return the number of times the Schedule has been skipped, due to the Task was still running. + */ + @UseDefaults + Property<Long> overrun(); /** * Called just before the {@link org.apache.zest.library.scheduler.Task#run()} method is called. @@ -121,5 +127,4 @@ public interface Schedule extends EntityComposite * @return A String representing this schedule. */ String presentationString(); - } http://git-wip-us.apache.org/repos/asf/zest-java/blob/d85a93fe/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/SchedulerTest.java ---------------------------------------------------------------------- diff --git a/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/SchedulerTest.java b/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/SchedulerTest.java index 7bc8aa5..5c0a226 100644 --- a/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/SchedulerTest.java +++ b/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/SchedulerTest.java @@ -166,9 +166,8 @@ public class SchedulerTest uow.complete(); } - Thread.sleep(5000); await( usecase.name() ) - .atMost( 30, SECONDS ) + .atMost( 6, SECONDS ) .until( taskOutput( taskIdentity ), equalTo( 4 ) ); try( UnitOfWork uow = module.newUnitOfWork( usecase ) ) http://git-wip-us.apache.org/repos/asf/zest-java/blob/d85a93fe/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/docsupport/SchedulerDocs.java ---------------------------------------------------------------------- diff --git a/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/docsupport/SchedulerDocs.java b/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/docsupport/SchedulerDocs.java index 6de65b7..9a78638 100644 --- a/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/docsupport/SchedulerDocs.java +++ b/libraries/scheduler/src/test/java/org/apache/zest/library/scheduler/docsupport/SchedulerDocs.java @@ -22,21 +22,25 @@ import org.apache.zest.api.association.Association; import org.apache.zest.api.injection.scope.Service; import org.apache.zest.api.injection.scope.This; import org.apache.zest.api.property.Property; +import org.apache.zest.api.unitofwork.concern.UnitOfWorkDiscardOn; +import org.apache.zest.api.unitofwork.concern.UnitOfWorkPropagation; +import org.apache.zest.api.unitofwork.concern.UnitOfWorkRetry; import org.apache.zest.library.scheduler.Scheduler; import org.apache.zest.library.scheduler.Task; import org.apache.zest.library.scheduler.schedule.Schedule; import org.apache.zest.library.scheduler.timeline.Timeline; - public class SchedulerDocs { -// START SNIPPET: timeline - @Service Timeline timeline; + // START SNIPPET: timeline + @Service + Timeline timeline; // END SNIPPET: timeline -// START SNIPPET: 2 - @Service Scheduler scheduler; + // START SNIPPET: 2 + @Service + Scheduler scheduler; public void method() { @@ -45,12 +49,13 @@ public class SchedulerDocs // myTask will be run in 10 seconds from now } -// END SNIPPET: 2 - MyTaskEntity todo() { + // END SNIPPET: 2 + MyTaskEntity todo() + { return null; } -// START SNIPPET: 1 + // START SNIPPET: 1 interface MyTaskEntity extends Task { Property<String> myTaskState(); @@ -60,19 +65,34 @@ public class SchedulerDocs class MyTaskMixin implements Runnable { - @This MyTaskEntity me; + @This + MyTaskEntity me; @Override public void run() { - me.myTaskState().set(me.anotherEntity().get().doSomeStuff(me.myTaskState().get())); + me.myTaskState().set( me.anotherEntity().get().doSomeStuff( me.myTaskState().get() ) ); } } -// END SNIPPET: 1 + // END SNIPPET: 1 interface AnotherEntity { - String doSomeStuff(String p); + String doSomeStuff( String p ); } + public class MyTask implements Runnable + { + + // START SNIPPET: strategy + @Override + @UnitOfWorkRetry( retries = 3 ) + @UnitOfWorkDiscardOn( IllegalArgumentException.class ) + @UnitOfWorkPropagation( value = UnitOfWorkPropagation.Propagation.REQUIRES_NEW, usecase = "Load Data" ) + public void run() + { + // END SNIPPET: strategy + + } + } } \ No newline at end of file
