UPDATE:
I now have added a suite of 4 QA tests, which give a good impression of
how the new workflow features work.
view source here:
https://github.com/dsiebeck/openxpki/tree/feature%2Fworkflow
1) normal execution without pause/exception
perl 10_run_without_pause.t
1..12
ok 1 - Connect to server
ok 2 - Create test workflow
ok 3 - Expecting state STEP2
ok 4 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 5 - Expecting state STEP3
ok 6 - Proc-State should be "manual"
ok 7 - count try should be "0"
ok 8 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 9 - Expecting state SUCCESS
ok 10 - Proc-State should be "finished"
ok 11 - count try should be "0"
ok 12 - wakeup should be empty
2) execution with pause and wake up
perl 11_run_wit_pause.t
1..22
ok 1 - Connect to server
ok 2 - Create test workflow
ok 3 - Expecting state STEP2
ok 4 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 5 - Paused (iteration 1): state should remain STEP2
ok 6 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 7 - Paused (iteration 2): state should remain STEP2
ok 8 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 9 - Paused (iteration 3): state should remain STEP2
ok 10 - cause of pause should be saved in wf context
ok 11 - wake_up was called 2 times
ok 12 - Proc-State should be "pause"
ok 13 - count try should be "3"
ok 14 - wakeup should not be empty
ok 15 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 16 - execute again without pause: state should be STEP3
ok 17 - Proc-State should be "manual"
ok 18 - count try should be "0"
ok 19 - wakeup should be empty
ok 20 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 21 - final state should be SUCCESS
ok 22 - Proc-State should be "finished"
3) maximal allowed retries exceeded:
perl 12_count_try_exceeded.t
1..18
ok 1 - Connect to server
ok 2 - Create test workflow
ok 3 - Expecting state STEP2
ok 4 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 5 - Paused (iteration 1): state should remain STEP2
ok 6 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 7 - Paused (iteration 2): state should remain STEP2
ok 8 - Executing action I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST
ok 9 - Paused (iteration 3): state should remain STEP2
ok 10 - 4th execution should result in exception
ok 11 - need correct error exception
ok 12 - exception code should be saved in wf context
ok 13 - State after exception must still be STEP2
ok 14 - Proc-State should be "retry_exceeded"
ok 15 - resumed execution after exception
ok 16 - Action::resume() was called
ok 17 - Proc-State should be "manual"
ok 18 - count try should be "0"
4) "unexpected" exception while execution of action
perl 15_workflow_exception.t
1..13
ok 1 - Connect to server
ok 2 - Create test workflow
ok 3 - Expecting state STEP2
ok 4 - execution should provoke exception
ok 5 - need correct error exception
ok 6 - exception code should be saved in wf context
ok 7 - State after exception must still be STEP2
ok 8 - Proc-State should be "exception"
ok 9 - resumed execution after exception
ok 10 - Action::resume() was called
ok 11 - Proc-State should be "manual"
ok 12 - State after exception should be STEP3
ok 13 - exception code in wf context should be empty again
This is the definition of the test workflow:
<workflow>
<type>I18N_OPENXPKI_WF_TYPE_TESTING</type>
...
<state name="STEP1" autorun="yes">
<description></description>
<action name="I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST"
resulting_state="STEP2">
</action>
</state>
<state name="STEP2">
<description></description>
<action name="I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST"
resulting_state="STEP3"
retry_count="3" retry_interval="+0000000015">
</action>
</state>
...
</workflow>
As one sees here, the maximal allowed count of retries is 3. This
overwrites the definition in "activity.xml":
<action name="I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST"
class="OpenXPKI::Server::Workflow::Activity::WorkflowTest"
retry_count="5" retry_interval="+0000000005">
<field name="cause"/>
<field name="action"/>
</action>
Have a nice day!
Dieter
Am 18.05.2012 15:20, schrieb Dieter Siebeck:
Hi,
my name is Dieter Siebeck and I am quite new to the OpenXPKI project.
I am working with Oli Welter since nearly 3 months now, mainly on his
idea/draft of "paused workflows" (see his mail to the list from
2012-02-14).
Now I have published my first implementation of this feature. You can
see and download it here:
https://github.com/dsiebeck/openxpki/tree/feature%2Fworkflow
(this branch has been rebased on the devel branch 22 days ago.)
POD Documentation of most of the new methods is still missing, but I
wanted to publish the current state, to give other interested people
the opportunity to see and maybe correct the big picture.
Also missing are test files/scripts. I would appreciate some
recommendation, which place to look for the most up2date state-of-the
art testing files for this.
Now the idea of the implementation: basic need was to gain relatively
low-level control to the execution of workflows and activities. To
achieve this, the best (and maybe only) way seemed, to install an
"own" object of class OpenXPKI::Server::Workflow, which replaces (and
of course extends) the CPAN Workflow class (the file existed already,
but only for documentation purposes). Now its a matter of fact, that
the CPAN workflow Module isn't *really* designed for extendability.
You have no way to tell Workflow::Factory that you want "My::Workflow"
instead of "Workflow". And the Workflow class itself has not many
access- or hook points to control the execution of code.(for example,
in method Workflow::set(), the name of workflow-class is hardcoded as
the one and only class, which is allowed to use the private set-Method).
So I had to use some perl tricks to get around this limitations (if we
dont want to copy the hole Workflow-Module and extend it directly):
1) OpenXPKI::Workflow::Factory returns always an object of class
OpenXPKI::Server::Workflow. This is achieved via a friendly capturing
maneuver:
# OpenXPKI::Workflow::Factory
sub fetch_workflow {
my ( $self, $wf_type, $wf_id ) = @_;
my $wf = $self->SUPER::fetch_workflow($wf_type, $wf_id);
...
my $oxiWf = OpenXPKI::Server::Workflow->new($wf);
return $oxiWf;
#OpenXPKI::Server::Workflow
use base qw( Workflow )
sub new{
my ( $class, $BaseWorkflow ) = @_;
my $self = bless {}, $class;
#take over all properties from (original) base workflow
while ( my ( $key, $val ) = each %{$BaseWorkflow} ) {
$self->{$key} = $val;
}
...
return $self;
}
not the most beautiful code at all, but - thanks to perls open oop
structure - it works. OpenXPKI::Server::Workflow has all methods and
properties from the original workflow.
Another limitation: Workflow::Factory::fetch_workflow() uses only
certain data (two hardcoded keys) of the returned db data hash of the
persister.
Since there is no entry point/hook to alter this behaviour, a second
DB-fetch is needed to retrieve other data from the workflow-table (and
pass it to the workflow-object):
my $persister = $self->get_persister( $wf_config->{persister} );
my $wf_info = $persister->fetch_workflow($wf_id);
my $oxiWf = OpenXPKI::Server::Workflow->new($wf,$self,$wf_info);
So the full signature of the worklow constructor is:
my ( $class, $BaseWorkflow, $Factory, $wfData ) = @_;
(The reference of the factory is used to call the
Factory::save_workflow() method several time to store the current states)
2) OpenXPKI::Server::Workflow overrides the execute_action() - method.
As a wrapper method, it executes some code before and after
SUPER::execute_action(). Here is the place to determine/set the
appropriate "proc state" (which is a new db field and can have the
values :
* init
* wakeup
* resume
* running
* manual
* finished
* pause
* exception
* retry_exceeded
The first 3 are not expected to see in "normal" circumstances -
because they are quite volatile by design. "running" will be set,
before SUPER::execute_action/Activity::run is called. After eexecution
of one or more Activities, either "manual" (waiting for interaction)
or "finished" will be set.
If an exception occurs, the proc state "exception" is set. Also the
message code (not translation) will be saved in WF context (key
"wf_exception")
The two states "pause" and "retry_exceeded" concern the new "pause"
feature.
3) OpenXPKI::Server::Workflow::Activity has some new methods and
utility features:
first the pause() method: every derived concrete activity-class can
now (somewhere in its "run"-method call:
$self->pause('some cause description');
This will immediately end the execution of the "run"-method (this is
achieved via OpenXPKI::Server::Workflow::Pause, an "pseudo" exception
class) .
Additionally, $workflow->pause() is called, which does the hole work:
check the count of allowed retries, sets the "wake up" timestamp,
notifiy observers, write history entries etc. The workflow state
remains unchained (this is achieved via overriding the
Workflow::_get_next_state() method). If "max_retry" is exceeded, an
special exception will be thrown, which results in the special proc
state "retry_exceeded". The given "cause" of the pause will also be
stored in workflow context, key "wf_pause_msg".
For convenience, the Activity objects can now always "speak" to their
workflow via $self->{CURRENT_WORKFLOW}. Also I have a new utility
method "_get_wf_action_param($key)", which retrieves the param $key
for the current action in the xml-conf of current workflow.
4) Wake up: if an paused workflow is executed again (via command line,
the new "watcher" demon etc ), the proc state changes to "wakeup" and
the hook method Activity::wake_up() will be called. Afterwards,
proc-state changes to "running" and the normal execution begins.
5) Resume: if a workflow with proc state "exception" is executed
again, the proc state "resume" is set and the corresponing hook method
is called. Afterwards, proc-state changes to "running" and the normal
execution beginns.
6) The settings of "count try" and "resume_at" can be set/configured
(in this order)
manual set via $self->set_count_try/max_retry()
in action definition of the xml config of current workflow
in the xml config of the activity
Additionally, the retry-intervall can be set as second parameter of
$self->pause('cause', $retry_interval), which has the highest priority.
These are the main points of the implementation. Any existing Activity
class should run without any modifications and problems.
Of course, the workflow table has obtained some new fields:
workflow_proc_state, workflow_wakeup_at, workflow_count_try and
workflow_reap_at (the later corresponds to the "reap at"-Feature,
which is not implemented yet. Here's the full layout of my workflow table:
CREATE TABLE IF NOT EXISTS `workflow` (
`workflow_id` decimal(49,0) NOT NULL,
`pki_realm` varchar(255) DEFAULT NULL,
`workflow_type` varchar(255) DEFAULT NULL,
`workflow_version_id` decimal(49,0) DEFAULT NULL,
`workflow_state` varchar(255) DEFAULT NULL,
`workflow_last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
ON UPDATE CURRENT_TIMESTAMP,
`workflow_proc_state` varchar(20) DEFAULT NULL,
`workflow_wakeup_at` timestamp NULL DEFAULT NULL,
`workflow_count_try` int(11) DEFAULT NULL,
`workflow_reap_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`workflow_id`)
);
Next steps are:
1. documentation
2. write some tests (suggestions welcome, see above)
3. setting the "reap at" timestamp (as proposed by oli)
4. implementing the "watcher" process, which runs in background and
periodically checks, if it finds some paused workflows to "wake up".
Any comments, suggestions and corrections are highly appreciated!
Another question: where would be the best place to put an explanation
like the one in this email here? In one perl class? Or is there an
extra place for such docs, which give an overview over the interplay
of several classes?
have a nice weekend,
Dieter
PS:
Since test files are missing, this is the way to test the feature
(relying on I18N_OPENXPKI_WF_TYPE_TESTING
(OpenXPKI::Server::Workflow::Activity::WorkflowTest):
openxpkictl restart --debug
OpenXPKI::Server::Workflow:128,OpenXPKI::Server::Workflow::Activity:20
openxpkicmd I18N_OPENXPKI_WF_TYPE_TESTING (expected output:
"Workflow created (ID: xxxxx), State: STEP2")
openxpkicmd --wfid xxxxx --param action=pause --wfaction
I18N_OPENXPKI_WF_ACTION_WORKFLOWTEST I18N_OPENXPKI_WF_TYPE_TESTING
(expected output: "New Workflow State: STEP2")
execute last line more than 4 times, and the "max retry exceeded"
event occurs.
Alternatively, you can set "--param action=crash" to provoke an
exception.
to continue normally (till SUCCES) you need to give an arbitrary value
for "action", e.g. "abc" (due to some limitation in the parameter
parsing here)
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
OpenXPKI-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openxpki-devel
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
OpenXPKI-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openxpki-devel
