I dont fully understand how you have implemented the ccb handling in the
AMF so bear with me if I
ask apparently stupid questions.
The validation part I assumed was being done purely in the active AMFD OI.
Below you mention "Validation may be queued in the AMFND".
Is that a typo or do you really mean that validation also involves the
AMFNDs ?
The core part of validation is to evaluate the logical correctness of
the change proposed by the CCB
relative to the prior state of the AMF config data. That core part
should thus in principle only need
the current config state and the proposed CCB as input.
Validation may also include resource checking and even resource allocation.
And insufficient resources may then be one cause for "validation" to fail.
If resource checking is included in the AMF validation then I could
understand why the
AMFNDs could need to be involved.
At any given node, the apply callback (to the OI and to any appliers)
always arrives in the same
order. In particular, thje apply of course always arrives after the
completed callback for the same ccb
and the apply callback always arrives after the ccb has been commited to
imm-ram locally at that ccb.
If there is only one main OI involved (not counting appliers here) which
should be the case for the AMF,
then that AMF OI will actually serialize the commit of its ccbs. The OI
can only procerss one completed callback
at a time. It has to reply to one completed callback before it can
receive the next completed callback.
And as soon as it has replied OK on a completed callback, that reply
message will be fevs ordered (pass the
active IMMD) before the reply on the next validated CCB gets fevs ordered.
This means that the apply events are also serialized by the AMFD OI.
Apply's are not acke'ed though so the subsequent apply callback may
arrive before the current apply
processing has finished. IF the processing is done by the same thread
then this would not happen
because the thread would be busy untill it was done with the apply and
only then go back to poll
and fine the next apply callback.
In principle it should be possible for the AMFD to just process the
validations for ccbs in the order they arrive.
Validation should in principle not interfere with the distributed apply
still being processed for the previous CCB.
The only exception would be if you have some kind of resource check.
By that I dont mean any check the the previous apply is done. That
should not matter in itself.
The only resource problem that could matter in validation is if the
apply involves using some resource of limited
supply and the AMFD main OI could not manage to guaranteed secure such a
resource.
In that case you should return ERR_NO_RESOURCES on the validated callback.
The implementation of the apply becomes cleanest if you can use the
applier interface to solve the distributed
apply. At least try to use the applier interface to trigger apply
processing at any particular processor, even if
you have other inter-processor communication involved. Any
inter-processor communication involved in
the apply processign of one ccb should use admin-operations as the
communication mechanism (if needed),
since it also goes over fevs and thus is guaranteed to arrive at the
receiver *after* imm apply for that ccb
has been processed by the imm also at that processor.
So in summary. I dont understand how/why validation of a CCB interferes
with the apply of a previous ccb.
I do understand that the apply of a ccb could interfere with the apply
of a previous ccb. But my solution
would there be to queue the apply "job" if it arrives before the
previous apply is finished.
A few comments inlinded belwo.
Nagendra Kumar wrote:
> Hi Hans/Andes,
>
> I am exploring the options and i have the below observation:
> 1. Validation may be queue in amfnd, but the processing has not been done and
> it may fail, so we can't allow second ccb operation in waiting to be applied.
I think there is some confusion/mixing between validation and apply
here. There should not be a problem in logically validating the next ccb
while
the previous one is still being applied. Only exception would be
inability to guarantee some resource in the next apply.
Perhaps the problem is that your logical validation "reads" from some
AMF representation and the update of that representation is incomplete ?
But you should be able to read from the imm using the unsafe
iteration/accessor API. As long as the OI is only reading objects that
it itself is
the OI for, then the OI technically does have a safe read. Currently it
needs to overlay the after immage of the ccb updates, which it has received
in the operation callbacks. This becomes much simpler with the safeRead
API.
> The error handling will be too complicated. We need to sequence the apply.
>
Here you say "sequence the apply". But the apply of AMF data is
sequenced already by the AMF OI.
So the problem can not be sequence, but overlap in the sense of the next
apply arriving before the previous is completed.
Thus the sequence is correct, but there is a problem synchronizing the
end of one apply with the acceptance to start the next.
Hwere I suggest you simply queue the apply. This may mean that you need
to keep the immutils ccb structure until that apply
job can start processing.
> The other practical problem is apply another ccb is : We are allowing ccb
> callbacks to send Act assignment first and then subsequently Standby
> assignments.
>
When you say "ccb callbacks" I assume you mean the apply callback only.
The active is the main OI and the standby is the AMFD applier.
If they just do their apply processing when they recveive the apply,
there should be no problem.
This as long as the apply processing is processor local at the active
and the standby.
Payloads should wither use the applier interface to get a correctly
ordered and in-sync-with-local-imm-ram apply event.
Alternatively, if you insist on sending an AMFD->AMFND message, then you
really need to make it fevs synchronous
for the receiver by using an admin-op. This will be simplified when #799
is implemented.
> When we apply second ccb even if first one is undergoing, we need to
> differentiate this one for sending Act assignment first for this ccb and when
> assignment response comes back, then we need to send Standby assignment for
> others SUs for that csi(protection group).
>
> 2. We can't handle many csis in the same SI to get applied at a time having
> consideration of CSI-CSI dep.
>
One other idea or questions is: Why can you not allow more than one CSI
to be created in ONE CCB?
I mean if all these CSI are being created so closely, it sounds like
they belong together in one transaction.
This would give atomicity to the entire change and make it much more
efficient.
You would validate everything as one big job and apply all the new CSIs
as one apply.
> 3. I explored whether we can have delay logic implemented in apply callback,
The only delay logic that makes sense to me is to queue the actual start
of the next apply (next ccb) as a whole.
You may then need to keep the immutils ccb operation colelction as part
of that queue element.
> but it looks pretty messy when the first one csi addition has failed and
> system is not stable and we would like to add another csi.
>
> To me, it only looks TRY_AGAIN option to be provided to Amf from imm oi
> completed callback, so that Amf can at least sequence it.
>
In theory the IMM could implement TRY_AGAIN like behavior for
ERR_NO_RESOURCES replies on validation (completed callbacks).
It can not do it for apply callbacks. Apply is commit and if you have
acked completed then you are commited to commit. I dont want to
implement "hesitant commit" (dont know how to do it. its probably
impossible.). The apply callback has no reply and no return code
precisely because it is the end of the IMMs responsibility for any given
CCB towards any OI or applier.
But as explained above validation should be a logical thing and not be
concerned with the progress of the previous apply.
AndersBj
> Hi Anders Bj:
> We changed TRY_AGAIN to BAD_OPERATION to get compliance with Specs, am i
> right? Can we make it possible to implement TRY_AGAIN in imm?
>
> Thanks
> -Nagu
>
>
> ---
>
> ** [tickets:#750] AMF: creating multiple CSIs are rejected**
>
> **Status:** accepted
> **Milestone:** future
> **Created:** Mon Jan 27, 2014 10:24 AM UTC by Hans Feldt
> **Last Updated:** Tue Mar 18, 2014 10:03 AM UTC
> **Owner:** Nagendra Kumar
>
> AMF supports adding one CSI per CCB to an unlocked SI. This means if several
> CSIs are to be added, many CCBs will be performed by SMF (from a campaign).
> However since the first CSI create is still "pending" when the second CSI one
> comes to AMF, the second one is can be rejected (by AMF). All depending in
> timing.
>
> One problem is that the OI completed callback does not give the OI any chance
> to say TRYAGAIN. It is either accept or reject.
>
> Adding a delay to the campaign between each CSI create is not a plausible
> solution thus AMF needs to support this case.
>
>
> ---
>
> Sent from sourceforge.net because you indicated interest in
> <https://sourceforge.net/p/opensaf/tickets/750/>
>
> To unsubscribe from further messages, please visit
> <https://sourceforge.net/auth/subscriptions/>
>
---
** [tickets:#750] AMF: creating multiple CSIs are rejected**
**Status:** accepted
**Milestone:** future
**Created:** Mon Jan 27, 2014 10:24 AM UTC by Hans Feldt
**Last Updated:** Tue Mar 18, 2014 10:03 AM UTC
**Owner:** Nagendra Kumar
AMF supports adding one CSI per CCB to an unlocked SI. This means if several
CSIs are to be added, many CCBs will be performed by SMF (from a campaign).
However since the first CSI create is still "pending" when the second CSI one
comes to AMF, the second one is can be rejected (by AMF). All depending in
timing.
One problem is that the OI completed callback does not give the OI any chance
to say TRYAGAIN. It is either accept or reject.
Adding a delay to the campaign between each CSI create is not a plausible
solution thus AMF needs to support this case.
---
Sent from sourceforge.net because opensaf-tickets@lists.sourceforge.net is
subscribed to http://sourceforge.net/p/opensaf/tickets/
To unsubscribe from further messages, a project admin can change settings at
http://sourceforge.net/p/opensaf/admin/tickets/options. Or, if this is a
mailing list, you can unsubscribe from the mailing list.
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Opensaf-tickets mailing list
Opensaf-tickets@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets
