Hi Vijay,
I have started to create a CCB handler handling all types of modifications of
the IMM model where a CCB is involved (object create, modify and delete). It is
created as a “separate module” that handles everything that has to do with IMM.
This means, all steps involved in creating a CCB handling admin owner etc. It
will do recovery when possible which includes everything from TRY_AGAIN to
aborted CCB. It is handled using a well-defined and documented API. I will also
include some shell commands that can be used as “demo” but also can be used for
testing. These shell command will be built if –enable-tests is set when OpenSAF
build is configured.
The plan is to replace all “inline” IMM handling regarding CCBs.
I have already implemented most of the code needed for a “create” and will soon
start to test that part. When I have something that at least have some part
working I will push for review and then push other parts for review in
increments. In the mean time you can look at the API (as it looks so far). I
have attached the .h file
Thanks
Lennart
From: Vijay Roy [mailto:vijay....@oracle.com]
Sent: den 20 november 2017 09:57
To: Lennart Lund <lennart.l...@ericsson.com>; Rafael Odzakow
<rafael.odza...@ericsson.com>; syam.tall...@oracle.com
Cc: opensaf-devel@lists.sourceforge.net
Subject: RE: [PATCH 1/1] smf: SMF created a CCB to create smfRollbackElement
object, but CCB was aborted due to IMM [#2676]
Hi Lennart,
In reality myself and Syam are also thinking on the same lines for refactoring
CCB & Admin operations flows.
In addition we need to have an eye on CLI & Customized Callback Actions.
Better we discuss more in detail to baseline the solution. Please suggest.
Regards
Vijay
From: Lennart Lund [mailto:lennart.l...@ericsson.com]
Sent: Friday, November 17, 2017 9:47 PM
To: Vijay Roy <vijay....@oracle.com<mailto:vijay....@oracle.com>>; Rafael
Odzakow <rafael.odza...@ericsson.com<mailto:rafael.odza...@ericsson.com>>;
Lennart Lund <lennart.l...@ericsson.com<mailto:lennart.l...@ericsson.com>>
Cc:
opensaf-devel@lists.sourceforge.net<mailto:opensaf-devel@lists.sourceforge.net>
Subject: RE: [PATCH 1/1] smf: SMF created a CCB to create smfRollbackElement
object, but CCB was aborted due to IMM [#2676]
Hi Vijay
It seems as this is getting a bit messy. These loops are added all over the
place and (at least) in one case one of the loops did not work since it
included a function that does more than one thing and at least one thing the
function does did not fail the first time and can because of that not be done
again even if some other part failed etc.
It seem as it is mostly IMM handling that is simplified so that possibilities
of recovery is not used.
I think that if we continue to fix things like this we will soon lose control.
I suggest that we do the following instead:
We need to handle recovery and “real” fails:
1. Creating CCBs:
Doing this is rather complicated if all rules shall be followed and all
recovery possibilities shall be used.
In SMF this is done in many places and in most cases only TRY_AGAIN is handled
(there is also a lot of redundant code), everything else is considered a Fail
(will fail the campaign).
I am currently working on a “generic” solution that can be used in all places
where the IMM model shall be modified using a CCB. I think this is the most
complicated part.
2. Admin operations:
This is also done in a simplified way in many places and in different ways with
redundant code. If we fix CCB handling to be able to recover from e.g.
BAD_HANDLE we will still fail here so also Admin operations should be handled
in a “generic” way. I think this is simpler to do than CCB handling.
3. There are probably other OM operations as well that needs to be fixed
What is your opinion about this? After CCB handling, what do you think is most
important?
Regards
Lennart
From: Vijay Roy [mailto:vijay....@oracle.com]
Sent: den 17 november 2017 07:13
To: Rafael Odzakow
<rafael.odza...@ericsson.com<mailto:rafael.odza...@ericsson.com>>
Cc:
opensaf-devel@lists.sourceforge.net<mailto:opensaf-devel@lists.sourceforge.net>;
Lennart Lund <lennart.l...@ericsson.com<mailto:lennart.l...@ericsson.com>>
Subject: RE: [PATCH 1/1] smf: SMF created a CCB to create smfRollbackElement
object, but CCB was aborted due to IMM [#2676]
Hi Rafael,
We need the while loop in “SmfCampaignWrapup::executeCampComplete()” as we
encountered the issue at Wrapup too while testing.
Thanks
Vijay
-----Original Message-----
From: Rafael Odzakow [mailto:rafael.odza...@ericsson.com]
Sent: Thursday, November 16, 2017 8:54 PM
To: Vijay Roy <vijay....@oracle.com<mailto:vijay....@oracle.com>>
Cc:
opensaf-devel@lists.sourceforge.net<mailto:opensaf-devel@lists.sourceforge.net>;
Lennart Lund <lennart.l...@ericsson.com<mailto:lennart.l...@ericsson.com>>
Subject: Re: [PATCH 1/1] smf: SMF created a CCB to create smfRollbackElement
object, but CCB was aborted due to IMM [#2676]
What about the added while loops in
SmfCampaignWrapup::executeCampComplete() should they be removed now with this
addition?
On 11/16/2017 10:44 AM, Vijay Roy wrote:
> Handling ERROR_EXIST in smfRollbackElement creation and handling
> TRY_AGAIN in immCCBOperations.
> ---
> src/smf/smfd/SmfUpgradeAction.cc | 26 +++++++++++++++++++-------
> 1 file changed, 19 insertions(+), 7 deletions(-)
>
> diff --git a/src/smf/smfd/SmfUpgradeAction.cc
> b/src/smf/smfd/SmfUpgradeAction.cc
> index 94c3dfd..af75cd7 100644
> --- a/src/smf/smfd/SmfUpgradeAction.cc
> +++ b/src/smf/smfd/SmfUpgradeAction.cc
> @@ -28,6 +28,7 @@
> #include "smf/smfd/SmfUtils.h"
> #include "smfd.h"
> #include "smf/smfd/SmfTargetTemplate.h"
> +#include "base/time.h"
>
> /* ========================================================================
> * DEFINITIONS
> @@ -460,6 +461,7 @@ SaAisErrorT SmfImmCcbAction::execute(SaImmOiHandleT
> i_oiHandle,
> const std::string* i_rollbackDn) {
> SaAisErrorT result = SA_AIS_OK;
> SmfRollbackCcb* rollbackCcb = NULL;
> + base::Timer doImmOpTimer(60000);
>
> TRACE_ENTER();
>
> @@ -473,8 +475,8 @@ SaAisErrorT SmfImmCcbAction::execute(SaImmOiHandleT
> i_oiHandle,
> immRollbackCcbDn += ",";
> immRollbackCcbDn += *i_rollbackDn;
>
> - if ((result = smfCreateRollbackElement(immRollbackCcbDn, i_oiHandle)) !=
> - SA_AIS_OK) {
> + result = smfCreateRollbackElement(immRollbackCcbDn, i_oiHandle);
> + if ((result != SA_AIS_OK) && (result != SA_AIS_ERR_EXIST)) {
> LOG_ER(
> "SmfImmCcbAction::execute failed to create rollback element %s,
> rc=%s",
> immRollbackCcbDn.c_str(), saf_error(result)); @@ -490,11
> +492,21 @@ SaAisErrorT SmfImmCcbAction::execute(SaImmOiHandleT i_oiHandle,
> }
>
> if (m_operations.size() > 0) {
> - SmfImmUtils immUtil;
> - if ((result = immUtil.doImmOperations(m_operations, rollbackCcb)) !=
> - SA_AIS_OK) {
> - delete rollbackCcb;
> - rollbackCcb = NULL;
> +
> + doImmOpTimer.set_timeout_time(60000);
> + while (doImmOpTimer.is_timeout() == false) {
> + SmfImmUtils immUtil;
> + result = immUtil.doImmOperations(m_operations, rollbackCcb);
> + if (result == SA_AIS_ERR_TRY_AGAIN) {
> + base::Sleep(base::kFiveHundredMilliseconds);
> + continue;
> + } else if (result != SA_AIS_OK) {
> + LOG_WA("%s: SmfImmCcbAction:execute Fail '%s'",
> + __FUNCTION__, saf_error(result));
> + delete rollbackCcb;
> + rollbackCcb = NULL;
> + }
> + break;
> }
> }
>
/* -*- OpenSAF -*-
*
* (C) Copyright 2008 The OpenSAF Foundation
* Copyright Ericsson AB 2017 - All Rights Reserved.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
* or FITNESS FOR A PARTICULAR PURPOSE. This file and program are licensed
* under the GNU Lesser General Public License Version 2.1, February 1999.
* The complete license can be accessed from the following location:
* http://opensource.org/licenses/lgpl-license.php
* See the Copying file included with the OpenSAF distribution for full
* licensing terms.
*
* Author(s): Ericsson AB
*
*/
#include <string>
#include <vector>
#include <atomic>
#include <memory>
#include "ais/include/saImm.h"
#include "ais/include/saAis.h"
#include "smf/smfd/imm_om_api/common/common.h"
#include "smf/smfd/imm_om_api/om_handle.h"
#include "smf/smfd/imm_om_api/om_admin_owner_handle.h"
#include "smf/smfd/imm_om_api/om_ccb_handle.h"
#include "smf/smfd/imm_om_api/om_ccb_object_create.h"
#ifndef IMMCCB_H
#define IMMCCB_H
/* Handle modifications in the IMM model
*
* USER-HANDLING
* To request a modification or a set of modifications the following has to be
* done:
* 1. Describe a modification. This is done by filling in a modification object
* Possible modifications are:
* - Create an object
* - Delete an object
* - Modify attribute(s) of an existing object
* 2. Add the modification to the modification list
* Only modifications that can be done in the same CCB can be added
* 3. Repeat 1 and 2 for all wanted modifications
* 4. Request modifications to be done. The modification list is given as input
* This is the step where all IMM handling is done and where the result is
* reported (Success or Fail)
*
* TBD(Lennart). Is the possibility to configure any settings needed?
*
* MODIFICATIONs are done in three steps:
* 1. Initialize the IMM OM API and become admin owner
* 2. Create an IMM CCB containing all modifications to be done.
* In this step admin ownership is set for objects to be modified
* All information needed to create the CCB is stored to make it possible to
* try again if the applying the CCB fails but a retry is valid to do.
* 3. Apply the CCB and clear admin ownership
*
* RETRY handling is done in three levels:
* 1. For each IMM API. This is mainly try again loops
* 2. For sequences of APIs. The main reason is if the OM handle has to be
* recreated (SA_AIS_ERR_BAD_HANDLE)
* 3. For failing to apply a CCB if the reason is not un-recoverable e.g.
* a validation error
*
* ERROR handling:
* The execute API returns a boolean where false means that the modification
* failed. All non recoverable internal fails are logged to syslog. The user
* should log a fail in order to track where the modification was requested but
* does not have to log any detailed information.
* Example of user logging:
* LOG_NO("%s: ExecuteImmModification Fail", __FUNCTION__);
* More detailed information about a fail is saved and can be requested.
*
* Possible reasons for a fail:
* * An IMM API has returned an unrecoverable error. This includes try again
* timeout
* * Validation error
* * Other unrecoverable error from an Object Implementer
* * All CCB apply retries has failed
* * A general timeout for the whole modify sequence has happened before
* the CCB was applied
*/
namespace modelmodify {
// AttributeDescriptor: Describes one attribute
// Note: Several attributes can be added to one modification
// An attribute always has a name and a type.
// An attribute value can be given as follows:
// * One value of the given value_type
// * No value at all. This means that the attribute will be given an "empty"
// value
// * Multiple values of the given value_type. This is allowed only if the
// attribute is defined as a multi value attribute
// Used with object creation and value modifications
//
class AttributeDescriptor {
public:
explicit AttributeDescriptor(const std::string name_of_attribute) {
attribute_name = name_of_attribute;
}
~AttributeDescriptor() {}
// Add an attribute value.
// Note: In case of multi value all values must be of the same type
// Return false if fail e.g. an illegal type is used.
// The problem is logged to syslog
// No value is added
template<typename T>
bool AddValue(T value);
// A vector for each possible type is defined. However only one will
// be used for each attribute (can only have a value of one type)
// Note: See the vector types for possible types
std::vector<SaInt32T> SaInt32T_values;
std::vector<SaUint32T> SaUint32T_values;
std::vector<SaInt64T> SaInt64T_values;
std::vector<SaUint64T> SaUint64T_values;
std::vector<SaTimeT> SaTimeT_values;
std::vector<SaNameT> SaNameT_values;
std::vector<SaFloatT> SaFloatT_values;
std::vector<SaDoubleT> SaDoubleT_values;
std::vector<SaStringT> SaStringT_values;
std::vector<SaAnyT> SaAnyT_values;
std::string attribute_name;
SaImmValueTypeT value_type;
//USE_DEFAULT_COPY_AND_MOVE_OPERATORS(AttributeDescriptor);
};
// AttributeModifyDescriptor: Describes one attribute to modify
// The value(s) of an attribute can be modified in the following ways:
// * ADD: Add one or several value(s) to an attribute. Can be used if an
// attribute is "empty" (has no value) or is a multi value
// attribute.
// - If it is a multi value attribute and the value already exists
// nothing will be added
// - If it is a single value attribute that is not "empty" nothing
// will be added or changed
// * DELETE: Remove one or several value(s) from an attribute. If all values
// are removed the attribute is considered to be "empty"
// * REPLACE: Replace all values with a new set of values. If the new set is
// "empty" the attribute will be "empty".
// - If the attribute is multi value all existing values will be
// removed and replaced with the new set
// - If the attribute is not multi value the new set shall consist
// of one value
//
struct AttributeModifyDescriptor {
// Type of modification
SaImmAttrModificationTypeT modification_type;
// The attribute to be modified and the new value(s)
AttributeDescriptor attribute;
};
// ========================
// Modification descriptors
//
// Structures to fill in to define one modification.
// Possible modifications are:
// * Create an IMM object
// * Delete an IMM object
// * Modify the value(s) of one attribute in an IMM object
// Note: Several modifications can be added to one CCB.
//
// Some general notes:
// * Object naming shall follow the IMM AIS rule:
// Full DN: "Object_RDN,Parent_Object_DN"
// RDN part: "RDN_attribute_name=RDN_attribute_value"
// * Some IMM types are used in the structures. They can be found in saImm.h
// =====================
// CreateDescriptor: Create one object
// Note: The name of the object to be created must always be given.
// This is done using an AttributeDescriptor:
// * The attribute name is the rdn attribute defined in the IMM class
// * The value type is SA_IMM_ATTR_SASTRINGT
// * The value is the RDN part of the object name
//
struct CreateDescriptor {
// The name of the class that defines the object to be created
std::string class_name;
// The full DN of the object that is parent to the object to be created
// If empty (no parent) a root object will be created
std::string parent_name;
// Attributes to be given values at object creation
std::vector<AttributeDescriptor> attributes;
void AddAttribute(AttributeDescriptor one_attribute) {
attributes.push_back(one_attribute);
}
};
// DeleteDescriptor: Delete one object
//
struct DeleteDescriptor {
// Full DN of the object to be deleted
std::string object_name;
};
// ModifyDescriptor: Modify value(s) of one or several attributes in one object
//
struct ModifyDescriptor {
// Full DN of the object to be modified
std::string object_name;
std::vector<AttributeModifyDescriptor> modifications;
void AddModification(AttributeModifyDescriptor one_modification) {
modifications.push_back(one_modification);
}
};
// ==============
// CCB descriptor
// CcbDescriptor: Holds information about a complete CCB to create and apply
// Here all modifications shall be added. Is used as in-parameter for the
// modification request
//
struct CcbDescriptor {
std::vector<CreateDescriptor> create_descriptors;
std::vector<DeleteDescriptor> delete_descriptors;
std::vector<ModifyDescriptor> modify_descriptors;
// Use these methods to add the modifications
void AddCreate(CreateDescriptor one_create_descriptor) {
create_descriptors.push_back(one_create_descriptor);
}
void AddDelete(DeleteDescriptor one_delete_descriptor) {
delete_descriptors.push_back(one_delete_descriptor);
}
void AddModify(ModifyDescriptor one_modify_descriptor) {
modify_descriptors.push_back(one_modify_descriptor);
}
};
// ErrorInfo: Used as out-parameter to give information about an operation
struct ErrorInfo {
// Name of the failed function / method
std::string function_name;
// Holds the ais_error code of the last called IMM API
SaAisErrorT ais_error;
};
// Recovery information
// An operation may Fail and no recovery is possible but i may also fail
// but some recovery action is possible e.g. bad handle meaning that recovery
// is possible by restarting the API sequence.
const int kNotSet = 0;
// No recovery needed, continue with next step
const int kContinue = 1;
// Restart sequence from Create OM Handle
const int kRestartOm = 2;
// Fail, no recovery possible
const int kFail = 3;
// Creates and applies a CCB with all described modifications
// Handles internally recovery actions based on AIS return values as described
// in AIS, IMM PR document and IMM README files
//
// Note1: The DoModification method is synchronous and may take a long time.
// It may take up to one minute to get an OM handle. This is also the
// case if the handle becomes invalid during the modify operation and
// has to be re-created however, this is not normally the case
// Note2: All IMM resources needed for the operation will be allocated
// internally when needed and released before DoModification returns.
//
// TBD(Lennart) We may also need:
// * Method to set some timeout for the operation (other settings?)
//
// Example: Create an IMM object Obj1 based on IMM class Class1. The rdn
// attribute name is obj1Name and the object name (rdn) shall be
// obj1=1. Parent is safApp=safSmfService. No other attributes will be
// set at creation time
//
// // 1. Fill in an attribute descriptor for the object name:
// modelmodify::AttributeDescriptor object_name("obj1Name");
// object_name.AddValue<SaStringT>("obj1=1");
//
// // 2. Fill in a create descriptor:
// modelmodify::CreateDescriptor create_obj1;
// create_obj1.class_name = "Class1";
// create_obj1.parent_name = "safApp=safSmfService";
// create_obj1.AddAttribute(object_name);
//
// // 3. Fill in a CCB descriptor:
// modelmodify::CcbDescriptor obj1_ccb;
// obj1_ccb.AddCreate(create_obj1);
//
// // 4. Create CCB according to the CCB descriptor and apply it:
// modelmodify::ObjectModification obj1_creator;
// if (obj1_creator.DoModification(obj1_ccb) == false) {
// LOG_NO("%s: DoModification() Fail", __FUNCTION__);
// // Do some other error handling...
// }
//
class ObjectModification {
public:
ObjectModification();
~ObjectModification(); // TBD(Lennart) Finalize the Object Manager
// Returns False if an unrecoverable problem occurs. This may for example be
// a validation error
bool DoModification(CcbDescriptor modifications);
// Get more information if DoModification return Fail
ErrorInfo GetErrorInfo(void) { return error_info_; }
private:
void FinalizeHandles(void);
int CreateHandles(void);
int CreateObjectManager(void);
int CreateAdminOwner(void); // Create a handle
int CreateCcb(void);
int AdminOwnerSet(std::vector<std::string>& objects, SaImmScopeT scope);
int AdminOwnerRelease(void);
// Handle om_ccb_object_create()
int AddCreates(std::vector<CreateDescriptor>& create_descriptors);
int AddCreate(CreateDescriptor& create_descriptor);
template<typename T>
void AddValues(std::string& name, std::vector<T>& values,
immom::ImmOmCcbObjectCreate& creator);
int ApplyModifications(void);
const SaVersionT imm_version{'A', 2, 11};
// Handle objects
std::unique_ptr<immom::ImmOmHandle> imm_om_handle_;
std::unique_ptr<immom::ImmOmCcbHandle> imm_ccb_handle_;
std::unique_ptr<immom::ImmOmAdminOwnerHandle> imm_ao_handle_;
std::unique_ptr<immom::ImmOmAdminOwnerSet> imm_ao_owner_handler_;
// Instance number for this class used to create a unique applier name
// This is needed if instances are used in several threads
// By using a unique name we can avoid taking admin ownership form some other
// thread that has an ongoing modification
// At construction time the static instance number is copied to the local
// instance number and the global instance number is incremented for next
// instance to use.
static std::atomic<unsigned int> next_instance_number_;
unsigned int instance_number_{0};
std::string admin_owner_name_;
CcbDescriptor ccb_descriptor_;
// A list of all objects to become admin owner of
std::vector<std::string> admin_owner_objects_;
// Store information about each step in the IMM API sequence
ErrorInfo error_info_;
// Main timeout for DoModification()
uint64_t modification_timeout_;
DELETE_COPY_AND_MOVE_OPERATORS(ObjectModification);
};
} // namespace modelmodify
# endif // IMMCCB_H
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel
