Ok, will do that. And point to the new file from the basic README. Next up will be the README entry for NO_DANGLING (#50). I may do a separate file for that also, depends on the ammount of text It turns out to be.
/AndersBj -----Original Message----- From: Hans Feldt Sent: den 20 november 2013 15:47 To: Anders Björnerstedt; reddy.neelaka...@oracle.com Cc: opensaf-devel@lists.sourceforge.net Subject: RE: [devel] [PATCH 1 of 1] IMM: Update the README file to document 2PBE [622] Hi, Just a thought, I think the README is getting too big and that it could be split it into several files. For example add the new 2PBE stuff into a new file README.2PBE Thanks, Hans > -----Original Message----- > From: Anders Bjornerstedt [mailto:anders.bjornerst...@ericsson.com] > Sent: den 20 november 2013 15:22 > To: reddy.neelaka...@oracle.com > Cc: opensaf-devel@lists.sourceforge.net > Subject: [devel] [PATCH 1 of 1] IMM: Update the README file to > document 2PBE [622] > > osaf/services/saf/immsv/README | 167 > +++++++++++++++++++++++++++++++++++++++++ > 1 files changed, 167 insertions(+), 0 deletions(-) > > > diff --git a/osaf/services/saf/immsv/README > b/osaf/services/saf/immsv/README > --- a/osaf/services/saf/immsv/README > +++ b/osaf/services/saf/immsv/README > @@ -1604,6 +1604,173 @@ Bit 2 controls OpenSAF4.1 protocols allo Bit > 3 controls OpenSAF4.3 protocols allowed or not. > > > +2PBE Allow IMM PBE to be configured without shared file system (4.4) > +=================================================================== > +https://sourceforge.net/p/opensaf/tickets/21/ > + > +The 2PBE enhancement allows the IMM to have PBE configred so that it > +does not rely on a shared filesystem, such as DRBD. > + > +Executing IMM without PBE configured or enabled (0PBE) of course also > +makes the IMM not rely on any shared filesystem, but you then do not > +get automatic incremental persistence. Deployments that rarely update > +the configuration data and rarely alter the "administrative state" on > +AMF data, should still consider the option of running without PBE. > +Persistence is then acheived by performing an explicit dump after > +each CCB, after each admin-op to change administrative state. or at > +least after a set of such persistent changes have been completed. > +This is the simplest configuration with least overhead and lowest resource > consumption. > + > +Regular 1PBE uses one Persistent Back End process started and > +controlled by the IMMND coordinator. > + > +With 2PBE a PBE process is started at *both* SCs. In 2PBE, the PBE > +started by the coordinator is called the *primaey* PBE and operates > +in much the same way as the regular 1PBE process, except it also > +synchronizes with the PBE at the other SC. The PBE started by the > +non-coord but SC resident IMMND is called the *slave* PBE. > + > +The primary PBE and the slave PBE each write to what should normally > +be a local sqlite file. The sqlite file has the same basic name as > +used by regular 1PBE (IMMSV_PBE_FILE in immnd.conf), except that a > +suffix is appended that consists of the processors node-id, as > +defined by MDS. This suffix *allows* 2PBE to be execued with the PBE files > actually residing on a shared file system. > +That would not be a good solution for deployment, but it may simplify > +some test framweworks, that can then use the same file system > +configuration for testing both 1PBE and 2PBE. > + > +Configuring 2PBE > +---------------- > +To configure for 2PBE comment in the following parameter in immd.conf > +(note immd.conf, not immnd.conf): > + > + #export IMMSV_2PBE_PEER_SC_MAX_WAIT=30 > + > +This has to be defined on both SCs so that both the active and the > +standby IMMD become aware that 2PBE should be run. The value of this > +parameter is the number of seconds the active IMMD must wait for both > +SC IMMNDs to complete "preloading" (explaine below) before the active > +IMMD can choose one of these IMMNDs to become IMMND coordinator. > + > +Note that it is not normal, i.e. not expected, for any given cluster > +to switch between 1PBE and 2PBE. The decision to use 1PBE or 2PBE > +should be based on what is available and intended to be used as > +storage for the imm service. Typically that does not change during > +the service lifetime for a cluster. Thus the choice of 1PBE or 2PBE (or > 0PBE) is normally done at cluster installation time. > + > +THe cardinal example for a deployment that should use 2PBE (or 0PBE) > +is a down sized embedded system, that does not have any shared filesystem > available. > + > +A 2PBE system can disable PBE and enable PBE in the same way that a > +1PBE system can. This is done in the same way, using the > +administrative operation for this, see the regular section on PBE above. > + > +Cluster-start & IMM loading from PBE-files > +------------------------------------------ > +The active IMMD will order each SC resident IMMND to execute a > +"preload" probing the SC local filesystem for the file state that > +*would* be loaded to the cluster if that SC IMMND was chosen as > +coord. The two SC IMMNDs send the preload stats to the active IMMD. > + > +The active IMMD will wait for the IMMNDs at *both* SCs to complete > +the preload task and then determine which SC has the apparently > +latest file state. The IMMND at that SC will then be chosen as IMMND > +coord. Should the timeout be reached, then the active IMMD declares > +the only avaialble SC IMMND as coordinator. This should be avoided. > + > +Actual loading then proceeds in the same way as for regular 1PBE. The > +IMMSV_2PBE_PEER_SC_MAX_WAIT is by default 30 seconds. This value > +should be high enough to make it extremely unlikely that the active > +IMMD is forced choose coord/loader when only a single SC IMMND has > +joined. If that happens, then the risk is that the cluster restart > +will be done *not* using the latest persistent imm state, effectively > +rewinding the imm state. Normally the two PBE files should be > +identical and the choice of coord/loader then does not mater. But if > +hey are not identical, due to one SC having been down for some time > +before the cluster restart, then the choice of the SC to load from > +does matter. [Note: the same type of problem will happen with regular > +1PBE based on a shared filesystem (DRBD) if one SC fails to come up > +in time to join the > +(DRBD) sync protocol. The corresponding DRBD timeout is on the order > +of 20 seconds. Even if the other SC later joins, it will be too late > +because by that time the loading has probably been completed. Even if > +loading is still in progress, DRBD can not correct/mutate the PBE > +file while it is being read by the sqlite-lib for loading.] > + > +Normal processing with 2PBE > +--------------------------- > +When loading has completed, two PBEs will be started. The primary PBE > +at the SC with the IMMND coord and the slave PBE at the other SC. > + > +In the same way as for 1PBE, the primary PBE, is the transaction > +coordinator for CCB commits, PRT operations and class-create/deletes. > +Specific for 2PBE, the slave PBE is in essence a class applier for > +all configuration classes, recording the same data as the primary PBE, but > on a file at the other SC. > + > +The primary PBE is thus the entity that decides the outcomme of a CCB > +that is in the crtitical state. [The critical state is defined by the > +commit request having been sent to the (primary) PBE]. IF the pirmary > +PBE acks the commit, the CCB commits in imm-ram. Finally, all > +appliers that where tracking the CCB get the commit/apply callback, > including the slave PBE. > + > +With 2PBE, *both* PBEs must be available for the imm to be > persistent-writable. > +If one or both PBEs are unavailable (or unresponsive) then persistent > +writes (CCBs, PRT operations, class changes) will fail. > + > +In 2PBE, a restarted PBE (primary or slave) will more often need to > +regenerate its sqlite file (from imm-ram). On the other hand, > +regeneration of the sqlite file should be faster in 2PBE than in > +regular 1PBE because the file is typically placed on a local file system. > + > +OneSafe2PBE > +----------- > +If an SC is taken down by order from the operator, i.e. a controlled > +shutdown, then the operator can also (directly or indirectly) request > +that persistent writes be allowed despite only one PBE being availble > +in a 2PBE system. This is also typically needed for an uncontrolled > +and unexpected departure of an SC if that SC does not immediately > +bounce back up. A repair is then apparently needed and the system > +*must* be allowed to function with only one PBE, despite that it only writes > to one local filesystem using one PBE. > + > +The 1safe2PBE mechanism allows a 2PBE OpenSAF cluster to open up for > +persistent writes using only one of the two PBEs - temporarily. This > +is only intended to be used as an temporary state when one SC is long > +term unavailable. As soon as the other SC returns, then the IMM will > +automatically re-enter normal 2-safe2PBE and will reject persistent > +writes and attempts to enter 1safe2PBE until the slave PBE has synced > (regenerated its sqlite file) and rejoined the cluster. > + > +The 1safe2PBE state is entered by the administrative opeation: > + > + immadm -o 1 -a safImmService -p opensafImmNostdFlags:SA_UINT32_T:8 \ > + opensafImm=opensafImm,safApp=safImmService > + > +It is exited either automatically by a rejoined SC or by an explicit > +administrative > +opertion: > + > + immadm -o 2 -a safImmService -p opensafImmNostdFlags:SA_UINT32_T:8 \ > + opensafImm=opensafImm,safApp=safImmService > + > +Note the explicit setting of admin-owner-name using "-a > +safImmService". This is has to be used for these admin-operations > +because the imm service needs admin-ownership over the object > +"opensafImm=opensafImm,safApp=safImmService" in order for 2PBE to work > properly. > + > +Hence the fourth bit in the opensafImmNostdFlags bitvector of the > +OpenSAF service object, is used to toggle on/off oneSafe2PBE. > +Toggling this bit, on older systems, or systems that do not have 2PBE > +configured, will have no effect. Toggling this bit on (on a 2PBE > +system) is only accepted by the IMM service when there is only one SC > available. > + > +We reccommend that any deployment of OpenSAF that intends to allow > +usage of 2PBE, invoke the toggling on of this bit in the wrapper > +function for performing a planned stop of an SC. Note however that > +the operation will only succeed when the SC has gone down, i.e. thre > +is only one SC available. Similarly, if there is any alarm generated > +when an SC has gone down and not come back up quickly enough (a node > +repair needed alarm), then we suggest that the alarm trigger the invocation > of the admin op to toggle this flag on. > + > +For "normal", but unplanned, processor restarts, we recommend that > +this flag not be toggled on. This means that for such processor > +restarts, persistent writes will not be allowed untill both SCs are > available again. > + > ---------------------------------------- > DEPENDENCIES > ============ > > ---------------------------------------------------------------------- > -------- Shape the Mobile Experience: Free Subscription Software > experts and developers: Be at the forefront of tech innovation. > Intel(R) Software Adrenaline delivers strategic insight and > game-changing conversations that shape the rapidly evolving mobile landscape. > Sign up now. > http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.c > lktrk _______________________________________________ > Opensaf-devel mailing list > Opensaf-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/opensaf-devel ------------------------------------------------------------------------------ Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk _______________________________________________ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
