Re: [onap-discuss] [OOM] - restart pod behavoir with SDNC data

[FREEMAN, BRIAN D]

I think we need to mount /opt/opendaylight/current and perhaps /opt/sdnc (or 
whatever).

/opt/opendaylight becuase unless we preserve the file system then SDNC will 
have to re-install the karaf feature bundles for SDNC/APPC on every restart and 
load the directed graphs etc before replication of data will work (the data 
models for the northbound API aren’t in ODL till the karaf features are loaded).

This will take minutes as I suspect you have noticed in OOM today vs in plain 
docker its seconds to restart.

Brian


From: Alexis de Talhouët [mailto:adetalhoue...@gmail.com]
Sent: Tuesday, January 16, 2018 8:38 AM
To: FREEMAN, BRIAN D <bf1...@att.com>
Cc: onap-discuss@lists.onap.org
Subject: Re: [onap-discuss] [OOM] - restart pod behavoir with SDNC data

Hi Brian,

Resiliency is part of this epic: 
OOM-346<https://urldefense.proofpoint.com/v2/url?u=https-3A__jira.onap.org_browse_OOM-2D346=DwMFaQ=LFYZ-o9_HUMeMTSQicvjIg=e3d1ehx3DI5AoMgDmi2Fzw=fWIvnACN-K4rnPKMvfgro5YtsbmlibTNYBjJV7rlACM=mtpL5G16KqSSvWZTylKsS_Ttiuakga7UHBVmXBiE61w=>
 and it has not being done yet.

I tend to think ODL data is not being persisted in the 
SDNC-163<https://urldefense.proofpoint.com/v2/url?u=https-3A__jira.onap.org_browse_SDNC-2D163=DwMFaQ=LFYZ-o9_HUMeMTSQicvjIg=e3d1ehx3DI5AoMgDmi2Fzw=fWIvnACN-K4rnPKMvfgro5YtsbmlibTNYBjJV7rlACM=hBqlT-p_FroPEePsSLURIi34cLsVQFrcV5VggMT95jU=>
 so we will have to address this, whether it’s for master or amsterdam.
Also, I’m not sure we should persist to whole ODL deployment 
(/opt/opendaylight/current), but instead have a mechanism that creates 
regularly snapshots and backup those upon restart (like 
cluster-admin:backup-datastore<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.opendaylight.org_en_stable-2Dnitrogen_getting-2Dstarted-2Dguide_common-2Dfeatures_persistence-5Fand-5Fbackup.html=DwMFaQ=LFYZ-o9_HUMeMTSQicvjIg=e3d1ehx3DI5AoMgDmi2Fzw=fWIvnACN-K4rnPKMvfgro5YtsbmlibTNYBjJV7rlACM=cAEI1eJZDtXOIG_jPjUrwPrVniHIJapzU4AlTeHiW5I=>
 or 
daexim<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.opendaylight.org_en_latest_user-2Dguide_daexim-2Duser-2Dguide.html=DwMFaQ=LFYZ-o9_HUMeMTSQicvjIg=e3d1ehx3DI5AoMgDmi2Fzw=fWIvnACN-K4rnPKMvfgro5YtsbmlibTNYBjJV7rlACM=7UK6F4nUoUL_l3vPCV7ZpjxO4GWe_SWvBcLjvpxTGds=>,
 that I’m sure you already know)

Alexis


On Jan 15, 2018, at 5:46 PM, FREEMAN, BRIAN D 
<bf1...@att.com<mailto:bf1...@att.com>> wrote:

Michael, Alexis,

I did a test where I created an OOM environment, sent preload into SDNC and 
then deleted the SDNC Pod.
K8 restarted the POD but the data saved to SDNC wasnt there.
This was Amsterdam.

Do we need to have the /opt/opendaylight/current/ directory persisted (or 
current/journal) ?

I think the clustering team may have addressed this as well but cold recovery 
of an SDNC cluster created by OOM requires something stored on the hard disk.

Brian

___
onap-discuss mailing list
onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>
https://lists.onap.org/mailman/listinfo/onap-discuss<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_mailman_listinfo_onap-2Ddiscuss=DwMFaQ=LFYZ-o9_HUMeMTSQicvjIg=e3d1ehx3DI5AoMgDmi2Fzw=fWIvnACN-K4rnPKMvfgro5YtsbmlibTNYBjJV7rlACM=ymPhYH3b58n2_lR5d93z9dSexOQWWeDOU3LVnvAhZ2o=>

___
onap-discuss mailing list
onap-discuss@lists.onap.org
https://lists.onap.org/mailman/listinfo/onap-discuss

Re: [onap-discuss] [OOM] - restart pod behavoir with SDNC data

[Alexis de Talhouët]

Hi Brian,

Resiliency is part of this epic: OOM-346  
and it has not being done yet.

I tend to think ODL data is not being persisted in the SDNC-163 
 so we will have to address this, 
whether it’s for master or amsterdam.
Also, I’m not sure we should persist to whole ODL deployment 
(/opt/opendaylight/current), but instead have a mechanism that creates 
regularly snapshots and backup those upon restart (like 
cluster-admin:backup-datastore 

 or daexim 
, 
that I’m sure you already know)

Alexis

> On Jan 15, 2018, at 5:46 PM, FREEMAN, BRIAN D  wrote:
> 
> Michael, Alexis,
>  
> I did a test where I created an OOM environment, sent preload into SDNC and 
> then deleted the SDNC Pod.
> K8 restarted the POD but the data saved to SDNC wasnt there.
> This was Amsterdam.
>  
> Do we need to have the /opt/opendaylight/current/ directory persisted (or 
> current/journal) ?
>  
> I think the clustering team may have addressed this as well but cold recovery 
> of an SDNC cluster created by OOM requires something stored on the hard disk.
>  
> Brian
>  
> ___
> onap-discuss mailing list
> onap-discuss@lists.onap.org 
> https://lists.onap.org/mailman/listinfo/onap-discuss 
> 
___
onap-discuss mailing list
onap-discuss@lists.onap.org
https://lists.onap.org/mailman/listinfo/onap-discuss

[onap-discuss] [OOM] - restart pod behavoir with SDNC data

[FREEMAN, BRIAN D]

Michael, Alexis,

I did a test where I created an OOM environment, sent preload into SDNC and 
then deleted the SDNC Pod.
K8 restarted the POD but the data saved to SDNC wasnt there.
This was Amsterdam.

Do we need to have the /opt/opendaylight/current/ directory persisted (or 
current/journal) ?

I think the clustering team may have addressed this as well but cold recovery 
of an SDNC cluster created by OOM requires something stored on the hard disk.

Brian

___
onap-discuss mailing list
onap-discuss@lists.onap.org
https://lists.onap.org/mailman/listinfo/onap-discuss