Hi Tri, We had not performed any upgrade from Elalto to Frankfurt, It's fully a latest openstack stack created by using the integration scripts to deploy the frankfurt release. I am confused in the fresh installation why it's doing upgrade ? As this was a new stack having new nfs volume (i.e dockernfs ) I think the pv's and pvc's had no issues.
Can someone suggest why this was happening ? Note :- We had an other openstack stack running for Elalto cluster, I hope that cluster will not effect the current frankfurt one. Regards, ________________________________________________________ Velugubantla Praveen Engineer - CTO-Common L&T TECHNOLOGY SERVICES LIMITED L3 Building, Manyata Embassy Business Park, Nagawara Hobli, Bengaluru-560045 ________________________________________________________ Mobile: +91 9154111420 www.LTTS.com<http://www.ltts.com/> [cid:8ebdf29f-8a22-46db-bd42-41467ac8411e] ________________________________ From: Tri Nguyen <[email protected]> Sent: Wednesday, June 17, 2020 7:57 PM To: [email protected] <[email protected]>; [email protected] <[email protected]>; Velugubantla Praveen <[email protected]>; RICHOMME Morgan TGI/OLN <[email protected]>; Naveen S. Sankad <[email protected]> Subject: RE: [onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi, >From the log you sent, it seems like you tried to perform the upgrade of >mariadb-galera to Frankfurt, and the upgrade failed in the middle of the >process. When it failed, there are some secrets like secrets "dev6-mariadb-galera-temp-upgrade-root" and secrets "dev6-mariadb-galera-temp-upgrade-usercred" created by the helm pre upgrade hooks that are not deleted. When you run the helm upgrade again, since these secrets are there, the upgrade process can’t be processed further, leaving you in the PENDING UPGRADE status and make the mariadb-galera statefulsets failed. Can you tell me, assuming that you already have a healthy mariadb-galera in El Alto running, what commands did you run to upgrade it to Frankfurt? Thank you, Tri From: [email protected] <[email protected]> Sent: June 17, 2020 8:52 AM To: [email protected]; [email protected]; RICHOMME Morgan TGI/OLN <[email protected]>; Naveen S. Sankad <[email protected]>; Tri Nguyen <[email protected]> Subject: RE:[onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi, this upgrade is a new feature for Frankfurt and should be considered as PoC. Did you upgraded an ONAP or did you installed from scratch? if from scratch, please remove all onap helm + suppress everything in /dockerdata-nfs, remove all remaining PV + reinstall ________________________________ De : [email protected]<mailto:[email protected]> [[email protected]] de la part de Velugubantla Praveen [[email protected]] Envoyé : mercredi 17 juin 2020 12:06 À : RICHOMME Morgan TGI/OLN; Naveen S. Sankad; [email protected]<mailto:[email protected]> Objet : Re: [onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi Morgan, Thanks a lot for analyzing the result logs and for providing us your detailed observation on this issue. Yes as you suggested we will do the re-installation of whole setup. But can you or someone from community please suggest us, whom to contact for these kind of GALERA-DB issues (Helm Chart dev6-mariadb-galera, STATUS: PENDING_UPGRADE, Chart problem) or can someone loop those DB-team folks here. That will be helpful, if the issue still remains same after re-installation. Regards, ________________________________________________________ Velugubantla Praveen Engineer - CTO-Common L&T TECHNOLOGY SERVICES LIMITED L3 Building, Manyata Embassy Business Park, Nagawara Hobli, Bengaluru-560045 ________________________________________________________ Mobile: +91 9154111420 www.LTTS.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ltts.com%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419471395&sdata=XKj%2FUDdNf6nAb4rYhkiyA6YnATwzEzWwhP93GRFHGQA%3D&reserved=0> [cid:[email protected]] ________________________________ From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Sent: Wednesday, June 17, 2020 2:51 PM To: Naveen S. Sankad <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Cc: Velugubantla Praveen <[email protected]<mailto:[email protected]>> Subject: RE:[onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi as you can see in onap-helm.log, the maria-galera chart is not installed properly Chart dev6-mariadb-galera STATUS: PENDING_UPGRADE Chart problem the status should be DEPLOYED as this DB is not installed, the installation of several components can not be finalized you can see that in index.html the first line must be fully green.. in your case, you have - 4 jobs failed - 11 kubernetes deployment failed - 3 statefulsets failed withjout surprise the maria-galera jobs are indicated as failed and is probably the root cause of the other errors. according to the log in pod-dev6-mariadb-galera-pre-upgrade-k6dbh-mariadb-job-pre-upgrade-logs.html you can see Error from server (AlreadyExists): secrets "dev6-mariadb-galera-temp-upgrade-root" already exists Error from server (AlreadyExists): secrets "dev6-mariadb-galera-temp-upgrade-usercred" already exists I suspect you tried to upgrade after the installation failed, but some resources remained leading to a bad resources state (old ones not cleaned + new ones) we do not have the logs of the installation problem but the ones of your upgrade attempt. I do not know this upgrade procedure - I never tested it - so I will not be able to help you more. I would suggest a reinstallation as reinitializing the components that get stuck due to DB init errors could take more time in troubleshooting that re-installing. But maybe installation project has a better view on that. /Morgan ________________________________ De : Naveen S. Sankad [[email protected]] Envoyé : mercredi 17 juin 2020 10:22 À : RICHOMME Morgan TGI/OLN; [email protected]<mailto:[email protected]> Cc : Velugubantla Praveen Objet : Re: [onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi Morgan, Here is the results zip file of the docker run command you provided. Thanks for your support in advance. Regards, Naveen ________________________________ From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Sent: 17 June 2020 13:33 To: Naveen S. Sankad <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Cc: Velugubantla Praveen <[email protected]<mailto:[email protected]>> Subject: RE:[onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release I need all the result artifacts, not only the test execution logs zip what is under /tmp/results, you should find lots of html pages.. I can already see that woth onap-helm that all your helm deployemnt were not successful. if it is only dmaap (timeout on post install scripts) it is OK, if not it means that theire is an infra/kubernetes/helm issue /Morgan ________________________________ De : Naveen S. Sankad [[email protected]] Envoyé : mercredi 17 juin 2020 09:53 À : RICHOMME Morgan TGI/OLN; [email protected]<mailto:[email protected]> Cc : Velugubantla Praveen Objet : Re: [onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi Morgon, Thanks for the response. Now i am able to to execute the command and PFA of output.txt file. Please provide your inputs for the same. Regards, Naveen ________________________________ From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Sent: 17 June 2020 12:18 To: Naveen S. Sankad <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: RE:[onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi I think the error is in your docker command, you probably indicate a dir instead of the full path the <the path to your kube config> must include the file e.g. in my case it looks like docker run -v /home/debian/.kube/config:/root/.kube/config -v /tmp/results:/var/lib/xtesting/results registry.gitlab.com/orange-opensource/lfn/onap/integration/xtesting/infra-healthcheck:latest and I can see the file cat /home/debian/.kube/config apiVersion: v1 clusters: - cluster: ................ Morgan ________________________________ De : Naveen S. Sankad [[email protected]] Envoyé : mercredi 17 juin 2020 06:18 À : RICHOMME Morgan TGI/OLN; [email protected]<mailto:[email protected]> Objet : Re: [onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi Morgon, PFA of log in which i am facing issue to execute the command provided by you. Please provide your inputs for this. IsADirectoryError: [Errno 21] Is a directory: '/root/.kube/config' ---> This error in file Regards, Naveen ________________________________ From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Sent: 16 June 2020 22:08 To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>>; Naveen S. Sankad <[email protected]<mailto:[email protected]>> Subject: RE:[onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi could you run the onap-k8s test to get a full view of the ONAP cluster (no only the pod) from somewhere where you have access to your cluster + docker, please run docker run -v <the path to your kube config>:/root/.kube/config -v /tmp/results:/var/lib/xtesting/results registry.gitlab.com/orange-opensource/lfn/onap/integration/xtesting/infra-healthcheck:latest then zip /tmp/results and send it /Morgan ________________________________ De : [email protected]<mailto:[email protected]> [[email protected]] de la part de Naveen S. Sankad [[email protected]] Envoyé : mardi 16 juin 2020 18:13 À : [email protected]<mailto:[email protected]> Cc : Velugubantla Praveen; JOMY JOSE; Devangam Manjunatha Objet : [onap-discuss] [Frankfurt] [Deploy] [AAF, DCAE, POLICY, SO, SDNC, SDC] Issues while deploying Frankfurt release Hi Team, As a part of Frankfurt Bring Up, we have started deploying on our openstack environment. All the 12 nodes are up and running. We have some issues and queries related to Deployment 1. After helm deploy is success, among 47 healthchecks only 41 are passing and 6 healthchecks (AAF, DCAE, POLICY, SO, SDNC, SDC) are failing. please refer the healthcheck_output.txt for trace. 2. The jobs and pods output trace is attached in jobs_pods_list.txt 3. The respective failed healthcheck pods and container logs are attched in txt files. For this respective components Is there any DB configuration we need to make to bring up the pods. Because in these pod-container logs we observed errors related to DB. Any inputs are highly appreciated. If anyone already succeeded in bring-up of Frankfurt deployment, please provide your inputs for these issues. Thanks and regards, Naveen L&T Technology Services Ltd www.LTTS.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Furlblockederror.aspx%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419471395&sdata=%2Fj8P4GVTcFpG2IxS3oE6w3WzvHI2e6s%2BuVdm6bA%2BtNw%3D&reserved=0> L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ltts.com%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419481390&sdata=zc6LHvCBD9%2FibKPqrNwiSF3SShVA%2Ff4dFf2wodQ818M%3D&reserved=0>. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. L&T Technology Services Ltd www.LTTS.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Furlblockederror.aspx%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419481390&sdata=JtWj1sa5cfpnfWtDmGi0pN9G8rdzZtsRSrP8W8fHQik%3D&reserved=0> L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ltts.com%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419491387&sdata=ACV%2FWbWw%2BlynwUVmlvM6nUojTDZBiZnKOfueK1lF0xo%3D&reserved=0>. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. L&T Technology Services Ltd www.LTTS.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Furlblockederror.aspx%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419491387&sdata=alA1biP7cnWEi%2B%2FzPBtKrLsgiMxFueQS%2FFJhNkoxhEE%3D&reserved=0> L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ltts.com%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419501378&sdata=RR8Cy5TCaVhfTVUK5ZvwER6ScgYT3yC0spSSEcoiEoc%3D&reserved=0>. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. L&T Technology Services Ltd www.LTTS.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Furlblockederror.aspx%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419501378&sdata=5RT2O36uhXA9DURWaKG8RioZR4W46JutFihZluwRDRo%3D&reserved=0> L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ltts.com%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419511372&sdata=ezM1KinGoyKPo%2BE20KZpx04H5frCR4xp4Aiu2VZBEE8%3D&reserved=0>. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. L&T Technology Services Ltd www.LTTS.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Furlblockederror.aspx%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419511372&sdata=rceVG%2FiBpBd5piyoMyP8IE6OExGQ2rJNrrlF%2BYCXy2U%3D&reserved=0> L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com<https://ind01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ltts.com%2F&data=02%7C01%7Cvelugubantla.praveen%40ltts.com%7Cf5a82e3b39584477e7cf08d812ca8ae6%7C311b33788e8a4b5ea33fe80a3d8ba60a%7C0%7C0%7C637280008419511372&sdata=ezM1KinGoyKPo%2BE20KZpx04H5frCR4xp4Aiu2VZBEE8%3D&reserved=0>. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. L&T Technology Services Ltd www.LTTS.com L&T Technology Services Limited (LTTS) is committed to safeguard your data privacy. For more information to view our commitment towards data privacy under GDPR, please visit the privacy policy on our website www.Ltts.com. This Email may contain confidential or privileged information for the intended recipient (s). If you are not the intended recipient, please do not use or disseminate the information, notify the sender and delete it from your system. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21400): https://lists.onap.org/g/onap-discuss/message/21400 Mute This Topic: https://lists.onap.org/mt/74919566/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
