IF you have been doing a lot of testing the robot file system can get pretty large (500Meg per test in some cases)
Cleanup /share/logs inside the container (that is log files stored on the nfs) and /tmp/csar where the distributed models are stored for use in testing like vCPE Brian From: [email protected] <[email protected]> On Behalf Of Rajendra Prasad Mishra Sent: Monday, March 23, 2020 9:25 AM To: STARK, STEVEN <[email protected]> Cc: [email protected] Subject: Re: [onap-discuss] [OVP][SO] Heatbridge failure while doing ovp validation on reference VNF Thanks Steve, that worked. I no longer see the openstack authentication issue. However, I am seeing the robot storage issues. When I run the validation test, robot pod is killed because it is consuming more "ephemeral-storage" then allocated. Do you know how to increase the " ephemeral-storage" for robot pod ? This is a standard El Alto installation with no change in default configuration. Why does validation tests take up so much of space (127421328Ki) ----------------- aarna@anod-master:~/oom/kubernetes/robot$ kubectl get pod -n onap | grep robot dev-robot-robot-6d444f4cdd-94kkk 1/1 Running 0 3m49s dev-robot-robot-6d444f4cdd-vspc9 0/1 Evicted 0 3d15h aarna@anod-master:~/oom/kubernetes/robot$ kubectl describe pod -n onap dev-robot-robot-6d444f4cdd-vspc9 Name: dev-robot-robot-6d444f4cdd-vspc9 Namespace: onap Priority: 0 Node: dublin06/ Start Time: Thu, 19 Mar 2020 15:24:57 +0000 Labels: app=robot pod-template-hash=6d444f4cdd release=dev-robot Annotations: cni.projectcalico.org/podIP<https://urldefense.proofpoint.com/v2/url?u=http-3A__cni.projectcalico.org_podIP&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=67kLb4PSwSX9Fj04dF9MkMMJ0A2LrX9Ze0p6tuDLxo4&s=MLwHGSQ5pPlDSCtDzE_p7lM5vcEUhSDD8zDoC3S4vn4&e=>: 10.42.1.56/32<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.42.1.56_32&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=67kLb4PSwSX9Fj04dF9MkMMJ0A2LrX9Ze0p6tuDLxo4&s=Pbnhw9Cxfa2A6klTNVsFKrsskeFRF0guKda-qrkjVC4&e=> Status: Failed Reason: Evicted Message: The node was low on resource: ephemeral-storage. Container robot was using 127421328Ki, which exceeds its request of 0. IP: Controlled By: ReplicaSet/dev-robot-robot-6d444f4cdd Containers: robot: Image: nexus3.onap.org:10001/onap/testsuite:1.5.4<https://urldefense.proofpoint.com/v2/url?u=http-3A__nexus3.onap.org-3A10001_onap_testsuite-3A1.5.4&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=67kLb4PSwSX9Fj04dF9MkMMJ0A2LrX9Ze0p6tuDLxo4&s=zGRkR7bU0ERGCcc4ahXTnuiECXcRN__EkBCF_EOPrsM&e=> Port: 88/TCP Host Port: 0/TCP Liveness: tcp-socket :88 delay=180s timeout=1s period=10s #success=1 #failure=3 Readiness: tcp-socket :88 delay=180s timeout=1s period=10s #success=1 #failure=3 Environment: <none> Mounts: /dev/shm from dshm (rw) /etc/localtime from localtime (ro) /share/config from robot-eteshare (rw) /share/logs from robot-logs (rw) /var/run/secrets/kubernetes.io/serviceaccount<https://urldefense.proofpoint.com/v2/url?u=http-3A__kubernetes.io_serviceaccount&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=67kLb4PSwSX9Fj04dF9MkMMJ0A2LrX9Ze0p6tuDLxo4&s=FAoROrnwGV5yFeLt7ZfIM_LtdbNGb7eMT0YJCiD7XLw&e=> from default-token-2mz7g (ro) Volumes: robot-logs: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: dev-robot-robot ReadOnly: false dshm: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: Memory SizeLimit: <unset> localtime: Type: HostPath (bare host directory volume) Path: /etc/localtime HostPathType: robot-eteshare: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-robot-robot-eteshare-configmap Optional: false default-token-2mz7g: Type: Secret (a volume populated by a Secret) SecretName: default-token-2mz7g Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute<https://urldefense.proofpoint.com/v2/url?u=http-3A__node.kubernetes.io_not-2Dready-3ANoExecute&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=67kLb4PSwSX9Fj04dF9MkMMJ0A2LrX9Ze0p6tuDLxo4&s=0b-LBWz7xZSxyHD54a1S970SRnyV92DCFb8OC5hGa6g&e=> for 300s node.kubernetes.io/unreachable:NoExecute<https://urldefense.proofpoint.com/v2/url?u=http-3A__node.kubernetes.io_unreachable-3ANoExecute&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=67kLb4PSwSX9Fj04dF9MkMMJ0A2LrX9Ze0p6tuDLxo4&s=lUcFt1b-6535RBbsO7aF5SFumNsJvLXBjdo1eIXnxgA&e=> for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Evicted 4m26s kubelet, dublin06 The node was low on resource: ephemeral-storage. Container robot was using 127421328Ki, which exceeds its request of 0. Normal Killing 4m24s kubelet, dublin06 Killing container with id docker://robot:Need to kill Pod ---------------------------------- thanks, -RPMishra On Fri, Mar 20, 2020 at 9:42 PM Steve Stark <[email protected]<mailto:[email protected]>> wrote: Great that helps. It looks like instantiation worked, the part that's failing is some custom heat stack validation after the instantiation. I was able to replicate the failure you're getting. Can you change the admin_tenant value for your identity service to the tenant name instead of the tenant ID and try again? I think in your case the tenant name = admin. This is in the mariadb pod - MariaDB [catalogdb]> update identity_services set admin_tenant="<tenant name>" where id="<identity service id>"; Thanks Steven -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20336): https://lists.onap.org/g/onap-discuss/message/20336 Mute This Topic: https://lists.onap.org/mt/72026138/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
