Hi All, I am installing ONAP Honolulu and seeing the following pods are crashing. Most of these pods are Postgress pods. all pods are failing with the same error. Has anybody observed this issue before ? or any insights to fix this problem will greatly help. Thanks.
aarna@anod-master:~$ *kubectl get pods -n onap -o wide |grep -i CrashLoopBackOff* dev-awx-postgres-58c8d8f47f-vhks5 0/1 CrashLoopBackOff 25 108m 10.244.2.154 anod-wk1 <none> <none> dev-cps-postgres-primary-574975d9bc-79pl8 0/1 CrashLoopBackOff 178 15h 10.244.3.41 anod-wk3 <none> <none> dev-dbc-pg-primary-67cffdcb-84thk 0/1 CrashLoopBackOff 186 15h 10.244.2.147 anod-wk1 <none> <none> dev-dcae-dashboard-pg-primary-89b4569fc-x6lwj 0/1 CrashLoopBackOff 191 16h 10.244.5.89 anod-wk5 <none> <none> dev-dcae-db-primary-5bc6c4b896-lwjst 0/1 CrashLoopBackOff 190 16h 10.244.4.66 anod-wk4 <none> <none> dev-dcae-inv-pg-primary-696458fcd-rg484 0/1 CrashLoopBackOff 190 16h 10.244.1.56 anod-wk2 <none> <none> dev-dcaemod-db-primary-5b44f59fcb-tbf9b 0/1 CrashLoopBackOff 227 19h 10.244.2.95 anod-wk1 <none> <none> dev-holmes-engine-mgmt-97b7c6946-xqsn6 0/1 CrashLoopBackOff 187 16h 10.244.2.145 anod-wk1 <none> <none> dev-holmes-pg-primary-786ccc8bb7-mmnhz 0/1 CrashLoopBackOff 190 16h 10.244.4.67 anod-wk4 <none> <none> dev-holmes-rule-mgmt-7f7844459d-ck2jw 0/1 CrashLoopBackOff 191 16h 10.244.3.28 anod-wk3 <none> <none> dev-netbox-app-686556c747-vzhv9 0/1 CrashLoopBackOff 246 14h 10.244.3.59 anod-wk3 <none> <none> dev-netbox-postgres-5b664bbf9f-l67pz 0/1 CrashLoopBackOff 192 16h 10.244.2.146 anod-wk1 <none> <none> dev-uui-server-6fd5fcbf4d-5br4g 0/1 CrashLoopBackOff 194 18h 10.244.4.51 anod-wk4 <none> <none> dev-vnfsdk-postgres-primary-f86c68678-c6kvk 0/1 CrashLoopBackOff 190 16h 10.244.1.57 anod-wk2 <none> <none> *Error:* child process was terminated by signal 7: Bus error initdb: removing contents of data directory "/pgdata/data" ================== aarna@anod-master:~$ kubectl logs -n onap dev-cps-postgres-primary-574975d9bc-79pl8 Thu Sep 2 07:29:11 UTC 2021 INFO: Image mode found: postgres Thu Sep 2 07:29:11 UTC 2021 INFO: Starting in 'postgres' mode Thu Sep 2 07:29:11 UTC 2021 INFO: Setting PGROOT to /usr/pgsql-13. Thu Sep 2 07:29:11 UTC 2021 INFO: PG_CTL_START_TIMEOUT set at: 60 Thu Sep 2 07:29:11 UTC 2021 INFO: PG_CTL_STOP_TIMEOUT set at: 60 Thu Sep 2 07:29:11 UTC 2021 INFO: PG_CTL_PROMOTE_TIMEOUT set at: 60 Thu Sep 2 07:29:11 UTC 2021 INFO: Cleaning up the old postmaster.pid file.. Thu Sep 2 07:29:11 UTC 2021 INFO: User ID is set to uid=26(postgres) gid=26(postgres) groups=26(postgres). Thu Sep 2 07:29:11 UTC 2021 INFO: Working on primary.. Thu Sep 2 07:29:11 UTC 2021 INFO: Initializing the primary database.. Thu Sep 2 07:29:11 UTC 2021 INFO: PGDATA is empty. ID is uid=26(postgres) gid=26(postgres) groups=26(postgres). Creating the PGDATA directory.. Thu Sep 2 07:29:11 UTC 2021 INFO: Starting initdb.. Thu Sep 2 07:29:11 UTC 2021 INFO: XLOGDIR not found. Using default pg_wal directory.. Thu Sep 2 07:29:11 UTC 2021 INFO: Data checksums enabled. Setting initdb to use data checksums.. Thu Sep 2 07:29:11 UTC 2021 INFO: Running initdb command: initdb -D /pgdata/data --data-checksums > /tmp/initdb.stdout 2> /tmp/initdb.stderr Thu Sep 2 07:29:19 UTC 2021 ERROR: Initializing the database (initdb): Unable to initialize the database: sh: line 1: 42 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=100 -c shared_buffers=1000 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 44 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=50 -c shared_buffers=500 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 46 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=40 -c shared_buffers=400 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 48 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=30 -c shared_buffers=300 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 50 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=200 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 52 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=16384 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 54 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=8192 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 56 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=4096 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 58 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=3584 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 60 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=3072 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 62 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=2560 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 64 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=2048 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 66 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=1536 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 68 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=1000 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 70 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=900 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 72 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=800 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 74 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=700 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 76 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=600 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 78 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=500 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 80 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=400 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 82 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=300 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 84 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=200 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 86 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=100 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 88 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=50 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 child process was terminated by signal 7: Bus error initdb: removing contents of data directory "/pgdata/data" -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#23526): https://lists.onap.org/g/onap-discuss/message/23526 Mute This Topic: https://lists.onap.org/mt/85322787/21656 Mute #honolulu:https://lists.onap.org/g/onap-discuss/mutehashtag/honolulu Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
aarna@anod-master:~$ kubectl logs -n onap dev-cps-postgres-primary-574975d9bc-79pl8 Thu Sep 2 07:23:59 UTC 2021 INFO: Image mode found: postgres Thu Sep 2 07:23:59 UTC 2021 INFO: Starting in 'postgres' mode Thu Sep 2 07:23:59 UTC 2021 INFO: Setting PGROOT to /usr/pgsql-13. Thu Sep 2 07:23:59 UTC 2021 INFO: PG_CTL_START_TIMEOUT set at: 60 Thu Sep 2 07:23:59 UTC 2021 INFO: PG_CTL_STOP_TIMEOUT set at: 60 Thu Sep 2 07:23:59 UTC 2021 INFO: PG_CTL_PROMOTE_TIMEOUT set at: 60 Thu Sep 2 07:23:59 UTC 2021 INFO: Cleaning up the old postmaster.pid file.. Thu Sep 2 07:23:59 UTC 2021 INFO: User ID is set to uid=26(postgres) gid=26(postgres) groups=26(postgres). Thu Sep 2 07:23:59 UTC 2021 INFO: Working on primary.. Thu Sep 2 07:23:59 UTC 2021 INFO: Initializing the primary database.. Thu Sep 2 07:23:59 UTC 2021 INFO: PGDATA is empty. ID is uid=26(postgres) gid=26(postgres) groups=26(postgres). Creating the PGDATA directory.. Thu Sep 2 07:23:59 UTC 2021 INFO: Starting initdb.. Thu Sep 2 07:23:59 UTC 2021 INFO: XLOGDIR not found. Using default pg_wal directory.. Thu Sep 2 07:23:59 UTC 2021 INFO: Data checksums enabled. Setting initdb to use data checksums.. Thu Sep 2 07:23:59 UTC 2021 INFO: Running initdb command: initdb -D /pgdata/data --data-checksums > /tmp/initdb.stdout 2> /tmp/initdb.stderr Thu Sep 2 07:24:08 UTC 2021 ERROR: Initializing the database (initdb): Unable to initialize the database: sh: line 1: 42 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=100 -c shared_buffers=1000 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 44 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=50 -c shared_buffers=500 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 46 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=40 -c shared_buffers=400 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 48 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=30 -c shared_buffers=300 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 50 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=200 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 52 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=16384 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 54 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=8192 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 56 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=4096 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 58 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=3584 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 60 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=3072 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 62 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=2560 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 64 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=2048 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 66 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=1536 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 68 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=1000 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 70 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=900 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 72 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=800 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 74 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=700 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 76 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=600 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 78 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=500 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 80 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=400 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 82 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=300 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 84 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=200 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 86 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=100 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 sh: line 1: 88 Bus error (core dumped) "/usr/pgsql-13/bin/postgres" --boot -x0 -F -c max_connections=20 -c shared_buffers=50 -c dynamic_shared_memory_type=posix < "/dev/null" > "/dev/null" 2>&1 child process was terminated by signal 7: Bus error initdb: removing contents of data directory "/pgdata/data"
