Public bug reported: With OVN build from source, devstack will fail on accessing /opt/stack/data/ovn, since ovn bits will be accessing with unprivileged user, while that directory will be created using elevated permissions. As a result, following crash can be observed:
+ lib/neutron_plugins/ovn_agent:_start_process:227 : sudo systemctl enable [email protected] Created symlink /etc/systemd/system/multi-user.target.wants/[email protected] → /etc/systemd/system/[email protected]. + lib/neutron_plugins/ovn_agent:_start_process:228 : sudo systemctl restart [email protected] Job for [email protected] failed because a timeout was exceeded. See "systemctl status [email protected]" and "journalctl -xe" for details. + lib/neutron_plugins/ovn_agent:_start_process:1 : exit_trap + ./devstack/stack.sh:exit_trap:507 : local r=1 ++ ./devstack/stack.sh:exit_trap:508 : jobs -p + ./devstack/stack.sh:exit_trap:508 : jobs=76154 + ./devstack/stack.sh:exit_trap:511 : [[ -n 76154 ]] + ./devstack/stack.sh:exit_trap:511 : [[ -n /opt/stack/logs/devstacklog.txt.2021-11-26-151741 ]] + ./devstack/stack.sh:exit_trap:511 : [[ True == \T\r\u\e ]] + ./devstack/stack.sh:exit_trap:512 : echo 'exit_trap: cleaning up child processes' exit_trap: cleaning up child processes + ./devstack/stack.sh:exit_trap:513 : kill 76154 + ./devstack/stack.sh:exit_trap:517 : '[' -f /tmp/tmp.vXTRCtpEh5 ']' + ./devstack/stack.sh:exit_trap:518 : rm /tmp/tmp.vXTRCtpEh5 + ./devstack/stack.sh:exit_trap:522 : kill_spinner + ./devstack/stack.sh:kill_spinner:417 : '[' '!' -z '' ']' + ./devstack/stack.sh:exit_trap:524 : [[ 1 -ne 0 ]] + ./devstack/stack.sh:exit_trap:525 : echo 'Error on exit' Error on exit + ./devstack/stack.sh:exit_trap:527 : type -p generate-subunit + ./devstack/stack.sh:exit_trap:528 : generate-subunit 1637939860 1408 fail + ./devstack/stack.sh:exit_trap:530 : [[ -z /opt/stack/logs ]] + ./devstack/stack.sh:exit_trap:533 : /usr/bin/python3.8 /home/ubuntu/devstack/tools/worlddump.py -d /opt/stack/logs + ./devstack/stack.sh:exit_trap:542 : exit 1 and status for the unit will reveal the issue: $ sudo systemctl status [email protected] ● [email protected] - Devstack [email protected] Loaded: loaded (/etc/systemd/system/[email protected]; enabled; vendor preset: enabled) Active: failed (Result: timeout) since Fri 2021-11-26 15:41:08 UTC; 3min 52s ago Process: 108582 ExecStart=/bin/bash /usr/local/share/ovn/scripts/ovn-ctl --no-monitor start_northd (code=killed, signal=TERM) Tasks: 0 (limit: 19175) Memory: 1.6M CGroup: /system.slice/system-devstack.slice/[email protected] Nov 26 15:39:38 devstack bash[108640]: chown: cannot access '/usr/local/etc/ovn': No such file or directory Nov 26 15:39:38 devstack ovsdb-server[108641]: ovs|00001|vlog|INFO|opened log file /opt/stack/logs/ovsdb-server-sb.log Nov 26 15:39:38 devstack ovsdb-server[108643]: ovs|00002|lockfile|WARN|/opt/stack/data/ovn/.ovnsb_db.db.~lock~: failed to open lock file: Permission denied Nov 26 15:39:38 devstack ovsdb-server[108643]: ovs|00003|lockfile|WARN|/opt/stack/data/ovn/.ovnsb_db.db.~lock~: failed to lock file: Resource temporarily unavailable Nov 26 15:39:38 devstack bash[108643]: ovsdb-server: I/O error: /opt/stack/data/ovn/ovnsb_db.db: failed to lock lockfile (Resource temporarily unavailable) Nov 26 15:39:38 devstack ovn-sbctl[108647]: ovs|00001|sbctl|INFO|Called as ovn-sbctl --no-leader-only init Nov 26 15:41:08 devstack systemd[1]: [email protected]: start operation timed out. Terminating. Nov 26 15:41:08 devstack systemd[1]: [email protected]: Killing process 108647 (ovn-sbctl) with signal SIGKILL. Nov 26 15:41:08 devstack systemd[1]: [email protected]: Failed with result 'timeout'. Nov 26 15:41:08 devstack systemd[1]: Failed to start Devstack [email protected]. ** Affects: devstack Importance: Undecided Assignee: Roman Dobosz (roman-dobosz) Status: In Progress ** Changed in: neutron Assignee: (unassigned) => Roman Dobosz (roman-dobosz) ** Project changed: neutron => devstack -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1952467 Title: OVN cannot create it's files when build from source Status in devstack: In Progress Bug description: With OVN build from source, devstack will fail on accessing /opt/stack/data/ovn, since ovn bits will be accessing with unprivileged user, while that directory will be created using elevated permissions. As a result, following crash can be observed: + lib/neutron_plugins/ovn_agent:_start_process:227 : sudo systemctl enable [email protected] Created symlink /etc/systemd/system/multi-user.target.wants/[email protected] → /etc/systemd/system/[email protected]. + lib/neutron_plugins/ovn_agent:_start_process:228 : sudo systemctl restart [email protected] Job for [email protected] failed because a timeout was exceeded. See "systemctl status [email protected]" and "journalctl -xe" for details. + lib/neutron_plugins/ovn_agent:_start_process:1 : exit_trap + ./devstack/stack.sh:exit_trap:507 : local r=1 ++ ./devstack/stack.sh:exit_trap:508 : jobs -p + ./devstack/stack.sh:exit_trap:508 : jobs=76154 + ./devstack/stack.sh:exit_trap:511 : [[ -n 76154 ]] + ./devstack/stack.sh:exit_trap:511 : [[ -n /opt/stack/logs/devstacklog.txt.2021-11-26-151741 ]] + ./devstack/stack.sh:exit_trap:511 : [[ True == \T\r\u\e ]] + ./devstack/stack.sh:exit_trap:512 : echo 'exit_trap: cleaning up child processes' exit_trap: cleaning up child processes + ./devstack/stack.sh:exit_trap:513 : kill 76154 + ./devstack/stack.sh:exit_trap:517 : '[' -f /tmp/tmp.vXTRCtpEh5 ']' + ./devstack/stack.sh:exit_trap:518 : rm /tmp/tmp.vXTRCtpEh5 + ./devstack/stack.sh:exit_trap:522 : kill_spinner + ./devstack/stack.sh:kill_spinner:417 : '[' '!' -z '' ']' + ./devstack/stack.sh:exit_trap:524 : [[ 1 -ne 0 ]] + ./devstack/stack.sh:exit_trap:525 : echo 'Error on exit' Error on exit + ./devstack/stack.sh:exit_trap:527 : type -p generate-subunit + ./devstack/stack.sh:exit_trap:528 : generate-subunit 1637939860 1408 fail + ./devstack/stack.sh:exit_trap:530 : [[ -z /opt/stack/logs ]] + ./devstack/stack.sh:exit_trap:533 : /usr/bin/python3.8 /home/ubuntu/devstack/tools/worlddump.py -d /opt/stack/logs + ./devstack/stack.sh:exit_trap:542 : exit 1 and status for the unit will reveal the issue: $ sudo systemctl status [email protected] ● [email protected] - Devstack [email protected] Loaded: loaded (/etc/systemd/system/[email protected]; enabled; vendor preset: enabled) Active: failed (Result: timeout) since Fri 2021-11-26 15:41:08 UTC; 3min 52s ago Process: 108582 ExecStart=/bin/bash /usr/local/share/ovn/scripts/ovn-ctl --no-monitor start_northd (code=killed, signal=TERM) Tasks: 0 (limit: 19175) Memory: 1.6M CGroup: /system.slice/system-devstack.slice/[email protected] Nov 26 15:39:38 devstack bash[108640]: chown: cannot access '/usr/local/etc/ovn': No such file or directory Nov 26 15:39:38 devstack ovsdb-server[108641]: ovs|00001|vlog|INFO|opened log file /opt/stack/logs/ovsdb-server-sb.log Nov 26 15:39:38 devstack ovsdb-server[108643]: ovs|00002|lockfile|WARN|/opt/stack/data/ovn/.ovnsb_db.db.~lock~: failed to open lock file: Permission denied Nov 26 15:39:38 devstack ovsdb-server[108643]: ovs|00003|lockfile|WARN|/opt/stack/data/ovn/.ovnsb_db.db.~lock~: failed to lock file: Resource temporarily unavailable Nov 26 15:39:38 devstack bash[108643]: ovsdb-server: I/O error: /opt/stack/data/ovn/ovnsb_db.db: failed to lock lockfile (Resource temporarily unavailable) Nov 26 15:39:38 devstack ovn-sbctl[108647]: ovs|00001|sbctl|INFO|Called as ovn-sbctl --no-leader-only init Nov 26 15:41:08 devstack systemd[1]: [email protected]: start operation timed out. Terminating. Nov 26 15:41:08 devstack systemd[1]: [email protected]: Killing process 108647 (ovn-sbctl) with signal SIGKILL. Nov 26 15:41:08 devstack systemd[1]: [email protected]: Failed with result 'timeout'. Nov 26 15:41:08 devstack systemd[1]: Failed to start Devstack [email protected]. To manage notifications about this bug go to: https://bugs.launchpad.net/devstack/+bug/1952467/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
