Sanjeev N created CLOUDSTACK-6819:
-------------------------------------
Summary: [OVs] delete network/account sends
OvsDestroyBridgeCommand to only one host
Key: CLOUDSTACK-6819
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6819
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Management Server
Affects Versions: 4.4.0
Environment: Latest build from 4.4 with commit
32bbc84db99d0e5f7f9b2a3fb41e4e783a2de350
Reporter: Sanjeev N
Assignee: Murali Reddy
Priority: Critical
Fix For: 4.4.0
[OVs] delete network/account sends OvsDestroyBridgeCommand to only one host
even though the network spanned more than one host
Steps to reproduce:
===============
1.Bring up CS in advanced zone with multiple clusters(2-3 clusters with 1 host
in each cluster)
2.Create network offering with connectivity service and OVS as the service
provider
3.Add one guest account and deploy few vms with this new account
4.Use host tags to deploy vms in all the clusters to make sure that network is
spanned accross all the clusters
5.Delete the account
Result:
=====
Account deletion was successful and also ovs bridges were deleted from both the
hosts but the ovsTunnel porr(vif) for this network was unplugged only from one
host's dom0(the host to which OvsDestroyBridgeCommand was sent) but not from
the other host's dom0
Observations:
===========
Following is the log snippet from MS log file during account deletion:
Network was snapped across two hosts Rack1Pod1Host14 and Rack1Pod1Host13 but
OvsDestroyBridgeCommand was sent only to Rack1Pod1Host13
2014-06-02 07:11:19,838 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-18:ctx-c73e49f0 job-56/job-57 ctx-e2515de4) Asking Ovs to
release NicProfile[26-16-2e06143e-28bd-4b43-ae0b-23a71bf0ed35-10.1.1.197-null
2014-06-02 07:11:19,839 DEBUG [c.c.n.e.OvsElement]
(Work-Job-Executor-18:ctx-c73e49f0 job-56/job-57 ctx-e2515de4) Checking if
OvsElement can handle service Connectivity on network acc2-cs-gre
2014-06-02 07:11:45,654 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Asking Ovs to
release NicProfile[30-18-273ebec1-867a-4081-8607-cb19af4c133d-10.1.1.52-null
2014-06-02 07:11:45,655 DEBUG [c.c.n.e.OvsElement]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Checking if
OvsElement can handle service Connectivity on network acc2-cs-gre
2014-06-02 07:11:45,665 DEBUG [c.c.n.o.OvsTunnelManagerImpl]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Destroying
bridge for network 207 on host:1
2014-06-02 07:11:45,670 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Seq
1-8504203471359062600: Sending { Cmd , MgmtId: 7332683579487, via:
1(Rack1Pod1Host13), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.OvsDestroyBridgeCommand":{"networkId":207,"name":"OVSTunnel992","hostId":1,"wait":0}}]
}
2014-06-02 07:11:45,670 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Seq
1-8504203471359062600: Executing: { Cmd , MgmtId: 7332683579487, via:
1(Rack1Pod1Host13), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.OvsDestroyBridgeCommand":{"networkId":207,"name":"OVSTunnel992","hostId":1,"wait":0}}]
}
2014-06-02 07:11:45,761 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-148:ctx-0a89d399) Xen Server network for tunnels found:OVSTunnel992
2014-06-02 07:11:45,807 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-148:ctx-0a89d399) Destroy temp dom0 vifOVSTunnel992 success
2014-06-02 07:11:46,127 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-148:ctx-0a89d399) OVS Bridge destroyed
2014-06-02 07:11:46,232 DEBUG [c.c.n.o.OvsTunnelManagerImpl]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Destroy bridge
fornetwork 207 successful
2014-06-02 07:11:46,234 DEBUG [c.c.n.o.OvsTunnelManagerImpl]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Destroying
tunnel to 1 from 4
2014-06-02 07:11:46,239 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Seq
4-8413287053881512061: Sending { Cmd , MgmtId: 7332683579487, via:
4(Rack1Pod1Host14), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.OvsDestroyTunnelCommand":{"networkId":207,"networkName":"OVSTunnel992","inPortName":"t992-4-1","wait":0}}]
}
2014-06-02 07:11:46,239 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Seq
4-8413287053881512061: Executing: { Cmd , MgmtId: 7332683579487, via:
4(Rack1Pod1Host14), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.OvsDestroyTunnelCommand":{"networkId":207,"networkName":"OVSTunnel992","inPortName":"t992-4-1","wait":0}}]
}
2014-06-02 07:11:46,324 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-296:ctx-5132f125) Xen Server network for tunnels found:OVSTunnel992
2014-06-02 07:11:46,460 DEBUG [c.c.n.o.OvsTunnelManagerImpl]
(Work-Job-Executor-19:ctx-39832471 job-56/job-58 ctx-67dba604) Destroy
tunnel(account:207,from:4, to:1) successful
2014-06-02 07:11:48,436 DEBUG [c.c.n.e.OvsElement]
(API-Job-Executor-31:ctx-bbed7dc8 job-56 ctx-fb3924b5) Checking if OvsElement
can handle service SourceNat on network acc2-cs-gre
2014-06-02 07:11:48,439 DEBUG [c.c.n.e.OvsElement]
(API-Job-Executor-31:ctx-bbed7dc8 job-56 ctx-fb3924b5) OvsElement can't provide
the SourceNat service on network acc2-cs-gre
2014-06-02 07:11:49,275 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-31:ctx-bbed7dc8 job-56 ctx-fb3924b5) Sending network shutdown
to Ovs
2014-06-02 07:11:49,276 DEBUG [c.c.n.e.OvsElement]
(API-Job-Executor-31:ctx-bbed7dc8 job-56 ctx-fb3924b5) Checking if OvsElement
can handle service Connectivity on network acc2-cs-gre
2014-06-02 07:12:00,061 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-20:ctx-a0854b67 job-56/job-59 ctx-03938585) Asking Ovs to
release NicProfile[27-17-adaa31ef-ad46-46ee-94ae-71b0b1f1858a-10.1.1.1-null
2014-06-02 07:12:00,061 DEBUG [c.c.n.e.OvsElement]
(Work-Job-Executor-20:ctx-a0854b67 job-56/job-59 ctx-03938585) Checking if
OvsElement can handle service Connectivity on network acc2-cs-gre
2014-06-02 07:12:00,312 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(API-Job-Executor-31:ctx-bbed7dc8 job-56 ctx-fb3924b5) Sending destroy to
com.cloud.network.element.OvsElement@6e3e9d6e
2014-06-02 07:12:00,312 DEBUG [c.c.n.e.OvsElement]
(API-Job-Executor-31:ctx-bbed7dc8 job-56 ctx-fb3924b5) Checking if OvsElement
can handle service Connectivity on network acc2-cs-gre
Attaching MS log file and ovstunnel log file from both the hosts
--
This message was sent by Atlassian JIRA
(v6.2#6252)
