Hi Pearl,
I had change to debug, following is the error captured:
2020-09-29 01:13:30,090 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:388b92e0) Processing command:
com.cloud.agent.api.SecurityGroupRulesCmd
2020-09-29 01:13:30,090 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-2:null) (logid:388b92e0) Looking for libvirtd
connection at: qemu:///system
2020-09-29 01:13:30,094 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) (logid:388b92e0) Checking default network
rules for vm i-23-77-VM
2020-09-29 01:13:30,094 ERROR [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:null) (logid:388b92e0) Unable to apply default
network rule for nic cloudbr0 for VM i-23-77-VM
2020-09-29 01:13:30,094 WARN
[resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper]
(agentRequest-Handler-2:null) (logid:388b92e0) Failed to program default
network rules for vm i-23-77-VM
2020-09-29 01:13:30,095 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) (logid:388b92e0) Seq 11-6057904448766738456: {
Ans: , MgmtId: 72752492851638, via: 11, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.SecurityGroupRuleAnswer":{"logSequenceNumber":15,"vmId":77,"reason":"PROGRAMMING_FAILED","result":false,"details":"programming
default network rules failed","wait":0}}] }
On Tue, Sep 29, 2020 at 12:38 PM Pearl d'Silva <pearl.dsi...@shapeblue.com>
wrote:
> Hi Hean,
>
>
> Could you try enabling debug logging on your hypervisor hosts, by running
> the following:
> *sed -i 's/INFO/DEBUG/g' /etc/cloudstack/agent/log4j-cloud.xml*
>
> And then restart the cloudstack-agent service on the hosts. Post this, try
> deploying a VM in the shared network. This may give a better insight into
> where exactly the issue lies.
>
> Thanks,
> Pearl
>
>
> Software Engineer
> pearl.dsi...@shapeblue.com
> www.shapeblue.com
>
>
>
>
>
>
> ------------------------------
> *From:* Ivan Kudryavtsev <i...@bw-sw.com>
> *Sent:* Monday, September 28, 2020 2:17 PM
> *To:* users <users@cloudstack.apache.org>
> *Subject:* Re: Cloudstack Advance with Security Group
>
> This is it.
>
> On Mon, Sep 28, 2020 at 3:46 PM Hean Seng <heans...@gmail.com> wrote:
>
> > In the log, I cannot see anything much, except a few lines showing above
> > . Not sure if this is a bug on 4.14.
> >
> >
> >
> > 2020-09-28 03:53:36,585 ERROR [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) (logid:b6a4c077) Unable to apply default
> > network rule for nic cloudbr0 for VM i-2-81-VM
> >
> > 2020-09-28 03:53:36,858 ERROR [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-3:null) (logid:74058678) Unable to apply default
> > network rule for nic cloudbr0 for VM i-2-81-VM
> >
> > 2020-09-28 03:53:36,858 WARN
> > [resource.wrapper.LibvirtSecurityGroupRulesCommandWrapper]
> > (agentRequest-Handler-3:null) (logid:74058678) Failed to program default
> > network rules for vm i-2-81-VM
> >
> >
> >
> >
> > On Mon, Sep 28, 2020 at 4:34 PM Ivan Kudryavtsev <i...@bw-sw.com> wrote:
> >
> > > Hi,
> > > no I'm on 4.11, so can not help with exact 4.14, and I'm on Ubuntu,
> > though,
> > > but for any KVM hypervisor Linux distribution, the logic is the same.
> > >
> > > On Mon, Sep 28, 2020 at 3:31 PM Hean Seng <heans...@gmail.com> wrote:
> > >
> > > > Hi
> > > >
> > > > Are you running on CentOS7 ?
> > > >
> > > > I am running on CentOS7 , ACS 4.14 , and seem there is no log at
> of
> > > > security_group.log
> > > >
> > > > # ls /var/log/cloudstack/agent/
> > > >
> > > > agent.log resizevolume.log setup.log
> > > >
> > > >
> > > > I recheck back the Intall guide, seems no missing anything.
> > > >
> > > >
> > > > Older intallation guide, 4.11 mentioned need , allow
> > > > /usr/lib/sysctl.d/00-system.conf
> > > >
> > > > # Enable netfilter on bridges. net.bridge.bridge-nf-call-ip6tables =
> 1
> > > > net.bridge.bridge-nf-call-iptables = 1
> > > net.bridge.bridge-nf-call-arptables
> > > > = 1
> > > >
> > > > And it has been done too.
> > > >
> > > >
> > > >
> > > > On Mon, Sep 28, 2020 at 4:05 PM Ivan Kudryavtsev <i...@bw-sw.com>
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > No, this is not the issue.
> > > > > It's a normal state of the system, as KVM hooks are a new and
> > optional
> > > > > feature of 4.14.
> > > > >
> > > > > You should find some sort of messages regarding security_groups at
> > > > > /var/log/cloudstack/agent/security_group.log
> > > > >
> > > > >
> > > > > On Mon, Sep 28, 2020 at 2:10 PM Hean Seng <heans...@gmail.com>
> > wrote:
> > > > >
> > > > > > I not sure where goes wrong, are you running on CentOS 7 ? I
> have
> > > this
> > > > > > error too, do you think is this contribute to the error as well:
> > > > > >
> > > > > > 2020-09-28 03:04:52,762 WARN [kvm.resource.LibvirtKvmAgentHook]
> > > > > > (agentRequest-Handler-5:null) (logid:4f23845b) Groovy script
> > > > > > '/etc/cloudstack/agent/hooks/libvirt-vm-xml-transformer.groovy'
> is
> > > not
> > > > > > available. Transformations will not be applied.
> > > > > >
> > > > > > 2020-09-28 03:04:52,762 WARN [kvm.resource.LibvirtKvmAgentHook]
> > > > > > (agentRequest-Handler-5:null) (logid:4f23845b) Groovy scripting
> > > engine
> > > > is
> > > > > > not initialized. Data transformation skipped.
> > > > > >
> > > > > > 2020-09-28 03:04:53,083 WARN [kvm.resource.LibvirtKvmAgentHook]
> > > > > > (agentRequest-Handler-5:null) (logid:4f23845b) Groovy script
> > > > > > '/etc/cloudstack/agent/hooks/libvirt-vm-state-change.groovy' is
> not
> > > > > > available. Transformations will not be applied.
> > > > > >
> > > > > > On Mon, Sep 28, 2020 at 2:27 PM Ivan Kudryavtsev <i...@bw-sw.com
> >
> > > > wrote:
> > > > > >
> > > > > > > This just means you installed it in the wrong way. Ebtables and
> > > > > Iptables
> > > > > > > must be filled with rules like
> > > > > > >
> > > > > > > -A i-6242-10304-def -m state --state RELATED,ESTABLISHED -j
> > ACCEPT
> > > > > > > -A i-6242-10304-def -p udp -m physdev --physdev-in vnet18
> > > > > > > --physdev-is-bridged -m udp --sport 68 --dport 67 -j ACCEPT
> > > > > > > -A i-6242-10304-def -p udp -m physdev --physdev-out vnet18
> > > > > > > --physdev-is-bridged -m udp --sport 67 --dport 68 -j ACCEPT
> > > > > > > -A i-6242-10304-def -m physdev --physdev-in vnet18
> > > > --physdev-is-bridged
> > > > > > -m
> > > > > > > set ! --match-set i-6242-10304-vm src -j DROP
> > > > > > > -A i-6242-10304-def -p udp -m physdev --physdev-in vnet18
> > > > > > > --physdev-is-bridged -m set --match-set i-6242-10304-vm src -m
> > udp
> > > > > > --dport
> > > > > > > 53 -j RETURN
> > > > > > > -A i-6242-10304-def -p tcp -m physdev --physdev-in vnet18
> > > > > > > --physdev-is-bridged -m set --match-set i-6242-10304-vm src -m
> > tcp
> > > > > > --dport
> > > > > > > 53 -j RETURN
> > > > > > > -A i-6242-10304-def -m physdev --physdev-in vnet18
> > > > --physdev-is-bridged
> > > > > > -m
> > > > > > > set --match-set i-6242-10304-vm src -j i-6242-10304-vm-eg
> > > > > > > -A i-6242-10304-def -m physdev --physdev-out vnet18
> > > > > --physdev-is-bridged
> > > > > > -j
> > > > > > > i-6242-10304-vm
> > > > > > > -A i-6242-10304-vm -p udp -m udp --dport 1:65535 -m state
> --state
> > > NEW
> > > > > -j
> > > > > > > ACCEPT
> > > > > > > -A i-6242-10304-vm -p tcp -m tcp --dport 1:65535 -m state
> --state
> > > NEW
> > > > > -j
> > > > > > > ACCEPT
> > > > > > > -A i-6242-10304-vm -p icmp -m icmp --icmp-type any -j ACCEPT
> > > > > > > -A i-6242-10304-vm -j DROP
> > > > > > >
> > > > > > >
> > > > > > > Bridge chain: i-4435-8929-vm-in, entries: 7, policy: ACCEPT
> > > > > > > -s ! 1e:0:32:0:2:2 -j DROP
> > > > > > > -p ARP -s ! 1e:0:32:0:2:2 -j DROP
> > > > > > > -p ARP --arp-mac-src ! 1e:0:32:0:2:2 -j DROP
> > > > > > > -p ARP -j i-4435-8929-vm-in-ips
> > > > > > > -p ARP --arp-op Request -j ACCEPT
> > > > > > > -p ARP --arp-op Reply -j ACCEPT
> > > > > > > -p ARP -j DROP
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Sep 28, 2020 at 1:10 PM Hean Seng <heans...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > I checked the hypervisor , it seems iptables is nothing
> inside
> > ,
> > > > > this
> > > > > > is
> > > > > > > > centos7 , initially i turnoff firewalld , but even i turn
> on
> > it
> > > > now
> > > > > > and
> > > > > > > > try to update the security group rules, it seems empty
> iptable
> > > > rules
> > > > > :
> > > > > > > >
> > > > > > > > [root@kvm03 ~]# iptables -L -v -n
> > > > > > > >
> > > > > > > > Chain INPUT (policy ACCEPT 82903 packets, 1170M bytes)
> > > > > > > >
> > > > > > > > pkts bytes target prot opt in out source
> > > > > > > > destination
> > > > > > > >
> > > > > > > >
> > > > > > > > Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
> > > > > > > >
> > > > > > > > pkts bytes target prot opt in out source
> > > > > > > > destination
> > > > > > > >
> > > > > > > >
> > > > > > > > Chain OUTPUT (policy ACCEPT 80505 packets, 25M bytes)
> > > > > > > >
> > > > > > > > pkts bytes target prot opt in out source
> > > > > > > > destination
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Sep 28, 2020 at 12:05 PM Pearl d'Silva <
> > > > > > > pearl.dsi...@shapeblue.com
> > > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Hean,
> > > > > > > > >
> > > > > > > > > In an Advanced Zone with Security Groups enabled, by
> default,
> > > > > egress
> > > > > > > > > traffic from the VM is allowed, while Ingress traffic is
> > > denied.
> > > > > > Hence,
> > > > > > > > as
> > > > > > > > > you rightly mentioned, security group rules are added
> > > > accordingly.
> > > > > > > These
> > > > > > > > > rules get added on the hypervisor host, and you can verify
> > > them,
> > > > by
> > > > > > > going
> > > > > > > > > into the host and searching for iptables rules
> corresponding
> > to
> > > > the
> > > > > > VM
> > > > > > > > > (internal name - i-x-y-VM).
> > > > > > > > > This blog maybe helpful in providing further details:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://shankerbalan.net/blog/cloudstack-advanced-zone-with-security-groups/
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Pearl
> > > > > > > > > ________________________________
> > > > > > > > > From: Hean Seng <heans...@gmail.com>
> > > > > > > > > Sent: Sunday, September 27, 2020 2:48 PM
> > > > > > > > > To: users@cloudstack.apache.org <
> users@cloudstack.apache.org
> > >
> > > > > > > > > Subject: Cloudstack Advance with Security Group
> > > > > > > > >
> > > > > > > > > Hi
> > > > > > > > >
> > > > > > > > > I created advance zone with security group, all working
> fine.
> > > > > > > > >
> > > > > > > > > But VMcreated ， seems the default security group that
> > assigned
> > > to
> > > > > the
> > > > > > > VM.
> > > > > > > > > all accept policy , i understand is Default Deny, and once
> > add
> > > > in
> > > > > > the
> > > > > > > > port
> > > > > > > > > in Security Group Ingress and Egress, only is allowed
> > > > > > > > >
> > > > > > > > > Also, is this rules created at VirtualRouter of the
> > > > SharedNetwork,
> > > > > or
> > > > > > > at
> > > > > > > > > the Hypervisor?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Regards,
> > > > > > > > > Hean Seng
> > > > > > > > >
> > > > > > > > > pearl.dsi...@shapeblue.com
> > > > > > > > > www.shapeblue.com
> > > > > > > > > 3 London Bridge Street, 3rd floor, News Building, London
> > SE1
> > > > > 9SGUK
> > > > > > > > > @shapeblue
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Regards,
> > > > > > > > Hean Seng
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > > Hean Seng
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Hean Seng
> > > >
> > >
> >
> >
> > --
> > Regards,
> > Hean Seng
> >
>
--
Regards,
Hean Seng
