[
https://issues.apache.org/jira/browse/CLOUDSTACK-8451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559300#comment-14559300
]
Andrija Panic commented on CLOUDSTACK-8451:
-------------------------------------------
*From agent logs...(setting source nat for eth2)*
2015-05-26 17:30:24,873 DEBUG [resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-3:null) Executing:
/usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh vpc_snat.sh
169.254.2.49 -A -l XXX.YYY.147.26 -c eth2
2015-05-26 17:30:25,001 DEBUG [resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-3:null) Execution is successful.
2015-05-26 17:30:25,001 DEBUG [resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-3:null) iptables: Bad rule (does a matching rule exist in
that chain?).
iptables: No chain/target/match by that name.
*From management logs (grep -i eth2 didnt give any explicit commands sent from
management server side, nor id=2 or similar)*
(mgmt log looks fine to me...)
2015-05-26 17:27:30,862 DEBUG [c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
(Job-Executor-19:ctx-d99dacc4 ctx-bb38451d) Removing nic
NicProfile[4903-2916-null-XXX.YYY.147.26-vlan://untagged of type Public from
the nics passed on vm start. The nic will be plugged later
2015-05-26 17:27:30,866 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-19:ctx-d99dacc4 ctx-bb38451d) Boot Args for
VM[DomainRouter|r-2916-VM]: vpccidr=10.0.0.0/8 domain=cs2cloud.internal
dns1=8.8.8.8 dns2= template=domP name=r-2916-VM eth0ip=169.254.2.49
eth0mask=255.255.0.0 type=vpcrouter disable_rp_filter=true
2015-05-26 17:27:30,947 DEBUG [c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
(Job-Executor-19:ctx-d99dacc4 ctx-bb38451d) Found 0 static routes to apply as a
part of vpc route VM[DomainRouter|r-2916-VM] start
2015-05-26 17:27:30,968 DEBUG [c.c.a.t.Request] (Job-Executor-19:ctx-d99dacc4
ctx-bb38451d) Seq 793-1613229855: Sending { Cmd , MgmtId: 161344838950, via:
793(cs12.domain.net), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.StartCommand":{"vm":{"id":2916,"name":"r-2916-VM","type":"DomainRouter","cpus":1,"minSpeed":166,"maxSpeed":1000,"minRam":268435456,"maxRam":268435456,"arch":"x86_64","os":"Debian
GNU/Linux 7(64-bit)","bootArgs":" vpccidr=10.0.0.0/8 domain=cs2cloud.internal
dns1=8.8.8.8 dns2= template=domP name=r-2916-VM eth0ip=169.254.2.49
eth0mask=255.255.0.0 type=vpcrouter
disable_rp_filter=true","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"ca8af10f1fd5804c","params":{"memoryOvercommitRatio":"1.0","cpuOvercommitRatio":"6.0"},"uuid":"518baeec-df0c-413e-9f26-07b7fb823601","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"59eab08e-4814-4e09-b1ee-34b357a430b2","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5b93422e-1a66-353d-88a8-2203f79b1dc6","id":209,"poolType":"RBD","host":"cephmon.domain.net","path":"cloudstack","port":6789,"url":"RBD://cephmon.domain.net/cloudstack/?ROLE=Primary&STOREUUID=5b93422e-1a66-353d-88a8-2203f79b1dc6"}},"name":"ROOT-2916","size":2621440000,"path":"59eab08e-4814-4e09-b1ee-34b357a430b2","volumeId":8416,"vmName":"r-2916-VM","accountId":2,"format":"RAW","id":8416,"deviceId":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"59eab08e-4814-4e09-b1ee-34b357a430b2","type":"ROOT","_details":{"managed":"false","storagePort":"6789","storageHost":"cephmon.domain.net","volumeSize":"2621440000"}}],"nics":[{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"2a657fb8-c645-47c2-a335-e2d2c7da030c","ip":"169.254.2.49","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:31","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"hostIp":"10.xxx.yyy.120","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.2.49","port":3922,"interval":6,"retries":100,"name":"r-2916-VM","wait":0}},{"com.cloud.agent.api.GetDomRVersionCmd":{"accessDetails":{"router.ip":"169.254.2.49","router.name":"r-2916-VM"},"wait":0}},{"com.cloud.agent.api.PlugNicCommand":{"nic":{"deviceId":1,"networkRateMbps":99999,"defaultNic":true,"uuid":"de8f637c-195d-4455-9035-81f8d4f74e09","ip":"XXX.YYY.147.26","netmask":"255.255.255.128","gateway":"XXX.YYY.147.1","mac":"06:f3:72:00:01:b2","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://untagged","isolationUri":"vlan://untagged","isSecurityGroupEnabled":false,"name":"breth1-500"},"instanceName":"r-2916-VM","vmType":"DomainRouter","wait":0}},{"com.cloud.agent.api.routing.IpAssocVpcCommand":{"ipAddresses":[{"accountId":2,"publicIp":"XXX.YYY.147.26","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":false,"broadcastUri":"vlan://untagged","vlanGateway":"XXX.YYY.147.1","vlanNetmask":"255.255.255.128","vifMacAddress":"06:f3:72:00:01:b2","networkRate":99999,"trafficType":"Public","networkName":"breth1-500"}],"accessDetails":{"router.guest.ip":"XXX.YYY.147.26","zone.network.type":"Advanced","router.ip":"169.254.2.49","router.name":"r-2916-VM"},"wait":0}},{"com.cloud.agent.api.routing.SetSourceNatCommand":{"ipAddress":{"accountId":2,"publicIp":"XXX.YYY.147.26","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":false,"broadcastUri":"vlan://untagged","vlanGateway":"XXX.YYY.147.1","vlanNetmask":"255.255.255.128","vifMacAddress":"06:f3:72:00:01:b2","networkRate":99999,"trafficType":"Public","networkName":"breth1-500"},"add":true,"accessDetails":{"zone.network.type":"Advanced","router.ip":"169.254.2.49","router.name":"r-2916-VM"},"wait":0}},{}]
}
...
...
2015-05-26 17:30:25,144 DEBUG [c.c.a.t.Request] (AgentManager-Handler-13:null)
Seq 793-1613229855: Processing: { Ans: , MgmtId: 161344838950, via: 793, Ver:
v1, Flags: 110,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":2916,"name":"r-2916-VM","type":"DomainRouter","cpus":1,"minSpeed":166,"maxSpeed":1000,"minRam":268435456,"maxRam":268435456,"arch":"x86_64","os":"Debian
GNU/Linux 7(64-bit)","bootArgs":" vpccidr=10.0.0.0/8 domain=cs2cloud.internal
dns1=8.8.8.8 dns2= template=domP name=r-2916-VM eth0ip=169.254.2.49
eth0mask=255.255.0.0 type=vpcrouter
disable_rp_filter=true","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"ca8af10f1fd5804c","vncAddr":"10.44.253.120","params":{"memoryOvercommitRatio":"1.0","cpuOvercommitRatio":"6.0"},"uuid":"518baeec-df0c-413e-9f26-07b7fb823601","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"59eab08e-4814-4e09-b1ee-34b357a430b2","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5b93422e-1a66-353d-88a8-2203f79b1dc6","id":209,"poolType":"RBD","host":"cephmon.domain.net","path":"cloudstack","port":6789,"url":"RBD://cephmon.domain.net/cloudstack/?ROLE=Primary&STOREUUID=5b93422e-1a66-353d-88a8-2203f79b1dc6"}},"name":"ROOT-2916","size":2621440000,"path":"59eab08e-4814-4e09-b1ee-34b357a430b2","volumeId":8416,"vmName":"r-2916-VM","accountId":2,"format":"RAW","id":8416,"deviceId":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"59eab08e-4814-4e09-b1ee-34b357a430b2","type":"ROOT","_details":{"managed":"false","storagePort":"6789","storageHost":"cephmon.domain.net","volumeSize":"2621440000"}}],"nics":[{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"2a657fb8-c645-47c2-a335-e2d2c7da030c","ip":"169.254.2.49","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:31","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"result":true,"wait":0}},{"com.cloud.agent.api.check.CheckSshAnswer":{"result":true,"wait":0}},{"com.cloud.agent.api.GetDomRVersionAnswer":{"templateVersion":"Cloudstack
Release 4.3.2 (64-bit) Wed Jan 28 18:38:51 UTC
2015","scriptsVersion":"253cafa254fc386e9fce204d9395a181","result":true,"details":"Cloudstack
Release 4.3.2 (64-bit) Wed Jan 28 18:38:51 UTC
2015&253cafa254fc386e9fce204d9395a181","wait":0}},{"com.cloud.agent.api.PlugNicAnswer":{"result":true,"details":"success","wait":0}},{"com.cloud.agent.api.routing.IpAssocAnswer":{"results":["XXX.YYY.147.26
-
success"],"result":true,"wait":0}},{"com.cloud.agent.api.routing.SetSourceNatAnswer":{"result":true,"details":"success","wait":0}},{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-2916-VM","bytesSent":0,"bytesReceived":0,"result":true,"wait":0}}]
}
Any idea why the script is run with eth2 as parameter when creating empty new
VPC (eth0, eth1, and lo present only) ?
I suspect this might have something to do with the fact that we dont use vlan
tags on public network,
I will try to test with tagging, since there was also some bugs arround this
*vlan://untagged* previously...so that is my best guess now :(
> Static Nat show wrong remote IP in VM behind VPC
> ------------------------------------------------
>
> Key: CLOUDSTACK-8451
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8451
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: KVM, Network Controller, Virtual Router
> Affects Versions: 4.4.3, 4.3.2, 4.5.1
> Environment: Ubuntu 14.04, ACS 4.5.1-SNAPSHOT
> Reporter: Andrija Panic
> Assignee: Rohit Yadav
>
> When configuring Port FOrwarding or Static NAT on VPC VR, and connect from
> outside world to VPC IP address, traffic gets forwarded to VM behind VPC.
> But if you run "netstat -antup | grep $PORT" (where port is i.e. ssh port) -
> given result will show that remote connections come from the Source NAT IP of
> the VR, instead of the real remote client IP.
> Example:
> private VM: 192.168.10.10
> Source NAT IP on VPC VR: 1.1.1.1
> Additional Public IP on VPC VR. 1.1.1.2
> Remote client public IP: 4.4.4.4 (external to VPC)
> Test:
> from 4.4.4.4 SSH to 1.1.1.2 port 22 (or any other port)
> inside 192.168.10.10 do "netstat -antup | grep 22"
> Result: Remote IP show is 1.1.1.1 instead of 4.4.4.4
> We found a solution (somwhat tested, and not sure if this would break
> anything...)
> Problem is in VRs iptables NAT table, POSTROUTING chain, rule:
> SNAT all -- * eth2 0.0.0.0/0 0.0.0.0/0 to:1.1.1.1
> where 1.1.1.1 is public IP of VR
> eth2: is Public Interface of VR
> When this rule is deleted, NAT is working fine.
> This is serious issue for anyone using VPC, since there is no way to see real
> remote client IP, and this no firewall funtionality inside VM, SIP doesnt
> work, web server logs are useless etc.
> I also experienced this problem with 4.3.x releases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
