Geoff,
Thanks for the help.
We're using the API (that's the only way we know for this operation).
We're specifying the two networkids (tier and shared) but not the IP's.
A pretty basic call in other words.
One strange thing which may be possibly related is that quite often when
we have multiple NIC's we see _inconsistent_ IP <--> networkId mappings
reported, even in the GUI. This has been observed outwith VPC's -- so
not sure this is a VPC issue (though that's where we've tested mostly).
For the same VM, it can show up correct in one place, then mismatched
when we look in another place.
It is easy for us to reproduce, if we create a VPC, then create one or
more VM's in a tier with an additional networkId for a shared network.
Sometimes it comes back with mismatched IP-networkId. Port forwarding
seems permanently broken in the VPC thereafter -- even if we remove the
VM's with multiple NIC's!
Multiple networks seem to work fine (really well) apart from these two
issues (port-forwarding broken in the VPC, and sometimes incorrect
ip-networkId mapping).
I will file a bug, and update the relevant other issues -- unless you or
anyone provides us with wisdom in the meantime! I know VPC is an active
area of development (and a really nice one).
PS: For those following the original thread, for mgmt access to a VPC,
we are reverting to tracking a whole bunch of port-forwarding rules for
now; then we'll upgrade to this dual-network approach or VPC "road
warrior" VPN when they are available.
Best
Alex
On 27/02/2013 10:48, Geoff Higginbottom wrote:
Alex,
How are you deploying your VMs, presumably via via the API, and if so
are you specifying an IP address for each Network.
Regards
Geoff Higginbottom
*CTO / Cloud Architect*
D: +44(0)20 3603 0542 <tel:+442036030542> | S: +44(0)20 3603 0540
<tel:+442036030540> | M: +44(0)7968161581 <tel:+447968161581>
geoff.higginbot...@shapeblue.com
<mailto:geoff.higginbot...@shapeblue.com> | www.shapeblue.com
ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS
On 26 Feb 2013, at 13:10, "Alex Heneveld"
<alex.henev...@cloudsoftcorp.com
<mailto:alex.henev...@cloudsoftcorp.com>> wrote:
Hi Geoff,
Thanks for the reply. We are indeed trying very similar things.
Are you seeing any strangeness with this?
About half the time, we're getting the NIC's wrongly associated with
IP's by the Cloudstack API (i.e. IP in Tier CIDR is reported against
shared network and vice versa). The machines themselves seem to work
fine. But port-forwarding seems not to work when this happens.
--A
On 08/02/2013 03:31, Geoff Higginbottom wrote:
Hi Alex,
I've actually been working on a similar problem myself. If I
understand your solution, it's to use a shared network connected to
each tier of the VPC. If this is correct, surely you are 'breaking'
the VPC model as you have now connected each Tier to a common shared
network, meaning they are no longer isolated.
What we tested today was to map a unique network to each Tier, each
with a unique VLAN and CIDR, using a custom network offering
providing only DHCP service. These networks were connected to a
Firewall which enables a management network to connect to each tier,
but prevents the Tiers communicating directly.
As you point out, VMs have to be deployed using the API not the GUI,
and I can confirm this works in CloudStack 4.0.
We did have to create 'shared' networks in order to create a VM
connected to both a VPC Tier and another Network (it does not work
with an Isolated Network) but as the Network is only connected to VMs
belonging to one Tier, we did not compromise the VPC isolation model.
One final step was to add a persistent route to each VM, ensuring
traffic destined for the 'management' network was routed over the
secondary network and not the default network which is connected to
the VPC. This route can be added manually, or by using the 'user
data' feature to tell each VM which Gateway to use (it's different
for every Tier so cannot be baked into the template)
Regards
Geoff Higginbottom
CTO / Cloud Architect
D: +44(0)20 3603 0542<tel:+442036030542> | S: +44(0)20 3603
0540<tel:+442036030540> | M: +44(0)7968161581<tel:+447968161581>
geoff.higginbot...@shapeblue.com
<mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbot...@shapeblue.com>
| www.shapeblue.com <http://www.shapeblue.com>
ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS
Find out about our free webinars on Citrix Cloud
Technologies<http://www.shapeblue.com/citrix-cloud-technologies-webinar-series/>
On 7 Feb 2013, at 19:51, "Alex Heneveld"
<alex.henev...@cloudsoftcorp.com
<mailto:alex.henev...@cloudsoftcorp.com><mailto:alex.henev...@cloudsoftcorp.com>>
wrote:
An update on this. It seems there are currently 4 viable options for
solving this VPC mgmt access problem, summarised below. Thanks Chip
and Alex (minor questions in-line).
We're trying to set up a VPC/nTier-App such that a single VM (call it a
management node) outside the VPC has ssh access to the VM's inside the
VPC. (And to do this for multiple VPC's, same mgmt node.) What's the
best way to implement this?
1) Shared-network solution -- this is what we've done and I'm pleased
to say it works! (Alex, although this isn't exposed in the GUI, the
API call to attach an additional network to VPC nodes works fine --
in cloudplatform 3.0.5 at least)
2) "DIY" remote access VPN [Chip's suggestion] -- Chip, if I
understand your suggestion correctly it's basically that we
roll-our-own VPN on a VM in the VPC w/ ip-sec gateway to enable
remote-access ("road warrior") VPN'ing, since this isn't available
out of the box for VPC's (as per [1])
3) Port-forwarding on extra VPC public IP -- managing these
assignments is tedious (as Chip notes) but it is not as bad as it
sounds; we've had to do this before, in LXC-land where containers are
isolated. it has benefit over #2 of not needing a dedicated VPN
endpoint VM in the VPC, and over #1 not attaching a free-for-all mgmt
network; but it isn't elegant or fun.
4) s2s VPN [Alex Huang's suggestion] - i think this is a nice option
if you already have the beefy hardware (Cisco / Juniper) that this
requires, based on the wiki [2] at least -- is that right? although
it would be inefficient if that hardware isn't local (b/c all traffic
is routed through the remote gateway) and also note there is a
question in the wiki whether it will work between zones (but I see no
reason why it wouldn't?)
The mgmt VM can be in the same zone here (re Chip's question). If
the mgmt VM is in a different zone then #1 doesn't work (not unless
you use it to set up a proxy then use one of the other techniques to
connect in to it).
It will be nice when #754 is implemented [1] as that seems better
than any of these. :)
I hope this is useful to anyone else needing to do this. Any
comments or corrections welcome. And happy to share the code if
that's of interest.
Alex
[1] https://issues.apache.org/jira/browse/CLOUDSTACK-754
[2] https://cwiki.apache.org/CLOUDSTACK/site-to-site-vpn.html
On 06/02/2013 17:23, Alex Huang wrote:
-----Original Message-----
From: Chip Childers [mailto:chip.child...@sungard.com]
Sent: Wednesday, February 06, 2013 7:43 AM
To: cloudstack-users@incubator.apache.org
<mailto:cloudstack-users@incubator.apache.org><mailto:cloudstack-users@incubator.apache.org>
Subject: Re: mgmt VM access to VPC
On Wed, Feb 06, 2013 at 02:23:08AM +0000, Alex Heneveld wrote:
Hi,
We're trying to set up a VPC/nTier-App such that a single VM (call it a
management node) outside the VPC has ssh access to the VM's inside the
VPC. (And to do this for multiple VPC's, same mgmt node.) What's the
best way to implement this?
It seems like #754 [1] would be the right way to go about this when
available (is that right?) but already there are a few things we could
do now:
- set up an extra public IP on each tier with careful port forwarding
and ACL restricted to the mgmt node
- use an s2s vpn where the other "site" is just the mgmt node
- use a shared network, seems supported based on #748 [2] (but this
would break isolation?)
Any thoughts on these or others?
TIA,
Alex
[1] https://issues.apache.org/jira/browse/CLOUDSTACK-754
[2] https://issues.apache.org/jira/browse/CLOUDSTACK-748
Is this "other VM" going to be in a different zone?
This seems like you would have to consider it as being a completely
different entity from the VPC that it will be connecting into. With
that being the case, you're best off setting up an IP sec tunnel
into the VPC from that VM. I don't think you'll want to manage a bunch
of port forwarding rules for each VM in the VPC.
+1 I don't think shared network is supported by VPC at this point so
s2s vpn should be the best way to go.
--Alex
ShapeBlue provides a range of strategic and technical consulting and
implementation services to help IT Service Providers and Enterprises
to build a true IaaS compute cloud. ShapeBlue's expertise, combined
with CloudStack technology, allows IT Service Providers and
Enterprises to deliver true, utility based, IaaS to the customer or
end-user.
________________________________
This email and any attachments to it may be confidential and are
intended solely for the use of the individual to whom it is
addressed. Any views or opinions expressed are solely those of the
author and do not necessarily represent those of Shape Blue Ltd. If
you are not the intended recipient of this email, you must neither
take any action based upon its contents, nor copy or show it to
anyone. Please contact the sender if you believe you have received
this email in error. Shape Blue Ltd is a company incorporated in
England & Wales.
ShapeBlue provides a range of strategic and technical consulting and
implementation services to help IT Service Providers and Enterprises
to build a true IaaS compute cloud. ShapeBlue's expertise, combined
with CloudStack technology, allows IT Service Providers and
Enterprises to deliver true, utility based, IaaS to the customer or
end-user.
------------------------------------------------------------------------
This email and any attachments to it may be confidential and are
intended solely for the use of the individual to whom it is addressed.
Any views or opinions expressed are solely those of the author and do
not necessarily represent those of Shape Blue Ltd. If you are not the
intended recipient of this email, you must neither take any action
based upon its contents, nor copy or show it to anyone. Please contact
the sender if you believe you have received this email in error. Shape
Blue Ltd is a company incorporated in England & Wales.
