Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
Hi, The work on SSL termination has started and is very near completion. the blue print is in https://blueprints.launchpad.net/neutron/+spec/lbaas-ssl-termination and wiki is in https://wiki.openstack.org/wiki/Neutron/LBaaS/SSL Do you see anything missing there? Regards, -Sam. -Original Message- From: Carlos Garza [mailto:carlos.ga...@rackspace.com] Sent: Saturday, April 19, 2014 2:47 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question On Apr 18, 2014, at 10:21 AM, Stephen Balukoff sbaluk...@bluebox.net wrote: Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) 1. Some customers want their servers to be external from our data centers for example the loadbalancer is in Chicago with rackspace hosting the loadbalancers and the back end pool members being on Amazon AWS servers. (We don't know why they would do this but a lot are doing it). They can't can't simply just audit the links between AWS and our DataCenters for PCI lots of backbones being crossed. so they just want encryption to their backend pool members. Also take note that amazon has chosen to support encryption http://aws.amazon.com/about-aws/whats-new/2011/10/04/amazon-s3-announces-server-side-encryption-support/ They've had it for a while now and for what ever reason a lot of customers are now demanding it from us as well. I agree they could simply just use HTTPS load balancing but they seem to think providers that don't offer encryption are inferior feature wise. 2. User that are on providers that are incapable of One Armed With Source Nat load balancing capabilities (See link below) are at the mercy of using X-forwarded for style headers to determine the original source of a connection. (A must if they want to know where abusive connections are coming from). Under traditional NAT routing the source ip will always be the loadbalancers ip so X-Forwarded for has been the traditional method of show the server this(This applies to HTTP load balancing as well). But in the case of SSL the loadbalancer unless its decrypting traffic won't be able to inject these headers in. and when the pool members are on an external network it is prudent to allow for encryption, this pretty much forces them to use a trusted LoadBalancer as a man in the middle to decrypt add X-forwarded for, then encrypting to the back end. http://docwiki.cisco.com/wiki/Basic_Load_Balancing_Using_One_Arm_Mode_with_Source_NAT_on_the_Cisco_Application_Control_Engine_Configuration_Example 3. Unless I'm mistaken it looks like encryption was already apart of the API or was accepted as a requirement for neutron lbaas. https://wiki.openstack.org/wiki/Neutron/LBaaS/SSL#Current_design is this document still valid? 4. We also assumed we were expected to support the use cases described in https://docs.google.com/document/d/1Ewl95yxAMq2fO0Z6Dz6fL-w2FScERQXQR1-mXuSINis/edit?pli=1 where case 7 specifically is asking for Re-encryption. We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) We terminate a lot of SSL connections on our loadbalancers as well and we get a lot of pressure for this kind of functionality. I think you have no customers using that functionality because you are unable to offer it which is the case for us as well. But due to a significant amount of pressure we have a solution already ready and waiting for testing on our CLB1.0 offering. We wished this was the case for us that only a few users are requesting this feature but we have customers that really do want their backend pool members on a separate non secure network or worse want this as a more advance form of HTTPS passthrough(TCP load balancing as your describing it). Providers may be able to secure their loadbalancers but they may not always be able to secure their back and forward connections. Users still want end to end encrypted connectivity but want the loadbalancer to be capable of making intelligent decisions(requiring decryption at the loadbalancer) as well as inject useful headers going to the back end pool member still need encryption functionality. When
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
Hi, Despite there are some good use cases for the re-encryption I think it’s out of scope for a Load Balancer. We can defer that functionality to the VPN – as long as we have a mechanism to insert a LoadBalancer as a VPN node we should get all kind of encryption infrastructure “for free”. I like the Unix philosophy of little programs doing one task very well and can be chained. So in our case we might want to chain a firewall to a load balancer to a VPN to get the functionality we want. Thoughts? German From: Stephen Balukoff [mailto:sbaluk...@bluebox.net] Sent: Friday, April 18, 2014 9:07 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Hi y'all! Carlos: When I say 'client cert' I'm talking about the certificate / key combination the load balancer will be using to initiate the SSL connection to the back-end server. The implication here is that if the back-end server doesn't like the client cert, it will reject the connection (as being not from a trusted source). By 'CA cert' I'm talking about the certificate (sans key) that the load balancer will be using the authenticate the back-end server. If the back-end server's server certificate isn't signed by the CA, then the load balancer should reject the connection. Of course, the use of a client cert or CA cert on the load balancer should be optional: As Clint pointed out, for some users, just using SSL without doing any particular authentication (on either the part of the load balancer or back-end) is going to be good enough. Anyway, the case for supporting re-encryption on the load-balancers has been solidly made, and the API proposal we're making will reflect this capability. Next question: When specific client certs / CAs are used for re-encryption, should these be associated with the pool or member? I could see an argument for either case: Pool (ie. one client cert / CA cert will be used for all members in a pool): * Consistency of back-end nodes within a pool is probably both extremely common, and a best practice. It's likely all will be accessed the same way. * Less flexible than certs associated with members, but also less complicated config. * For CA certs, assumes user knows how to manage their own PKI using a CA. Member (ie. load balancer will potentially use a different client cert / CA cert for each member individually): * Customers will sometimes run with inconsistent back-end nodes (eg. local nodes in a pool treated differently than remote nodes in a pool). * More flexible than certs associated with members, more complicated configuration. * If back-end certs are all individually self-signed (ie. no single CA used for all nodes), then certs must be associated with members. What are people seeing in the wild? Are your users using inconsistently-signed or per-node self-signed certs in a single pool? Thanks, Stephen On Fri, Apr 18, 2014 at 5:56 PM, Carlos Garza carlos.ga...@rackspace.commailto:carlos.ga...@rackspace.com wrote: On Apr 18, 2014, at 12:36 PM, Stephen Balukoff sbaluk...@bluebox.netmailto:sbaluk...@bluebox.net wrote: Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. If you choose to support re-encryption on your service then you are free to charge for the extra CPU cycles. I'm convinced re-encryption and SslTermination is general needs to be mandatory but I think the API should allow them to be specified. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. Yes but considering you have no problem advocating pure ssl termination for your customers(Decryption on the front end and plain text) on the back end I'm actually surprised this disturbs you. I would recommend users use Straight SSL passthrough or re-enecryption but I wouldn't force this on them should they choose naked encryption with no checking. Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a pool or to a member.) When you say client Cert
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
Excerpts from Eichberger, German's message of 2014-04-21 11:51:05 -0700: Hi, Despite there are some good use cases for the re-encryption I think it’s out of scope for a Load Balancer. We can defer that functionality to the VPN – as long as we have a mechanism to insert a LoadBalancer as a VPN node we should get all kind of encryption infrastructure “for free”. I like the Unix philosophy of little programs doing one task very well and can be chained. So in our case we might want to chain a firewall to a load balancer to a VPN to get the functionality we want. I think that only makes things simpler in an IPv6+IPSec situation (which, btw, would be fantastic and should be something we all drive OpenStack toward). But if you have to add something like OpenVPN to the load balancer service nodes, I'm not sure you're saving any complexity by using VPN vs. something like stunnel. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
German: I'm hearing from a lot of different sources / organizations on this list that re-encryption at the load balancer is a must-have feature. And I was already part of previous discussions on SSL functionality. ( https://wiki.openstack.org/wiki/Neutron/LBaaS/SSL ) Also, even if the load balancer does support re-encryption on the back-end, this doesn't prevent users from using a VPN-based solution either. Stephen On Mon, Apr 21, 2014 at 11:51 AM, Eichberger, German german.eichber...@hp.com wrote: Hi, Despite there are some good use cases for the re-encryption I think it’s out of scope for a Load Balancer. We can defer that functionality to the VPN – as long as we have a mechanism to insert a LoadBalancer as a VPN node we should get all kind of encryption infrastructure “for free”. I like the Unix philosophy of little programs doing one task very well and can be chained. So in our case we might want to chain a firewall to a load balancer to a VPN to get the functionality we want. Thoughts? German *From:* Stephen Balukoff [mailto:sbaluk...@bluebox.net] *Sent:* Friday, April 18, 2014 9:07 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Hi y'all! Carlos: When I say 'client cert' I'm talking about the certificate / key combination the load balancer will be using to initiate the SSL connection to the back-end server. The implication here is that if the back-end server doesn't like the client cert, it will reject the connection (as being not from a trusted source). By 'CA cert' I'm talking about the certificate (sans key) that the load balancer will be using the authenticate the back-end server. If the back-end server's server certificate isn't signed by the CA, then the load balancer should reject the connection. Of course, the use of a client cert or CA cert on the load balancer should be optional: As Clint pointed out, for some users, just using SSL without doing any particular authentication (on either the part of the load balancer or back-end) is going to be good enough. Anyway, the case for supporting re-encryption on the load-balancers has been solidly made, and the API proposal we're making will reflect this capability. Next question: When specific client certs / CAs are used for re-encryption, should these be associated with the pool or member? I could see an argument for either case: *Pool* (ie. one client cert / CA cert will be used for all members in a pool): * Consistency of back-end nodes within a pool is probably both extremely common, and a best practice. It's likely all will be accessed the same way. * Less flexible than certs associated with members, but also less complicated config. * For CA certs, assumes user knows how to manage their own PKI using a CA. *Member* (ie. load balancer will potentially use a different client cert / CA cert for each member individually): * Customers will sometimes run with inconsistent back-end nodes (eg. local nodes in a pool treated differently than remote nodes in a pool). * More flexible than certs associated with members, more complicated configuration. * If back-end certs are all individually self-signed (ie. no single CA used for all nodes), then certs must be associated with members. What are people seeing in the wild? Are your users using inconsistently-signed or per-node self-signed certs in a single pool? Thanks, Stephen On Fri, Apr 18, 2014 at 5:56 PM, Carlos Garza carlos.ga...@rackspace.com wrote: On Apr 18, 2014, at 12:36 PM, Stephen Balukoff sbaluk...@bluebox.net wrote: Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. If you choose to support re-encryption on your service then you are free to charge for the extra CPU cycles. I'm convinced re-encryption and SslTermination is general needs to be mandatory but I think the API should allow them to be specified. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. Yes but considering you have no problem
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
On Apr 21, 2014, at 1:51 PM, Eichberger, German german.eichber...@hp.commailto:german.eichber...@hp.com wrote: Hi, Despite there are some good use cases for the re-encryption I think it’s out of scope for a Load Balancer. We can defer that functionality to the VPN – as long as we have a mechanism to insert a LoadBalancer as a VPN node we should get all kind of encryption infrastructure “for free”. I think the feature should be apart of the API but I think it should be up to the vender to implement the feature or not since some venders can't. Plus an end user might not be able to append a vpn tunnel on the tail of the loadbalancer. I like the Unix philosophy of little programs doing one task very well and can be chained. So in our case we might want to chain a firewall to a load balancer to a VPN to get the functionality we want. I like that philosophy as well but must admit that the chains do break when versions or interactions of these components change. GNU's Autotools for example is a nightmare compared to Maven for Java. Even simpler tools like sort, tail, broke some tools I used to use. Monolithic tools like emacs likewise seem to be doing daily well. I get the impression that a the simple chained tool philosophy came from the era where individual programs had to be small enough to fit in memory and data would be spooled to tape as the intermediary pipe. Still a nice idea though for admins. Thoughts? German From: Stephen Balukoff [mailto:sbaluk...@bluebox.nethttp://bluebox.net] Sent: Friday, April 18, 2014 9:07 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Hi y'all! Carlos: When I say 'client cert' I'm talking about the certificate / key combination the load balancer will be using to initiate the SSL connection to the back-end server. The implication here is that if the back-end server doesn't like the client cert, it will reject the connection (as being not from a trusted source). By 'CA cert' I'm talking about the certificate (sans key) that the load balancer will be using the authenticate the back-end server. If the back-end server's server certificate isn't signed by the CA, then the load balancer should reject the connection. Of course, the use of a client cert or CA cert on the load balancer should be optional: As Clint pointed out, for some users, just using SSL without doing any particular authentication (on either the part of the load balancer or back-end) is going to be good enough. Anyway, the case for supporting re-encryption on the load-balancers has been solidly made, and the API proposal we're making will reflect this capability. Next question: When specific client certs / CAs are used for re-encryption, should these be associated with the pool or member? I could see an argument for either case: Pool (ie. one client cert / CA cert will be used for all members in a pool): * Consistency of back-end nodes within a pool is probably both extremely common, and a best practice. It's likely all will be accessed the same way. * Less flexible than certs associated with members, but also less complicated config. * For CA certs, assumes user knows how to manage their own PKI using a CA. Member (ie. load balancer will potentially use a different client cert / CA cert for each member individually): * Customers will sometimes run with inconsistent back-end nodes (eg. local nodes in a pool treated differently than remote nodes in a pool). * More flexible than certs associated with members, more complicated configuration. * If back-end certs are all individually self-signed (ie. no single CA used for all nodes), then certs must be associated with members. What are people seeing in the wild? Are your users using inconsistently-signed or per-node self-signed certs in a single pool? Thanks, Stephen On Fri, Apr 18, 2014 at 5:56 PM, Carlos Garza carlos.ga...@rackspace.commailto:carlos.ga...@rackspace.com wrote: On Apr 18, 2014, at 12:36 PM, Stephen Balukoff sbaluk...@bluebox.netmailto:sbaluk...@bluebox.net wrote: Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. If you choose to support re-encryption on your service then you are free to charge for the extra CPU cycles. I'm convinced re-encryption and SslTermination is general needs to be mandatory but I think the API
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
to install CA certificates that sign backend servers Keys via PKIX path building. I would even like to offer the customer the ability to skip hostname validation since not every one wants to expose DNS entries for IPs that are not publicly routable anyways. Unless your suggesting that we should force this on the user which likewise forces us to host a name server that maps hosts to the X509s subject CN fields. Users should be free to validate back end hostnames, just the subject name and key or no validation at all. It should be up to them. It's a bit of a rabbit hole, eh. Stephen On Fri, Apr 18, 2014 at 10:21 AM, Eichberger, German german.eichber...@hp.commailto:german.eichber...@hp.com wrote: Hi Stephen, The use case is that the Load Balancer needs to look at the HTTP requests be it to add an X-Forward field or change the timeout – but the network between the load balancer and the nodes is not completely private and the sensitive information needs to be again transmitted encrypted. This is admittedly an edge case but we had to implement a similar scheme for HP Cloud’s swift storage. German From: Stephen Balukoff [mailto:sbaluk...@bluebox.netmailto:sbaluk...@bluebox.net] Sent: Friday, April 18, 2014 8:22 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807tel:%28800%29613-4305%20x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807tel:%28800%29613-4305%20x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
On 04/18/2014 11:21 AM, Stephen Balukoff wrote: Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) Look at it this way. SSL to the Endpoint protects you on the public internet. That means that at each of the hops from you to the Datacenter, no one can read your traffic. So, if you are at the local coffee shop, wokring on your Neutron setup, no one can see more than the URLs that you are using. From there, it goes to the shop's ISP, thorugh a couple of hops, and then ends up at your datacenter. From the ISP to the datacenter, while it is good to be secure, the likelihood of random attack is low: these are arelatviely secured links, and with companies that have economic incentive not to hack your traffic. Don't get me wrong, there is a real possibility for attack, but that is not your big risk. So, now you are at your datacenter, and you want to talk to Neutron' API server. You hit the SSL terminiation, and your traffic is decrypted. And send, in the clear, with your userid and password, to Keystone to get a token. Same as everyone else talking to that keystone server. Same as everyone else talking to every public server in this data center. So what you think no one has the ability to run custom code. Um, this is OpenStack. Random VMs just teeming with all sorts of code, malicious, friendly, intentional, whatever, is being run all over the place. So what is protecting your unsecure socket connection from all of this code? Neutron.Specifically, making sure that no one has messed up neutron connectivity and managed to keep that route from the SSL terminator to the Neutron API server locked up, so none of those nasty VMs can grab and sniff it. Oh sure...its never gonna happen, right? Look at it like swimming in a public pool. There, the number of swimmers would be limited by the size of the pool, fire regulations, and physical access. This is the Virtual world. There are hundreds if not thousands of people swimming in this pool. I'll stop the biological analogy because some people reading this might be eating. SSL. Everywhere. We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
Hello Stephen, One use case we have, which was actually a highly requested feature for our service, was to ensure that traffic within the internal cloud network was not passed in the clear. I believe this mainly stems from the customers security requirements. I understand this reasoning to allow a centralized place to correct/prevent potential SSL attacks while still assuring data is secure all the way to the backend. I could probably dig up more details if this isn't clear enough, but is the way I understand this particular feature. Thanks, Phil From: Stephen Balukoff sbaluk...@bluebox.netmailto:sbaluk...@bluebox.net Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Friday, April 18, 2014 10:21 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
Hi Stephen, The use case is that the Load Balancer needs to look at the HTTP requests be it to add an X-Forward field or change the timeout – but the network between the load balancer and the nodes is not completely private and the sensitive information needs to be again transmitted encrypted. This is admittedly an edge case but we had to implement a similar scheme for HP Cloud’s swift storage. German From: Stephen Balukoff [mailto:sbaluk...@bluebox.net] Sent: Friday, April 18, 2014 8:22 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a pool or to a member.) It's a bit of a rabbit hole, eh. Stephen On Fri, Apr 18, 2014 at 10:21 AM, Eichberger, German german.eichber...@hp.com wrote: Hi Stephen, The use case is that the Load Balancer needs to look at the HTTP requests be it to add an X-Forward field or change the timeout – but the network between the load balancer and the nodes is not completely private and the sensitive information needs to be again transmitted encrypted. This is admittedly an edge case but we had to implement a similar scheme for HP Cloud’s swift storage. German *From:* Stephen Balukoff [mailto:sbaluk...@bluebox.net] *Sent:* Friday, April 18, 2014 8:22 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
There is no reasoning mentioned in AWS, but they do allow re-encryption. http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/config-backend-auth.html For reasons I don’t understand, the workflow allows to configure backend-server certificates to be trusted and it doesn’t accept client certificates or CA certificates. Thanks, Vijay V. From: Stephen Balukoff [mailto:sbaluk...@bluebox.net] Sent: Friday, April 18, 2014 11:06 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a pool or to a member.) It's a bit of a rabbit hole, eh. Stephen On Fri, Apr 18, 2014 at 10:21 AM, Eichberger, German german.eichber...@hp.commailto:german.eichber...@hp.com wrote: Hi Stephen, The use case is that the Load Balancer needs to look at the HTTP requests be it to add an X-Forward field or change the timeout – but the network between the load balancer and the nodes is not completely private and the sensitive information needs to be again transmitted encrypted. This is admittedly an edge case but we had to implement a similar scheme for HP Cloud’s swift storage. German From: Stephen Balukoff [mailto:sbaluk...@bluebox.netmailto:sbaluk...@bluebox.net] Sent: Friday, April 18, 2014 8:22 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807tel:%28800%29613-4305%20x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
Excerpts from Stephen Balukoff's message of 2014-04-18 10:36:11 -0700: Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) No one person or even group of people on the operator's network will have full access to everything. Security is best when it comes in layers. Area 51 doesn't just have a guard shack and then you drive right into the hangars with the UFO's and alien autopsies. There are sensors, mobile guards, secondary checkpoints, locks on the outer doors, and locks on the inner doors. And perhaps most importantly, the MP who approves your entry into the first gate, does not even have access to the next one. Your SSL terminator is a gate. What happens once an attacker (whoever that may be, your disgruntled sysadmin, or rogue hackers) is behind that gate _may_ be important. Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. A passive attack where the MITM does not have to witness the initial handshake or decrypt/reencrypt to sniff things is quite a bit easier to pull off and would be harder to detect. So almost no extra security is not really accurate. But this is just one point of data for risk assessment. Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a pool or to a member.) It's a bit of a rabbit hole, eh. Security turns into an endless rat hole when you just look at it as a product, such as A secure load balancer. If, however, you consider that it is really just a process of risk assessment and mitigation, then you can find a sweet spot that works in your business model. How much does it cost to mitigate the risk of unencrypted backend traffic from the load balancer? What is the potential loss if the traffic is sniffed? How likely is it that it will be sniffed? .. Those are ongoing questions that need to be asked and then reevaluated, but they don't have a fruitless stream of what-if's that have to be baked in like the product discussion. It's just part of your process, and processes go on until they aren't needed anymore. IMO a large part of operating a cloud is decoupling the ability to setup a system from the ability to enable your business with a system. So if you can communicate the risks of doing without backend encryption, and charge the users appropriately when they choose that the risk is worth the added cost, then I think it is worth it to automate the setup of CA's and client certs and put that behind an API. Luckily, you will likely find many in the OpenStack community who can turn that into a business opportunity and will help. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
+1 for German's use cases. We need SSL re-encryption for decisions the load balancer needs to make at the l7 layer as well. Thanks Clint, for your thorough explanation from a security standpoint. Cheers, --Jorge On 4/18/14 1:38 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Stephen Balukoff's message of 2014-04-18 10:36:11 -0700: Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) No one person or even group of people on the operator's network will have full access to everything. Security is best when it comes in layers. Area 51 doesn't just have a guard shack and then you drive right into the hangars with the UFO's and alien autopsies. There are sensors, mobile guards, secondary checkpoints, locks on the outer doors, and locks on the inner doors. And perhaps most importantly, the MP who approves your entry into the first gate, does not even have access to the next one. Your SSL terminator is a gate. What happens once an attacker (whoever that may be, your disgruntled sysadmin, or rogue hackers) is behind that gate _may_ be important. Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. A passive attack where the MITM does not have to witness the initial handshake or decrypt/reencrypt to sniff things is quite a bit easier to pull off and would be harder to detect. So almost no extra security is not really accurate. But this is just one point of data for risk assessment. Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a pool or to a member.) It's a bit of a rabbit hole, eh. Security turns into an endless rat hole when you just look at it as a product, such as A secure load balancer. If, however, you consider that it is really just a process of risk assessment and mitigation, then you can find a sweet spot that works in your business model. How much does it cost to mitigate the risk of unencrypted backend traffic from the load balancer? What is the potential loss if the traffic is sniffed? How likely is it that it will be sniffed? .. Those are ongoing questions that need to be asked and then reevaluated, but they don't have a fruitless stream of what-if's that have to be baked in like the product discussion. It's just part of your process, and processes go on until they aren't needed anymore. IMO a large part of operating a cloud is decoupling the ability to setup a system from the ability to enable your business with a system. So if you can communicate the risks of doing without backend encryption, and charge the users appropriately when they choose that the risk is worth the added cost, then I think it is worth it to automate the setup of CA's and client certs and put that behind an API. Luckily, you will likely find many in the OpenStack community who can turn that into a business opportunity and will help. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
+1 for the discussion Remember, a cloud does not always have all its backend co-located. There are sometimes AZs and often other hidden network hops. And, to ask the obvious, what do you think the response is when you whisper NSA in a crowded Google data center? --Rocky -Original Message- From: Jorge Miramontes [mailto:jorge.miramon...@rackspace.com] Sent: Friday, April 18, 2014 2:13 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question +1 for German's use cases. We need SSL re-encryption for decisions the load balancer needs to make at the l7 layer as well. Thanks Clint, for your thorough explanation from a security standpoint. Cheers, --Jorge On 4/18/14 1:38 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Stephen Balukoff's message of 2014-04-18 10:36:11 -0700: Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) No one person or even group of people on the operator's network will have full access to everything. Security is best when it comes in layers. Area 51 doesn't just have a guard shack and then you drive right into the hangars with the UFO's and alien autopsies. There are sensors, mobile guards, secondary checkpoints, locks on the outer doors, and locks on the inner doors. And perhaps most importantly, the MP who approves your entry into the first gate, does not even have access to the next one. Your SSL terminator is a gate. What happens once an attacker (whoever that may be, your disgruntled sysadmin, or rogue hackers) is behind that gate _may_ be important. Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. A passive attack where the MITM does not have to witness the initial handshake or decrypt/reencrypt to sniff things is quite a bit easier to pull off and would be harder to detect. So almost no extra security is not really accurate. But this is just one point of data for risk assessment. Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a pool or to a member.) It's a bit of a rabbit hole, eh. Security turns into an endless rat hole when you just look at it as a product, such as A secure load balancer. If, however, you consider that it is really just a process of risk assessment and mitigation, then you can find a sweet spot that works in your business model. How much does it cost to mitigate the risk of unencrypted backend traffic from the load balancer? What is the potential loss if the traffic is sniffed? How likely is it that it will be sniffed? .. Those are ongoing questions that need to be asked and then reevaluated, but they don't have a fruitless stream of what-if's that have to be baked in like the product discussion. It's just part of your process, and processes go on until they aren't needed anymore. IMO a large part of operating a cloud is decoupling the ability to setup a system from the ability to enable your business with a system. So if you can communicate the risks of doing without backend encryption, and charge the users appropriately when they choose that the risk is worth the added cost, then I think it is worth it to automate the setup of CA's and client certs and put that behind an API. Luckily, you will likely find many in the OpenStack community who can turn that into a business opportunity and will help. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
On Apr 18, 2014, at 10:21 AM, Stephen Balukoff sbaluk...@bluebox.net wrote: Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) 1. Some customers want their servers to be external from our data centers for example the loadbalancer is in Chicago with rackspace hosting the loadbalancers and the back end pool members being on Amazon AWS servers. (We don't know why they would do this but a lot are doing it). They can't can't simply just audit the links between AWS and our DataCenters for PCI lots of backbones being crossed. so they just want encryption to their backend pool members. Also take note that amazon has chosen to support encryption http://aws.amazon.com/about-aws/whats-new/2011/10/04/amazon-s3-announces-server-side-encryption-support/ They've had it for a while now and for what ever reason a lot of customers are now demanding it from us as well. I agree they could simply just use HTTPS load balancing but they seem to think providers that don't offer encryption are inferior feature wise. 2. User that are on providers that are incapable of One Armed With Source Nat load balancing capabilities (See link below) are at the mercy of using X-forwarded for style headers to determine the original source of a connection. (A must if they want to know where abusive connections are coming from). Under traditional NAT routing the source ip will always be the loadbalancers ip so X-Forwarded for has been the traditional method of show the server this(This applies to HTTP load balancing as well). But in the case of SSL the loadbalancer unless its decrypting traffic won't be able to inject these headers in. and when the pool members are on an external network it is prudent to allow for encryption, this pretty much forces them to use a trusted LoadBalancer as a man in the middle to decrypt add X-forwarded for, then encrypting to the back end. http://docwiki.cisco.com/wiki/Basic_Load_Balancing_Using_One_Arm_Mode_with_Source_NAT_on_the_Cisco_Application_Control_Engine_Configuration_Example 3. Unless I'm mistaken it looks like encryption was already apart of the API or was accepted as a requirement for neutron lbaas. https://wiki.openstack.org/wiki/Neutron/LBaaS/SSL#Current_design is this document still valid? 4. We also assumed we were expected to support the use cases described in https://docs.google.com/document/d/1Ewl95yxAMq2fO0Z6Dz6fL-w2FScERQXQR1-mXuSINis/edit?pli=1 where case 7 specifically is asking for Re-encryption. We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) We terminate a lot of SSL connections on our loadbalancers as well and we get a lot of pressure for this kind of functionality. I think you have no customers using that functionality because you are unable to offer it which is the case for us as well. But due to a significant amount of pressure we have a solution already ready and waiting for testing on our CLB1.0 offering. We wished this was the case for us that only a few users are requesting this feature but we have customers that really do want their backend pool members on a separate non secure network or worse want this as a more advance form of HTTPS passthrough(TCP load balancing as your describing it). Providers may be able to secure their loadbalancers but they may not always be able to secure their back and forward connections. Users still want end to end encrypted connectivity but want the loadbalancer to be capable of making intelligent decisions(requiring decryption at the loadbalancer) as well as inject useful headers going to the back end pool member still need encryption functionality. When your customers do Straight TCP load balancing are you noticing you can only offer IP based session persistence at that point? If you only allow ip based persistence customers that share a NAT router will all hit the same node every time. We have lots of customers behind corporate NAT routers and they notice very quickly that hundreds of clients are all being shoved onto one back end pool member. They as of now only have the option to turn off session persistence but that breaks applications that require locally maintained sessions. We could offer TLS
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
On Apr 18, 2014, at 12:59 PM, Vijay Venkatachalam vijay.venkatacha...@citrix.commailto:vijay.venkatacha...@citrix.com wrote: There is no reasoning mentioned in AWS, but they do allow re-encryption. Is their also no reason to mention: BigIp's F5 LoadBalancers http://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/ltm_implementation/sol_http_ssl.html A10 LoadBalaners http://www.a10networks.com/resources/files/CS-Earth_Class_Mail.pdf Netscaler http://support.citrix.com/proddocs/topic/netscaler-traffic-management-10-map/ns-ssl-offloading-end-to-end-encypt-tsk.html Finally Stingray https://splash.riverbed.com/thread/5473 All big players in LoadBalancing. would be the reasoning. http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/config-backend-auth.html For reasons I don’t understand, the workflow allows to configure backend-server certificates to be trusted and it doesn’t accept client certificates or CA certificates. Thanks, Vijay V. From: Stephen Balukoff [mailto:sbaluk...@bluebox.nethttp://bluebox.net] Sent: Friday, April 18, 2014 11:06 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a pool or to a member.) It's a bit of a rabbit hole, eh. Stephen On Fri, Apr 18, 2014 at 10:21 AM, Eichberger, German german.eichber...@hp.commailto:german.eichber...@hp.com wrote: Hi Stephen, The use case is that the Load Balancer needs to look at the HTTP requests be it to add an X-Forward field or change the timeout – but the network between the load balancer and the nodes is not completely private and the sensitive information needs to be again transmitted encrypted. This is admittedly an edge case but we had to implement a similar scheme for HP Cloud’s swift storage. German From: Stephen Balukoff [mailto:sbaluk...@bluebox.netmailto:sbaluk...@bluebox.net] Sent: Friday, April 18, 2014 8:22 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807tel:%28800%29613-4305%20x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
On Apr 18, 2014, at 12:36 PM, Stephen Balukoff sbaluk...@bluebox.netmailto:sbaluk...@bluebox.net wrote: Dang. I was hoping this wasn't the case. (I personally think it's a little silly not to trust your service provider to secure a network when they have root access to all the machines powering your cloud... but I digress.) Part of the reason I was hoping this wasn't the case, isn't just because it consumes a lot more CPU on the load balancers, but because now we potentially have to manage client certificates and CA certificates (for authenticating from the proxy to back-end app servers). And we also have to decide whether we allow the proxy to use a different client cert / CA per pool, or per member. If you choose to support re-encryption on your service then you are free to charge for the extra CPU cycles. I'm convinced re-encryption and SslTermination is general needs to be mandatory but I think the API should allow them to be specified. Yes, I realize one could potentially use no client cert or CA (ie. encryption but no auth)... but that actually provides almost no extra security over the unencrypted case: If you can sniff the traffic between proxy and back-end server, it's not much more of a stretch to assume you can figure out how to be a man-in-the-middle. Yes but considering you have no problem advocating pure ssl termination for your customers(Decryption on the front end and plain text) on the back end I'm actually surprised this disturbs you. I would recommend users use Straight SSL passthrough or re-enecryption but I wouldn't force this on them should they choose naked encryption with no checking. Do any of you have a use case where some back-end members require SSL authentication from the proxy and some don't? (Again, deciding whether client cert / CA usage should attach to a pool or to a member.) When you say client Cert are you referring to the end users X509 Certificate (To be rejected by the backend server)or are you referring to the back end servers X509Certificate which the loadbalancer would reject if it discovered the back end server had a bad signature or mismatched key? I am speaking of the case where the user wants re-encryption but wants to be able to install CA certificates that sign backend servers Keys via PKIX path building. I would even like to offer the customer the ability to skip hostname validation since not every one wants to expose DNS entries for IPs that are not publicly routable anyways. Unless your suggesting that we should force this on the user which likewise forces us to host a name server that maps hosts to the X509s subject CN fields. Users should be free to validate back end hostnames, just the subject name and key or no validation at all. It should be up to them. It's a bit of a rabbit hole, eh. Stephen On Fri, Apr 18, 2014 at 10:21 AM, Eichberger, German german.eichber...@hp.commailto:german.eichber...@hp.com wrote: Hi Stephen, The use case is that the Load Balancer needs to look at the HTTP requests be it to add an X-Forward field or change the timeout – but the network between the load balancer and the nodes is not completely private and the sensitive information needs to be again transmitted encrypted. This is admittedly an edge case but we had to implement a similar scheme for HP Cloud’s swift storage. German From: Stephen Balukoff [mailto:sbaluk...@bluebox.netmailto:sbaluk...@bluebox.net] Sent: Friday, April 18, 2014 8:22 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807tel:%28800%29613-4305%20x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev
Re: [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question
hostnames, just the subject name and key or no validation at all. It should be up to them. It's a bit of a rabbit hole, eh. Stephen On Fri, Apr 18, 2014 at 10:21 AM, Eichberger, German german.eichber...@hp.com wrote: Hi Stephen, The use case is that the Load Balancer needs to look at the HTTP requests be it to add an X-Forward field or change the timeout – but the network between the load balancer and the nodes is not completely private and the sensitive information needs to be again transmitted encrypted. This is admittedly an edge case but we had to implement a similar scheme for HP Cloud’s swift storage. German *From:* Stephen Balukoff [mailto:sbaluk...@bluebox.net] *Sent:* Friday, April 18, 2014 8:22 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* [openstack-dev] [Neutron][LBaaS] SSL re-encryption scenario question Howdy, folks! Could someone explain to me the SSL usage scenario where it makes sense to re-encrypt traffic traffic destined for members of a back-end pool? SSL termination on the load balancer makes sense to me, but I'm having trouble understanding why one would be concerned about then re-encrypting the traffic headed toward a back-end app server. (Why not just use straight TCP load balancing in this case, and save the CPU cycles on the load balancer?) We terminate a lot of SSL connections on our load balancers, but have yet to have a customer use this kind of functionality. (We've had a few ask about it, usually because they didn't understand what a load balancer is supposed to do-- and with a bit of explanation they went either with SSL termination on the load balancer + clear text on the back-end, or just straight TCP load balancing.) Thanks, Stephen -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
