I understand the motivation behind ignoring existing stuff, but I have some concerns.
Onboarding existing cloud infrastructure is a valid use case for jclouds users, and that can already be done (limited by the bounds of the Compute abstraction) with the current jclouds implementation. It is fair to expect that if the SecurityGroupExtension is present, and there is a method called "listSecurityGroupsForNode" you should be able to onboard the firewall rules of your existing node. As a user, I'd have these expectations so I don't think we can discard the onboarding use case just because we've found it difficult to map in a consistent way. This said, if we went that path, I think we still need to refine the proposal, or at least be explicit and enumerate all affected points and how we plan to fill the gaps. Say you create a node in jclouds with "options.inboundPorts(22, 80)". With the actual code, the node would be behind a couple firewall rules, but the security group extension won't be able to return any security group for the node. That is inconsistent, as something you created using the abstraction, cannot be retrieved with the same information using the abstraction extension. It will give users the impression that the node has no firewall rules applied. This case could be as simple as creating the firewall rules for the "inboundPorts" following our SG naming and tag convention. I just want to illustrate that the problem is not exclusive of the resources created "outside" jclouds. On 15 June 2017 at 11:09, Svetoslav Neykov <svetoslav.ney...@cloudsoftcorp.com> wrote: >> Svet you don't mention GCE "tags" or the question of subnets in the above: > > That's right, tags are an important part of the convention and I didn't > expand on that. Your description nicely captures the idea. > I believe the security groups will work across subnets in a single vpc > network. > > Svet. > > >> On 15.06.2017 г., at 11:57, Geoff Macartney >> <geoff.macart...@cloudsoftcorp.com> wrote: >> >> I like the sound of this proposal, I think it's certainly worth >> investigating. >> >> Ignasi those are good questions you've asked. Here's my thoughts: >> >> We can regard 'security groups' on GCE as a jclouds "overlay", and be >> explicit about that in documentation. What I mean here is that we don't >> need to do a "reverse mapping" of existing stuff back into jclouds >> concepts. So if there are existing firewall rules they simply don't show up >> in the results of 'listSecurityGroups()", "listSecurityGroupsInLocation()" >> or "listSecurityGroupsForNode()". That is, not all rules participate in >> "security groups" from the jclouds point of view, only ones that jclouds >> added for security group purposes. >> >> Svet you don't mention GCE "tags" or the question of subnets in the above: >> >> 1. What do you think about the idea of using tags to identify security >> groups - each security group would correspond to a tag. These tags would >> correspond to groupIds in the IPPermission object, i.e. you could create an >> IPPermission using a groupId [1] which would relate to a tag in the GCE >> platform. Then any nodes with that tag (= in that security group) would be >> permitted access. >> 2. Are there any restrictions/implications relating to subnets in your >> proposal, e.g. would you need to use them to achieve what you want, or can >> it work irrespective of any subnets on the network? >> >> Geoff >> >> [1] >> https://jclouds.apache.org/reference/javadoc/1.9.x/org/jclouds/net/domain/IpPermission.Builder.html#groupId(java.lang.String) >> >> >> On Thu, 15 Jun 2017 at 09:18 Ignasi Barrera <n...@apache.org> wrote: >> >>> Thanks for starting this discussion Svet! >>> >>> It would be great to come up with a solution we are happy with. >>> >>> The main issue I see here is that the primary entity of the extension, >>> the SecurityGroup, does not have a mapping in GCE (which was one of >>> the reasons we decided to remove the old implementation). >>> >>> I am OK applying conventions to implement the extension somehow, but >>> conventions are tricky when it comes to deal with existing stuff in >>> the provider. With this proposal, what are your plans to implement the >>> "listSecurityGroups()" and "listSecurityGroupsInLocation(Location >>> location)" methods? There is also a method to list the security groups >>> from a node. Given that the ComputeService is able to "listNodes" that >>> weren't created by jclouds, in case of a returned Node that already >>> has firewall rules assigned but that don't fit in our naming >>> conventions, how would the "listSecurityGroupsForNode(String id)" >>> behave? >>> >>> I. >>> >>> On 15 June 2017 at 09:03, Svetoslav Neykov >>> <svetoslav.ney...@cloudsoftcorp.com> wrote: >>>> I'd like to try implement the SecurityGroupExtension interface for GCE. >>> Looking at the documentation it seems that the combination of firewall >>> rules and node tags is flexible enough to allow us implement the >>> functionality. >>>> It's been tried before but the implementation's been removed (see [1]). >>> It's main drawback was that for each security group the code creates a new >>> network. >>>> Currently the biggest mismatch between the jclouds abstraction and the >>> GCE functionality is that firewall rules must be attached to a network. >>>> Here's my suggested approach: >>>> IpPermission roughly corresponds to a firewall rule >>>> SecurityGroup is just a collection of firewall rules (there's no cloud >>> resource that corresponds to it). The firewall rules of a security group >>> share the same prefix - jclouds-sg-<sg name>-<permission suffix>. They all >>> belong to the same network. >>>> They key bit: createSecurityGroup accepts a Location with a scope of >>> Network, returning a custom implementation of SecurityGroup which keeps a >>> reference to the network, so all IpPermission objects added subsequently >>> will be on it. >>>> While the suggested approach fits into the SecurityGroupExtension >>> interface it's different enough from the other implementations that it >>> might not be worth the trouble of implementing and supporting (even be >>> harmful as users might be surprised by the different behaviour). >>>> Would be interested in hearing other opinions on the approach. I've >>> created a Jira to track the effort at [2]. >>>> >>>> Svet. >>>> >>>> >>>> [1] >>> https://github.com/jclouds/jclouds/commit/2ba48dc9f66416b5d8515bd6a07b27a213d89a7c >>> < >>> https://github.com/jclouds/jclouds/commit/2ba48dc9f66416b5d8515bd6a07b27a213d89a7c >>>> >>>> [2] https://issues.apache.org/jira/browse/JCLOUDS-1309 < >>> https://issues.apache.org/jira/browse/JCLOUDS-1309> >>>> >>> >
