Re: [ovs-discuss] [OVN]: IP address representing external destinations

[Han Zhou]

On Fri, Sep 18, 2020 at 12:58 AM Alexander Constantinescu <
acons...@redhat.com> wrote:

> Hi Han
>
> Sorry for the late reply.
>
> Is this the current situation?
>>
>
> Yes, it is.
>
> When you say there are too many default routes, what do you mean in the
>> above example? How would the SOUTH_TO_NORTH_IP solve the problem?
>>
>
> Each  corresponds to a node in our cluster, like this:
>
>   ip4.src ==  && ip4.dst ==
> , allow
>   ip6.src ==  && ip6.dst ==
> , allow
>   ip4.src ==  && ip4.dst ==
> , allow
> ...
>   ip6.src ==  && ip6.dst ==
> , allow
>
> so on large clusters (say 1000 nodes) with IPv6 and IPv4 enabled we can
> reach ~2000 logical router policies. By having the SOUTH_TO_NORTH_IP we can
> completely remove all of them and have the "default route" logical router
> policy specify:
>
> default route (lowest priority): ip4.src == 
> ip4.dst == SOUTH_TO_NORTH_IP, nexthop = 
>
> In addition, if SOUTH_TO_NORTH_IP is a user defined IP,
>>
>
> I didn't think it should be user defined, more so "system defined", like
> 0.0.0.0/0
>
>  I am not sure how would it work,  because ip4.dst is the dst IP from
>> packet header
>>
>
> I didn't intend for such an IP to be used solely as a destination IP, but
> as source too, if the user requires it.
>
> Comparing it with SOUTH_TO_NORTH_IP would just result in mismatch, unless
>> all south-to-north traffic really has this IP as destination (I guess
>> that's not the case).
>>
>
> Sure, I just wanted to assess the feasibility of such an IP from OVN's
> point of view. Obviously the real destination IP would be different, but I
> (without knowing the underlying works of OVN) thought there might be a
> programmable way of saying: "this IP is unknown to my network topology, so
> I could use identifier/alias grouping all such IPs under an umbrella
> identifier such as X.X.X.X/X"
>
>
I see. You don't really want the packet's dst IP to be replaced, so what
you need is some value in metadata and then you could define a policy to
match that metadata and specify the nexthop. This is supported by the
policy route table.

However, what really matters here is how to define "unknown IPs". We can't
say an IP is unknown just because there is no port with the IP assigned.
There can still be east-west traffic for such IP, e.g. for nested workload
in a VM, etc. From OVN's point of view this is completely user defined, and
the best way for the user to tell OVN this information is through the route
policies (which means you don't really need that extra metadata to achieve
this). I think it is the way how k8s-ovn is using OVN that causes this many
routes required. If the IP allocation can be better managed so that a big
range of IP is for east-west and otherwise north-south, then you probably
end up with a much smaller number of policies.


>
> On Wed, Sep 16, 2020 at 11:09 PM Han Zhou  wrote:
>
>>
>>
>> On Wed, Sep 16, 2020 at 10:07 AM Alexander Constantinescu <
>> acons...@redhat.com> wrote:
>>
>>> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.

>>>
>>> Yes, I was just using it as an example (though, granted, noop example)
>>>
>>> Some background to help steer the discussion:
>>>
>>> Essentially the functionality here is to have south -> north traffic
>>> from certain logical switch ports exit the cluster through a dedicated node
>>> (an egress node if you will). To do this we currently have a set of default
>>> logical router policies, intended to leave east <-> west traffic untouched,
>>> and then logical router policies with a lower priority, which specify
>>> reroute actions for this functionality to happen. However, on large
>>> clusters, there's this concern that the default logical router policies
>>> will become too many. Hence why the idea here would be to drop them
>>> completely and have this "special IP" that we can use to filter on the
>>> destination, south -> north, traffic .
>>>
>>> If you have a default route, anything "unknown" would just hit the
 default route, right? Why would you need another IP for this purpose?

>>>
>>> As to remove the default logical router policies, which can become a
>>> lot, on big clusters - as described above. With only reroute policies of
>>> type: "ip4.src == 10.244.2.5/32 && ip4.dst == SOUTH_TO_NORTH_IP" things
>>> would become lighter.
>>>
>>
>> Thanks for the background. So you have:
>> 
>> 
>> ...
>> default route (lowest priority): ip4.src == ,
>> nexthop = 
>> default route (lowest priority): ip4.src == ,
>> nexthop = 
>>
>> Is this the current situation?
>> When you say there are too many default routes, what do you mean in the
>> above example? How would the SOUTH_TO_NORTH_IP solve the problem?
>>
>> In addition, if SOUTH_TO_NORTH_IP is a user defined IP, I am not sure how
>> would it work, because ip4.dst is the dst IP from packet header. Comparing
>> it with SOUTH_TO_NORTH_IP would just result in mismatch, unless all
>> south-to-north traffic really has this IP as destination (I guess that's

Re: [ovs-discuss] [OVN]: IP address representing external destinations

[Alexander Constantinescu]

Hi Han

Sorry for the late reply.

Is this the current situation?
>

Yes, it is.

When you say there are too many default routes, what do you mean in the
> above example? How would the SOUTH_TO_NORTH_IP solve the problem?
>

Each  corresponds to a node in our cluster, like this:

  ip4.src ==  && ip4.dst ==
, allow
  ip6.src ==  && ip6.dst ==
, allow
  ip4.src ==  && ip4.dst ==
, allow
...
  ip6.src ==  && ip6.dst ==
, allow

so on large clusters (say 1000 nodes) with IPv6 and IPv4 enabled we can
reach ~2000 logical router policies. By having the SOUTH_TO_NORTH_IP we can
completely remove all of them and have the "default route" logical router
policy specify:

default route (lowest priority): ip4.src == 
ip4.dst == SOUTH_TO_NORTH_IP, nexthop = 

In addition, if SOUTH_TO_NORTH_IP is a user defined IP,
>

I didn't think it should be user defined, more so "system defined", like
0.0.0.0/0

 I am not sure how would it work,  because ip4.dst is the dst IP from
> packet header
>

I didn't intend for such an IP to be used solely as a destination IP, but
as source too, if the user requires it.

Comparing it with SOUTH_TO_NORTH_IP would just result in mismatch, unless
> all south-to-north traffic really has this IP as destination (I guess
> that's not the case).
>

Sure, I just wanted to assess the feasibility of such an IP from OVN's
point of view. Obviously the real destination IP would be different, but I
(without knowing the underlying works of OVN) thought there might be a
programmable way of saying: "this IP is unknown to my network topology, so
I could use identifier/alias grouping all such IPs under an umbrella
identifier such as X.X.X.X/X"


On Wed, Sep 16, 2020 at 11:09 PM Han Zhou  wrote:

>
>
> On Wed, Sep 16, 2020 at 10:07 AM Alexander Constantinescu <
> acons...@redhat.com> wrote:
>
>> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.
>>>
>>
>> Yes, I was just using it as an example (though, granted, noop example)
>>
>> Some background to help steer the discussion:
>>
>> Essentially the functionality here is to have south -> north traffic from
>> certain logical switch ports exit the cluster through a dedicated node (an
>> egress node if you will). To do this we currently have a set of default
>> logical router policies, intended to leave east <-> west traffic untouched,
>> and then logical router policies with a lower priority, which specify
>> reroute actions for this functionality to happen. However, on large
>> clusters, there's this concern that the default logical router policies
>> will become too many. Hence why the idea here would be to drop them
>> completely and have this "special IP" that we can use to filter on the
>> destination, south -> north, traffic .
>>
>> If you have a default route, anything "unknown" would just hit the
>>> default route, right? Why would you need another IP for this purpose?
>>>
>>
>> As to remove the default logical router policies, which can become a
>> lot, on big clusters - as described above. With only reroute policies of
>> type: "ip4.src == 10.244.2.5/32 && ip4.dst == SOUTH_TO_NORTH_IP" things
>> would become lighter.
>>
>
> Thanks for the background. So you have:
> 
> 
> ...
> default route (lowest priority): ip4.src == ,
> nexthop = 
> default route (lowest priority): ip4.src == ,
> nexthop = 
>
> Is this the current situation?
> When you say there are too many default routes, what do you mean in the
> above example? How would the SOUTH_TO_NORTH_IP solve the problem?
>
> In addition, if SOUTH_TO_NORTH_IP is a user defined IP, I am not sure how
> would it work, because ip4.dst is the dst IP from packet header. Comparing
> it with SOUTH_TO_NORTH_IP would just result in mismatch, unless all
> south-to-north traffic really has this IP as destination (I guess that's
> not the case).
>
>
>>  In policies/ACL you will need to make sure the priorities are set
>>> properly to achieve the default-route behavior.
>>>
>>
>> Yes, so this is currently done, as described above.
>>
>> On Wed, Sep 16, 2020 at 6:35 PM Han Zhou  wrote:
>>
>>>
>>>
>>> On Wed, Sep 16, 2020 at 5:42 AM Alexander Constantinescu <
>>> acons...@redhat.com> wrote:
>>> >
>>> > Hi
>>> >
>>> > I was wondering if anybody is aware of an IP address signifying
>>> "external IP destinations"?
>>> >
>>> > Currently in OVN we can use the IP address 0.0.0.0/0 for match
>>> expressions in logical routing policies / ACLs when we want to specify a
>>> source or destination IP equating to the pseudo term: "all IP
>>> addresses",ex: 'match="ip4.src == 10.244.2.5/32 && ip4.dst ==0.0.0.0/0"'
>>> >
>>> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.
>>>
>>> > Essentially what I would need to do for an OVN-Kubernetes feature is
>>> specify such a match condition for south -> north traffic, i.e when the
>>> destination IP address is external to the cluster, and most likely
>>> "unknown" to OVN. Thus, when OVN does not know how to route it within the
>>> OVN network topology and 

Re: [ovs-discuss] [OVN]: IP address representing external destinations

[Han Zhou]

On Wed, Sep 16, 2020 at 10:07 AM Alexander Constantinescu <
acons...@redhat.com> wrote:

> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.
>>
>
> Yes, I was just using it as an example (though, granted, noop example)
>
> Some background to help steer the discussion:
>
> Essentially the functionality here is to have south -> north traffic from
> certain logical switch ports exit the cluster through a dedicated node (an
> egress node if you will). To do this we currently have a set of default
> logical router policies, intended to leave east <-> west traffic untouched,
> and then logical router policies with a lower priority, which specify
> reroute actions for this functionality to happen. However, on large
> clusters, there's this concern that the default logical router policies
> will become too many. Hence why the idea here would be to drop them
> completely and have this "special IP" that we can use to filter on the
> destination, south -> north, traffic .
>
> If you have a default route, anything "unknown" would just hit the default
>> route, right? Why would you need another IP for this purpose?
>>
>
> As to remove the default logical router policies, which can become a
> lot, on big clusters - as described above. With only reroute policies of
> type: "ip4.src == 10.244.2.5/32 && ip4.dst == SOUTH_TO_NORTH_IP" things
> would become lighter.
>

Thanks for the background. So you have:


...
default route (lowest priority): ip4.src == ,
nexthop = 
default route (lowest priority): ip4.src == ,
nexthop = 

Is this the current situation?
When you say there are too many default routes, what do you mean in the
above example? How would the SOUTH_TO_NORTH_IP solve the problem?

In addition, if SOUTH_TO_NORTH_IP is a user defined IP, I am not sure how
would it work, because ip4.dst is the dst IP from packet header. Comparing
it with SOUTH_TO_NORTH_IP would just result in mismatch, unless all
south-to-north traffic really has this IP as destination (I guess that's
not the case).


>  In policies/ACL you will need to make sure the priorities are set
>> properly to achieve the default-route behavior.
>>
>
> Yes, so this is currently done, as described above.
>
> On Wed, Sep 16, 2020 at 6:35 PM Han Zhou  wrote:
>
>>
>>
>> On Wed, Sep 16, 2020 at 5:42 AM Alexander Constantinescu <
>> acons...@redhat.com> wrote:
>> >
>> > Hi
>> >
>> > I was wondering if anybody is aware of an IP address signifying
>> "external IP destinations"?
>> >
>> > Currently in OVN we can use the IP address 0.0.0.0/0 for match
>> expressions in logical routing policies / ACLs when we want to specify a
>> source or destination IP equating to the pseudo term: "all IP
>> addresses",ex: 'match="ip4.src == 10.244.2.5/32 && ip4.dst ==0.0.0.0/0"'
>> >
>> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.
>>
>> > Essentially what I would need to do for an OVN-Kubernetes feature is
>> specify such a match condition for south -> north traffic, i.e when the
>> destination IP address is external to the cluster, and most likely
>> "unknown" to OVN. Thus, when OVN does not know how to route it within the
>> OVN network topology and has no choice except sending it out the default
>> route.
>> >
>> > Do we have such an IP address in OVN/OVS? Would it be feasible to
>> introduce, in case there is none?
>> >
>> We don't have such a special IP except 0.0.0.0/0. If you have a default
>> route, anything "unknown" would just hit the default route, right? Why
>> would you need another IP for this purpose? In logical_router_static_route
>> the priority is based on prefix length. In policies/ACL you will need to
>> make sure the priorities are set properly to achieve the default-route
>> behavior.
>>
>> Thanks,
>> Han
>>
>> > Thanks in advance!
>> >
>> > --
>> >
>> > Best regards,
>> >
>> >
>> > Alexander Constantinescu
>> >
>> > Software Engineer, Openshift SDN
>> >
>> > Red Hat
>> >
>> > acons...@redhat.com
>> >
>> > ___
>> > discuss mailing list
>> > disc...@openvswitch.org
>> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>>
>
>
> --
>
> Best regards,
>
>
> Alexander Constantinescu
>
> Software Engineer, Openshift SDN
>
> Red Hat 
>
> acons...@redhat.com
> 
>
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] [OVN]: IP address representing external destinations

[Alexander Constantinescu]

>
> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.
>

Yes, I was just using it as an example (though, granted, noop example)

Some background to help steer the discussion:

Essentially the functionality here is to have south -> north traffic from
certain logical switch ports exit the cluster through a dedicated node (an
egress node if you will). To do this we currently have a set of default
logical router policies, intended to leave east <-> west traffic untouched,
and then logical router policies with a lower priority, which specify
reroute actions for this functionality to happen. However, on large
clusters, there's this concern that the default logical router policies
will become too many. Hence why the idea here would be to drop them
completely and have this "special IP" that we can use to filter on the
destination, south -> north, traffic .

If you have a default route, anything "unknown" would just hit the default
> route, right? Why would you need another IP for this purpose?
>

As to remove the default logical router policies, which can become a
lot, on big clusters - as described above. With only reroute policies of
type: "ip4.src == 10.244.2.5/32 && ip4.dst == SOUTH_TO_NORTH_IP" things
would become lighter.

 In policies/ACL you will need to make sure the priorities are set properly
> to achieve the default-route behavior.
>

Yes, so this is currently done, as described above.

On Wed, Sep 16, 2020 at 6:35 PM Han Zhou  wrote:

>
>
> On Wed, Sep 16, 2020 at 5:42 AM Alexander Constantinescu <
> acons...@redhat.com> wrote:
> >
> > Hi
> >
> > I was wondering if anybody is aware of an IP address signifying
> "external IP destinations"?
> >
> > Currently in OVN we can use the IP address 0.0.0.0/0 for match
> expressions in logical routing policies / ACLs when we want to specify a
> source or destination IP equating to the pseudo term: "all IP
> addresses",ex: 'match="ip4.src == 10.244.2.5/32 && ip4.dst ==0.0.0.0/0"'
> >
> In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.
>
> > Essentially what I would need to do for an OVN-Kubernetes feature is
> specify such a match condition for south -> north traffic, i.e when the
> destination IP address is external to the cluster, and most likely
> "unknown" to OVN. Thus, when OVN does not know how to route it within the
> OVN network topology and has no choice except sending it out the default
> route.
> >
> > Do we have such an IP address in OVN/OVS? Would it be feasible to
> introduce, in case there is none?
> >
> We don't have such a special IP except 0.0.0.0/0. If you have a default
> route, anything "unknown" would just hit the default route, right? Why
> would you need another IP for this purpose? In logical_router_static_route
> the priority is based on prefix length. In policies/ACL you will need to
> make sure the priorities are set properly to achieve the default-route
> behavior.
>
> Thanks,
> Han
>
> > Thanks in advance!
> >
> > --
> >
> > Best regards,
> >
> >
> > Alexander Constantinescu
> >
> > Software Engineer, Openshift SDN
> >
> > Red Hat
> >
> > acons...@redhat.com
> >
> > ___
> > discuss mailing list
> > disc...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>


-- 

Best regards,


Alexander Constantinescu

Software Engineer, Openshift SDN

Red Hat 

acons...@redhat.com

___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] [OVN]: IP address representing external destinations

[Han Zhou]

On Wed, Sep 16, 2020 at 5:42 AM Alexander Constantinescu <
acons...@redhat.com> wrote:
>
> Hi
>
> I was wondering if anybody is aware of an IP address signifying "external
IP destinations"?
>
> Currently in OVN we can use the IP address 0.0.0.0/0 for match
expressions in logical routing policies / ACLs when we want to specify a
source or destination IP equating to the pseudo term: "all IP
addresses",ex: 'match="ip4.src == 10.244.2.5/32 && ip4.dst ==0.0.0.0/0"'
>
In this example it is equivalent to just "ip4.src == 10.244.2.5/32"'.

> Essentially what I would need to do for an OVN-Kubernetes feature is
specify such a match condition for south -> north traffic, i.e when the
destination IP address is external to the cluster, and most likely
"unknown" to OVN. Thus, when OVN does not know how to route it within the
OVN network topology and has no choice except sending it out the default
route.
>
> Do we have such an IP address in OVN/OVS? Would it be feasible to
introduce, in case there is none?
>
We don't have such a special IP except 0.0.0.0/0. If you have a default
route, anything "unknown" would just hit the default route, right? Why
would you need another IP for this purpose? In logical_router_static_route
the priority is based on prefix length. In policies/ACL you will need to
make sure the priorities are set properly to achieve the default-route
behavior.

Thanks,
Han

> Thanks in advance!
>
> --
>
> Best regards,
>
>
> Alexander Constantinescu
>
> Software Engineer, Openshift SDN
>
> Red Hat
>
> acons...@redhat.com
>
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss