Hi Mark, thanks for writing up the interesting topic. I agree with Guru that holding packets and waiting for control plane change to get ready doesn't seem to work/scale. I am curious about the problem and I want to make sure I understand it correctly. It sounds like some FaaS scenario, is it? Is it reasonable in such scenario to keep at least one pod for the VIP even if it is idle? If not, is there any example system that actually works well by spinning up pods after requests came?
Regards, Han On Fri, Nov 16, 2018 at 11:28 AM Guru Shetty <[email protected]> wrote: > I don't have specific comments. But I do believe that "holding" packet may > not work well as the time taken to create new pods, assign IPs to those > pod, create new load-balancer rules for the new pods, will be in "seconds". > > > On Fri, 9 Nov 2018 at 12:26, Mark Michelson <[email protected]> wrote: > > > Yesterday during the OVN IRC meeting, I brought up a subject that > > requires a bit more discussion. > > > > The issue: OpenShift has a use case where it uses a load balancer. The > > VIP of the load balancer is backed with k8s pods. During times of > > inactivity, these pods may be deleted due to idleness. Eventually, all > > backing pods may be removed due to idleness. What OpenShift wants is > > when a packet arrives for the load balancer VIP in this scenario, they > > want to be able to react by creating new backing pods, updating load > > balancer configuration, and reprocessing the packet so that it will > > reach a destination pod. > > > > From an OVS/OVN perspective, what needs to happen is that based on an > > incoming packet, an outside party needs to be notified to take an > > action. The interesting part is at what layer the third party is > > notified and how the incoming packet is further handled. We've discussed > > this within Red Hat and have three ideas for how to do this. > > > > 1) The database approach > > We add a new table to the southbound database, perhaps called > > CMS_Events. In the situation where OpenShift wants unidling behavior, it > > sets an option that results in ovn-controller installing a flow in OVS > > to send the incoming packet to ovn-controller. ovn-controller, upon > > receiving the incoming packet, buffers the packet and puts a new row in > > the CMS_Events table. The CMS, upon seeing the event, takes its > > necessary action. In this particular case, the CMS action is to spin up > > pods and alter load balancer configuration. The CMS then sets a value in > > row to indicate it has handled the event. ovn-controller, seeing the > > event is handled, deletes the event and continues processing the packet, > > allowing it to reach its destination. > > > > 2) The message passing approach > > In the situation where OpenShift wants unidling behavior, it sets an > > option that results in ovn-controller installing a flow in OVS to send > > the incoming packet to ovn-controller. ovn-controller, upon receiving > > the packet, buffers the packet and sends some sort of message to the > > CMS. The contents of this message and the method by which the message is > > sent would likely be dependent on the CMS in use. The CMS would act on > > this message, spinning up new pods and altering load balancer > > configuration. The CMS would then send a message back to ovn-controller, > > where ovn-controller would continue processing the packet, allowing it > > to reach its destination. > > > > 3) The second controller approach > > In the situation where OpenShift wants unidling behavior, it sets an > > option that results in ovn-controller installing a flow in OVS to send > > the incoming packet to an external controller. The external controller, > > upon receiving the packet, spins up new pods and alters load balancer > > configuration. The external controller then (somehow) sends the packet > > back into OVS so it can reach its destination. > > > > I presented the database approach during the meeting yesterday, but it's > > worth presenting the other ideas we've discussed. Does any one of these > > three ideas jump out as being significantly better than the others? Do > > you have other ideas for how an external service can be contacted based > > on the reception of a packet destined for a certain IP address? > > > > Thanks, > > Mark Michelson > > _______________________________________________ > > dev mailing list > > [email protected] > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
