Re: [ovs-dev] one issue in vxlan functionality of the kernel-datapath type of ovs

[pei Jikui]

In my test bed there is no any dropping flow entry generated in the datapath, 
which causes all the unwanted vxlan packets will go to the slow path.  That’s a 
little time-consuming for this case.



发送自 Outlook


发件人: Ben Pfaff 
发送时间: 2019年5月25日 2:36
收件人: pei Jikui
抄送: ovs-dev@openvswitch.org; ovs-disc...@openvswitch.org
主题: Re: [ovs-dev] one issue in vxlan functionality of the kernel-datapath type 
of ovs

On Sat, May 18, 2019 at 10:23:50AM +, pei Jikui wrote:
> I found one issue in the vxlan functionality in kernel-datapath type of ovs 
> which could be potentially optimize?
>
>
> For example, there is a machine (A) running ovs with one configured one vxlan 
> interface with tunnel value 123,  then all the vxlan traffics destinate to 
> this machine(A) will be dealt with the ovs.
>
>
> Although the ovs in machine A only configured with one vxlan tunnel (123), 
> all the vxlan traffics regardless the tunnel value is 123 or not, will be 
> delivered to the vswitchd to do the slow path if there is no flow tables 
> found in the datapath.
>
> This is due to the vxlan configuration such as the configured vxlan tunnel 
> valuse does not exist in the datapath. (There is only one vxlan tunnel with 
> vni value 0 exist in the datapath’s vxlan configuration).

It's true, but does it matter?  It would be unusual for a host to
receive much VXLAN traffic that it is only going to drop.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] ovn-controller: Fix parsing of OVN tunnel IDs

[0-day Robot]

Bleep bloop.  Greetings Dumitru Ceara, I am a robot and I have tried out your 
patch.
Thanks for your contribution.

I encountered some error that I wasn't expecting.  See the details below.


git-am:
fatal: sha1 information is lacking or useless (ovn/controller/pinctrl.c).
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.
Patch failed at 0001 ovn-controller: Fix parsing of OVN tunnel IDs
The copy of the patch that failed is found in:
   
/var/lib/jenkins/jobs/upstream_build_from_pw/workspace/OVN/.git/rebase-apply/patch
When you have resolved this problem, run "git am --resolved".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".


Please check this out.  If you feel there has been an error, please email 
acon...@bytheb.org

Thanks,
0-day Robot
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovn-controller: Fix parsing of OVN tunnel IDs

[Dumitru Ceara]

On Thu, May 23, 2019 at 11:56 PM Ben Pfaff  wrote:
>
> On Mon, May 13, 2019 at 01:30:59PM +0200, Dumitru Ceara wrote:
> > Add tunnel-id creation and parsing functions in encaps.h.
> >
> > Reported-at: https://bugzilla.redhat.com/1708131
> > Reported-by: Haidong Li 
> > Fixes: b520ca7 ("Support for multiple VTEP in OVN")
> > Signed-off-by: Dumitru Ceara 
>
> Please add a description of the problem that this fixes to the commit
> message.  I know that the bug report probably describes it, but it's
> best if the commit log makes sense by itself.
>

Sorry, my bad. I sent a v2 patch now with an updated commit message
that supplies all the info.

Thanks,
Dumitru

> Thanks,
>
> Ben.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2] ovn-controller: Fix parsing of OVN tunnel IDs

[Dumitru Ceara]

Encap tunnel-ids are of the form:
.
In physical_run we were checking if a tunnel-id corresponds
to the local chassis-id by searching if the chassis-id string
is included in the tunnel-id (strstr). This can break quite
easily, for example, if the local chassis-id is a substring
of a remote chassis-id. In that case we were wrongfully
skipping the tunnel creation.

To fix that new tunnel-id creation and parsing functions are added in
encaps.[ch]. These functions are now used everywhere where applicable.

Acked-by: Venu Iyer 
Reported-at: https://bugzilla.redhat.com/1708131
Reported-by: Haidong Li 
Fixes: b520ca7 ("Support for multiple VTEP in OVN")
Signed-off-by: Dumitru Ceara 
---

v2: Update commit message
---
 ovn/controller/bfd.c| 31 ---
 ovn/controller/encaps.c | 87 -
 ovn/controller/encaps.h |  6 +++
 ovn/controller/ovn-controller.h |  7 
 ovn/controller/physical.c   | 51 +---
 ovn/controller/pinctrl.c|  4 +-
 6 files changed, 130 insertions(+), 56 deletions(-)

diff --git a/ovn/controller/bfd.c b/ovn/controller/bfd.c
index d016e27..b6aef04 100644
--- a/ovn/controller/bfd.c
+++ b/ovn/controller/bfd.c
@@ -15,6 +15,7 @@
 
 #include 
 #include "bfd.h"
+#include "encaps.h"
 #include "lport.h"
 #include "ovn-controller.h"
 
@@ -72,14 +73,16 @@ bfd_calculate_active_tunnels(const struct ovsrec_bridge 
*br_int,
 const char *id = smap_get(_rec->external_ids,
   "ovn-chassis-id");
 if (id) {
-char *chassis_name;
-char *save_ptr = NULL;
-char *tokstr = xstrdup(id);
-chassis_name = strtok_r(tokstr, 
OVN_MVTEP_CHASSISID_DELIM, _ptr);
-if (chassis_name && !sset_contains(active_tunnels, 
chassis_name)) {
-sset_add(active_tunnels, chassis_name);
+char *chassis_name = NULL;
+
+if (encaps_tunnel_id_parse(id, _name,
+   NULL)) {
+if (!sset_contains(active_tunnels,
+   chassis_name)) {
+sset_add(active_tunnels, chassis_name);
+}
+free(chassis_name);
 }
-free(tokstr);
 }
 }
 }
@@ -193,17 +196,17 @@ bfd_run(const struct ovsrec_interface_table 
*interface_table,
 const char *tunnel_id = smap_get(_int->ports[k]->external_ids,
   "ovn-chassis-id");
 if (tunnel_id) {
-char *chassis_name;
-char *save_ptr = NULL;
-char *tokstr = xstrdup(tunnel_id);
+char *chassis_name = NULL;
 char *port_name = br_int->ports[k]->name;
 
 sset_add(, port_name);
-chassis_name = strtok_r(tokstr, OVN_MVTEP_CHASSISID_DELIM, 
_ptr);
-if (chassis_name && sset_contains(_chassis, chassis_name)) {
-sset_add(_ifaces, port_name);
+
+if (encaps_tunnel_id_parse(tunnel_id, _name, NULL)) {
+if (sset_contains(_chassis, chassis_name)) {
+sset_add(_ifaces, port_name);
+}
+free(chassis_name);
 }
-free(tokstr);
 }
 }
 
diff --git a/ovn/controller/encaps.c b/ovn/controller/encaps.c
index dcf7810..d467540 100644
--- a/ovn/controller/encaps.c
+++ b/ovn/controller/encaps.c
@@ -26,6 +26,13 @@
 
 VLOG_DEFINE_THIS_MODULE(encaps);
 
+/*
+ * Given there could be multiple tunnels with different IPs to the same
+ * chassis we annotate the ovn-chassis-id with
+ * OVN_MVTEP_CHASSISID_DELIM.
+ */
+#defineOVN_MVTEP_CHASSISID_DELIM '@'
+
 void
 encaps_register_ovs_idl(struct ovsdb_idl *ovs_idl)
 {
@@ -78,6 +85,83 @@ tunnel_create_name(struct tunnel_ctx *tc, const char 
*chassis_id)
 return NULL;
 }
 
+/*
+ * Returns a tunnel-id of the form 'chassis_id'-delimiter-'encap_ip'.
+ */
+char *
+encaps_tunnel_id_create(const char *chassis_id, const char *encap_ip)
+{
+return xasprintf("%s%c%s", chassis_id, OVN_MVTEP_CHASSISID_DELIM,
+ encap_ip);
+}
+
+/*
+ * Parses a 'tunnel_id' of the form .
+ * If the 'chassis_id' argument is not NULL the function will allocate memory
+ * and store the chassis-id part of the tunnel-id at '*chassis_id'.
+ * If the 'encap_ip' argument is not NULL the function will allocate memory
+ * and store the encapsulation IP part of the tunnel-id at '*encap_ip'.
+ */
+bool encaps_tunnel_id_parse(const char *tunnel_id, char **chassis_id,
+char **encap_ip)

Re: [ovs-dev] [PATCH] ovn: Properly set the index for chassis lookup

[Dumitru Ceara]

On Thu, May 23, 2019 at 10:58 PM Ben Pfaff  wrote:
>
> On Thu, May 23, 2019 at 01:11:46PM -0700, Han Zhou wrote:
> > On Thu, May 9, 2019 at 1:09 AM Dumitru Ceara  wrote:
> > >
> > > The chassis_lookup_by_name function now calls
> > > sbrec_chassis_index_set_name instead of sbrec_chassis_set_name. Due to
> > > the use of the wrong API memory was leaked every time a chassis was
> > > looked up by name. This was mostly visible when chassis lookups had to
> > > be done continuously (e.g., when two chassis were misconfigured
> > > with the same system-id).
> > >
> > > Reported-at: https://bugzilla.redhat.com/1698462
> > > Reported-by: Alexander 
> > > Signed-off-by: Dumitru Ceara 
> > > ---
> > >  ovn/lib/chassis-index.c | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/ovn/lib/chassis-index.c b/ovn/lib/chassis-index.c
> > > index 34d4a31..423 100644
> > > --- a/ovn/lib/chassis-index.c
> > > +++ b/ovn/lib/chassis-index.c
> > > @@ -30,7 +30,7 @@ chassis_lookup_by_name(struct ovsdb_idl_index
> > *sbrec_chassis_by_name,
> > >  {
> > >  struct sbrec_chassis *target = sbrec_chassis_index_init_row(
> > >  sbrec_chassis_by_name);
> > > -sbrec_chassis_set_name(target, name);
> > > +sbrec_chassis_index_set_name(target, name);
> > >
> > >  struct sbrec_chassis *retval = sbrec_chassis_index_find(
> > >  sbrec_chassis_by_name, target);
> > > --
> > > 1.8.3.1
> > >
> > > ___
> > > dev mailing list
> > > d...@openvswitch.org
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> >
> > Thanks Dumitru for fixing this. I see same wrong way of using index in the
> > same file:
> >
> > struct sbrec_ha_chassis_group *target =
> > sbrec_ha_chassis_group_index_init_row(sbrec_ha_chassis_grp_by_name);
> > sbrec_ha_chassis_group_set_name(target, name);
> >
> > May it be fixed together?
> > I did check all the other files under ovn, and it seems these are the only
> > places with this problem.
> >
> > Acked-by: Han Zhou 
>
> I folded in the additional fix and applied this to master.  Thank you!

Thanks Han for the additional fix (I completely missed it) and thanks
Ben for applying it!

Cheers,
Dumitru

>
> Here is the final version:
>
> --8<--cut here-->8--
>
> From: Dumitru Ceara 
> Date: Thu, 9 May 2019 10:09:23 +0200
> Subject: [PATCH] ovn: Properly set the index for chassis lookup
>
> The chassis_lookup_by_name function now calls
> sbrec_chassis_index_set_name instead of sbrec_chassis_set_name. Due to
> the use of the wrong API memory was leaked every time a chassis was
> looked up by name. This was mostly visible when chassis lookups had to
> be done continuously (e.g., when two chassis were misconfigured
> with the same system-id).
>
> Acked-by: Han Zhou 
> Reported-at: https://bugzilla.redhat.com/1698462
> Reported-by: Alexander 
> Signed-off-by: Dumitru Ceara 
> Signed-off-by: Ben Pfaff 
> ---
>  ovn/lib/chassis-index.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/ovn/lib/chassis-index.c b/ovn/lib/chassis-index.c
> index 34d4a31eb339..10f70fb4a18f 100644
> --- a/ovn/lib/chassis-index.c
> +++ b/ovn/lib/chassis-index.c
> @@ -30,7 +30,7 @@ chassis_lookup_by_name(struct ovsdb_idl_index 
> *sbrec_chassis_by_name,
>  {
>  struct sbrec_chassis *target = sbrec_chassis_index_init_row(
>  sbrec_chassis_by_name);
> -sbrec_chassis_set_name(target, name);
> +sbrec_chassis_index_set_name(target, name);
>
>  struct sbrec_chassis *retval = sbrec_chassis_index_find(
>  sbrec_chassis_by_name, target);
> @@ -55,7 +55,7 @@ ha_chassis_group_lookup_by_name(
>  {
>  struct sbrec_ha_chassis_group *target =
>  sbrec_ha_chassis_group_index_init_row(sbrec_ha_chassis_grp_by_name);
> -sbrec_ha_chassis_group_set_name(target, name);
> +sbrec_ha_chassis_group_index_set_name(target, name);
>
>  struct sbrec_ha_chassis_group *retval =
>  sbrec_ha_chassis_group_index_find(sbrec_ha_chassis_grp_by_name,
> --
> 2.20.1
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovn: Add a new logical switch port type - 'virtual'

[Ben Pfaff]

This commit adds new columns virtual_ip and virtual_parents in the
Logical_Switch_Port table, which are used for the new 'virtual' port
type.  Most of the time, if a particular port type has type-specific
options, they go in the "options" column.  This has the advantage that
we don't end up with lots of columns that are rarely used.  It does have
some disadvantages.  For example, "options" can't be strong or weak
references, and it requires some convention for having more than one
value.  virtual_ip and virtual_parents don't have the former issue,
though virtual_parents does have the latter issue.  Did you think
through whether these should be options or new columns?
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC V2] netdev-rte-offloads: HW offload virtio-forwarder

[Roni Bar Yanai]

Hi Ilya,
(S

See inline.

>-Original Message-
>From: Ilya Maximets 
>Sent: Friday, May 24, 2019 3:21 PM
>To: Simon Horman ; Roni Bar Yanai
>
>Cc: ovs-dev@openvswitch.org; Ian Stokes ; Kevin Traynor
>; Oz Shlomo ; Eli Britstein
>; Eyal Lavee ; Rony Efraim
>; Ben Pfaff 
>Subject: Re: [ovs-dev] [RFC V2] netdev-rte-offloads: HW offload 
>virtio-forwarder
>
>On 22.05.2019 15:10, Simon Horman wrote:
>> Hi,
>>
>> On Thu, May 16, 2019 at 08:44:31AM +, Roni Bar Yanai wrote:
 -Original Message-
 From: Ilya Maximets 
 Sent: Wednesday, May 15, 2019 4:37 PM
 To: Roni Bar Yanai ; ovs-dev@openvswitch.org; Ian
 Stokes ; Kevin Traynor 
 Cc: Eyal Lavee ; Oz Shlomo ;
>Eli
 Britstein ; Rony Efraim ; Asaf
 Penso 
 Subject: Re: [RFC V2] netdev-rte-offloads: HW offload virtio-forwarder

 On 15.05.2019 16:01, Roni Bar Yanai wrote:
> Hi Ilya,
>
> Thanks for the comment.
>
> I think the suggested arch is very good and has many advantages, and
> in fact I had something very similar as my internally first approach.
>
> However, I had one problem: it doesn't solves the kernel case. It make
> sense doing forwarding using dpdk also when OVS is kernel (port
> representor and rule offloads are done with kernel OVS). It makes
> sense because we can have one solution and because DPDK has better
> performance.

 I'm not sure if it makes practical sense to run separate userpace
 datapath just to pass packets between vhost and VF. This actually
 matches with some of your own disadvantages of separate DPDK apps.
 Separate userspace datapath will need its own complex start,
 configuration and maintenance. Also it will consume additional cpu cores
 which will not be shared with kernel packet processing.  I think that
 just move everything to userspace in this case would be much more simple
 for user than maintaining such configuration.
>>>
>>> Maybe It doesn't make sense for OVS-DPDK but for OVS users it does.  When
>>> you run offload with OVS-kernel, and for some vendors this is the current
>>> status, and virtio is a requirement, you now have millions of packets
>>> that should be forwarded. Basically you have two options:
>>>
>>> 1. use external application (we discussed that).
>>>
>>> 2. create user space data plane and configure forwarding (OVS), but then
>>> you have performance issues as OVS is not optimized for this. And for
>>> kernel data plane much worse off course.
>>>
>>> Regarding burning a core. In case of HW offload you will do it either
>>> way, and there is no benefit for adding FW functionality for kernel data
>>> path, mainly because of kernel performance limitations.
>>>
>>> I agree that in such case moving to user space is a solution for some,
>>> but keep in mind that some doesn't have such support for DPDK and for
>>> others they have their own OVS based data path with their adjustments, so
>>> it will be a hard transition.
>>>
>>> While arch is good for the two DPDK use cases, it leaves the kernel one
>>> out.  Any thoughts how we can add this use case as well and still keep
>>> the suggested arch?
>>
>> ...
>>
>> At Netronome we have an Open Source standalone application,
>> called virtio-forwarder
>(https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.co
>m%2FNetronome%2Fvirtio-
>forwarderdata=02%7C01%7Croniba%40mellanox.com%7C72d768141c794f4
>59eee08d6e0424d0a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C6369
>42972699205388sdata=Nk1a9tRUD4%2BjDFOh%2BUwrdnQ6sJ2cnhXfvJsXJs
>F0X3E%3Dreserved=0).
>> The reason that we provide this solution is that we see this as a
>> requirement for some customers. This includes customers using OVS
>> with the kernel based HW offload (OVS-TC).
>>
>> In general I agree that integration with OVS has some advantages and
>> I'm happy to see this issue being discussed. But as we see demand
>> for use of virtio-forwarder in conjunction with OVS-TC I see that
>> as a requirement for a solution that is integrated with OVS, which leads
>> me to lean towards the proposal put forward by Roni.
>>
>> I also feel that the proposal put forward by Roni is likely to prove more
>> flexible that a port-based approach, as proposed by Ilya. For one thing
>> such a design ought to allow for arbitrary combinations of port types.
>> In fact, it would be entirely feasible to run this in conjunction with a
>> non-OVS offload aware NIC (SR-IOV in VEB mode).
>>
>> Returning to the stand-alone Netronome implementation, I would welcome
>> discussion of how any useful portions of this could be reused.
>>
>
>Hi Simon. Thanks for link. It's very interesting.
>
>My key point about the proposal put forward by Roni is that Open vSwitch
>is an *OF switch* first of all and *not the multitool*. This proposal adds

Let's not forget performance is a major factor for a switch and currently there 
is a gap between the market demand + hw capabilities and 

Re: [ovs-dev] [PATCHv9] netdev-afxdp: add new netdev type for AF_XDP.

[William Tu]

On Fri, May 24, 2019 at 11:32 AM Ben Pfaff  wrote:
>
> On Fri, May 24, 2019 at 11:03:33AM -0700, William Tu wrote:
> > The patch introduces experimental AF_XDP support for OVS netdev.
> > AF_XDP, the Address Family of the eXpress Data Path, is a new Linux socket
> > type built upon the eBPF and XDP technology.  It is aims to have comparable
> > performance to DPDK but cooperate better with existing kernel's networking
> > stack.  An AF_XDP socket receives and sends packets from an eBPF/XDP program
> > attached to the netdev, by-passing a couple of Linux kernel's subsystems
> > As a result, AF_XDP socket shows much better performance than AF_PACKET
> > For more details about AF_XDP, please see linux kernel's
> > Documentation/networking/af_xdp.rst. Note that by default, this feature is
> > not compiled in.
> >
> > Signed-off-by: William Tu 
>
> How heavy is the x86(_64)-only dependency?  It seems undesirable. is

Now in my patch, the cycles_counter_update has x86-specific instructions.
Other part of the code has no x86-only dependency.

The reason I made this x86-only is that AF_XDP is rarely tested on
non-x86 system. So I'm not sure whether it works or not.

Regards,
William
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v3] datapath: Fix build errors for 4.9.172+ kernels

[Yi-Hung Wei]

On Fri, May 24, 2019 at 11:24 AM Yifeng Sun  wrote:
>
> 4.9.172+ kernel backported upstream patch 70b095c843266
> ("ipv6: remove dependency of nf_defrag_ipv6 on ipv6 module")
> and this caused compilation errors of OVS kernel module.
>
> This patch fixes it by checking and using new functions
> introduced by the upstream patch.
>
> Travis tests passed at
> https://travis-ci.org/yifsun/ovs-travis/builds/536527230
> with latest Linux kernel version.
>
> In addition, this patch doesn't introduce failed tests for latest kernels
> of Ubuntu (bionic, trusty, xenial), fedora, centos 73, rhel (74, 75, 76).
>
> Reported-by: Ilya Maximets 
> Signed-off-by: Yifeng Sun 
> ---
Thanks for the update. It looks good to me.

Acked-by: Yi-Hung Wei 
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovn-ctl: Set max open files limit for SB DB.

[Ben Pfaff]

On Tue, May 14, 2019 at 03:15:29PM -0700, Han Zhou wrote:
> From: Han Zhou 
> 
> Increase the limit on the number of open file descriptors, because
> SB DB may connect to large number of chassises.
> 
> Signed-off-by: Han Zhou 

Applied, thanks!
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] OVN: Fix the ovn-controller 100% usage issue with put_mac_bindings

[Ben Pfaff]

Applied to master, thanks!
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [patch v2] conntrack: Expand 'conn_to_ct_dpif_entry()' locking.

[Ben Pfaff]

On Mon, May 20, 2019 at 09:50:28PM -0700, Darrell Ball wrote:
> When displaying a connection entry, several TCP fields are read
> from a connection entry. Hence, expand the 'conn' locking so the display
> does not potentially include fields values from different aggregate
> states.
> 
> Fixes: 967bb5c5cd90 ("conntrack: Add rcu support.")
> Signed-off-by: Darrell Ball 

Applied to master, thanks!
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] Bitte hilf mir, diesen Traum zu verwirklichen.

[Flora Michael]

Bitte hilf mir, diesen Traum zu verwirklichen.

Ich bin Frau Flora Michael aus Kanada. Ich bin mit Herrn Kellen Michael 
verheiratet, der neun Jahre lang mit der kanadischen Botschaft hier in der 
Republik Benin gearbeitet hat, bevor er 2015 starb. Wir waren elf Jahre ohne 
Kind verheiratet. Er starb nach kurzer Krankheit, die nur vier Tage dauerte. 
Vor seinem Tod wurden wir beide wiedergeboren.

Seit seinem Tod habe ich mich entschlossen, weder wieder zu heiraten noch ein 
Kind außerhalb meines Ehehauses zu bekommen, gegen das sich die Bibel richtet. 
Als mein verstorbener Ehemann lebte, hinterlegte er die Summe von vier 
Millionen Siebenhunderttausend US-Dollar (7,5 Millionen USD) in einer Bank hier 
in der Hauptstadt der Benin-Republik Porto Novo. Derzeit ist dieses Geld noch 
bei der Bank hier in der Hauptstadt der beninischen Republik Porto Novo.

Vor kurzem, nach meiner Krankheit, sagte mir mein Arzt, ich würde die nächsten 
acht Monate wegen Krebserkrankungen nicht durchhalten, sagte ich, dass sich 
diese Art von Knochenkrebs in den nächsten acht Monaten nicht ausbreiten wird 
und ich nicht mehr lebe. Was mich aber am meisten stört, ist meine 
Schlaganfallkrankheit.

Nachdem ich meinen Zustand gekannt hatte, entschied ich mich, diesen Fonds 
einer Kirche, einer Organisation oder einer Einzelperson zu spenden, die dieses 
Geld auf die Art und Weise verwenden wird, die ich hier erteile. Ich möchte 
eine Kirche, Organisation oder Einzelperson, die diesen Fonds für Waisenhäuser, 
Witwen wie mich, die Armen und Bedürftigen und auch für die Verbreitung des 
Wortes Gottes und für die Erhaltung des Hauses Gottes einsetzen wird.

Die Bibel ließ uns verstehen, dass "gesegnet die Hand ist, die gibt". Ich habe 
diese Entscheidung getroffen, weil ich kein Kind habe, das dieses Geld erben 
wird, und die Verwandten meines Mannes nicht Christen sind und ich nicht 
möchte, dass die Bemühungen meines Mannes von Ungläubigen genutzt werden. Ich 
möchte keine Situation, in der dieses Geld auf gottlose Weise verwendet wird. 
Deshalb treffe ich diese Entscheidung.

Ich habe keine Angst vor dem Tod, daher weiß ich, wohin ich gehen soll. Ich 
weiß, ich werde in der Brust des Herrn sein. In Exodus 14 VS 14 heißt es, dass 
der Herr meinen Fall bekämpfen wird und ich werde meine Ruhe bewahren. "Ich 
brauche in dieser Hinsicht keine telefonische Kommunikation wegen meiner 
Gesundheit und daher der Anwesenheit der Verwandten meines Mannes immer um mich 
herum Ich möchte, dass die Verwandten meines Mannes von dieser Entwicklung 
erfahren. Mit Gott ist alles möglich.

Sobald ich Ihre Antwort erhalte, gebe8 ich Ihnen die Kontaktperson der Bank, 
bei der die Gelder meines verstorbenen Mannes eingezahlt wurden, damit Sie sie 
direkt kontaktieren können. Ich sende Ihnen auch ein Ermächtigungsschreiben, 
das Sie als gegenwärtigen Begünstigten dieses Fonds beweist. Ich möchte, dass 
Sie und die Kirche immer für mich beten, weil der Herr mein Hirte ist. Mein 
Glück ist, dass ich ein Leben eines würdigen Christen gelebt habe.

Wer dem Herrn dienen will, muss ihm in Geist und Wahrheit dienen. Sei bitte 
immer ein Leben lang im Gebet. Jede Verzögerung in Ihrer Antwort gibt mir Raum, 
um eine andere Person für diesen Zweck zu finden. Bitte versichern Sie mir, 
dass Sie sich wie hier beschrieben verhalten werden. Ich hoffe auf eine Antwort 
von dir.

Danke und Gott segne dich.
Mit freundlichen Grüßen

Frau Flora Michael.

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v6 00/17] ovn-controller incremental processing

[Ben Pfaff]

On Fri, May 17, 2019 at 12:56:25PM -0700, Han Zhou wrote:
> As discussed at last OVN meeting, I am reposting the patch series after
> rebasing on latest master branch. There has been a lot a discussion since
> last review. The reason it was hold on were:
> 1. The concern of long term maintenance effort, since the change handler
>implementation can be complex if we want to support more incremental
>processing.
> 2. DDlog is showing promissing progress and we concluded that it not only
>fits for ovn-northd but also for ovn-controller. Once implemented it
>should be easier to maintain than the approach taken by this series.
> 
> These points are still valid and unchanged. The reason we are revisiting
> this patch series are:
> 1. Incremental processing is critical for scalability in production. Although
>DDlog is the long term solution, it can't be ready soon enough to solve
>current problem.
> 2. The current approach taken in this series is proved stable and effective
>in production, and it doesn't necessarily cause maintenance problems
>because it allows easily fall-back to full recomputing path.
> 3. This series sorted out dependency graph in flow compute related components
>which will help future DDlog convertion. So it doesn't conflict with
>the DDlog path we are targeting and could provide a smooth transition.

I applied this series to master.  Thank you!
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] one issue in vxlan functionality of the kernel-datapath type of ovs

[Ben Pfaff]

On Sat, May 18, 2019 at 10:23:50AM +, pei Jikui wrote:
> I found one issue in the vxlan functionality in kernel-datapath type of ovs 
> which could be potentially optimize?
> 
> 
> For example, there is a machine (A) running ovs with one configured one vxlan 
> interface with tunnel value 123,  then all the vxlan traffics destinate to 
> this machine(A) will be dealt with the ovs.
> 
> 
> Although the ovs in machine A only configured with one vxlan tunnel (123), 
> all the vxlan traffics regardless the tunnel value is 123 or not, will be 
> delivered to the vswitchd to do the slow path if there is no flow tables 
> found in the datapath.
> 
> This is due to the vxlan configuration such as the configured vxlan tunnel 
> valuse does not exist in the datapath. (There is only one vxlan tunnel with 
> vni value 0 exist in the datapath’s vxlan configuration).

It's true, but does it matter?  It would be unusual for a host to
receive much VXLAN traffic that it is only going to drop.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCHv9] netdev-afxdp: add new netdev type for AF_XDP.

[Ben Pfaff]

On Fri, May 24, 2019 at 11:03:33AM -0700, William Tu wrote:
> The patch introduces experimental AF_XDP support for OVS netdev.
> AF_XDP, the Address Family of the eXpress Data Path, is a new Linux socket
> type built upon the eBPF and XDP technology.  It is aims to have comparable
> performance to DPDK but cooperate better with existing kernel's networking
> stack.  An AF_XDP socket receives and sends packets from an eBPF/XDP program
> attached to the netdev, by-passing a couple of Linux kernel's subsystems
> As a result, AF_XDP socket shows much better performance than AF_PACKET
> For more details about AF_XDP, please see linux kernel's
> Documentation/networking/af_xdp.rst. Note that by default, this feature is
> not compiled in.
> 
> Signed-off-by: William Tu 

How heavy is the x86(_64)-only dependency?  It seems undesirable.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH V2 0/2] Add include mode to priority tags port option

[Ben Pfaff]

On Sun, May 12, 2019 at 05:50:58AM +, Eli Britstein wrote:
> Setting priority-tags to "true" Open vSwitch still omits the
> 802.1Q header on output if both the VLAN ID and priority would be zero.
> Add an option to retain the 8021Q header in such frames as well.
> 
> Patch #1: change boolean to enum as a pre-step to adding addition option
> Patch #2: add "include" mode for priority-tags configuration
> 
> V2 changes:
> - Added NEWS item.
> - A bit changed documentation.
> 
> Eli Britstein (2):
>   ofproto-dpif-xlate: Change priority tags from boolean to enum
>   ofproto-dpif-xlate: Add include mode to priority tags

I applied these to master.

I decided that I wanted to change the enums from
omit/include-non-zero/include to never/if-nonzero/always, so I made that
change, and I simplified the NEWS item as well.

Thanks,

Ben.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v3] datapath: Fix build errors for 4.9.172+ kernels

[Yifeng Sun]

4.9.172+ kernel backported upstream patch 70b095c843266
("ipv6: remove dependency of nf_defrag_ipv6 on ipv6 module")
and this caused compilation errors of OVS kernel module.

This patch fixes it by checking and using new functions
introduced by the upstream patch.

Travis tests passed at
https://travis-ci.org/yifsun/ovs-travis/builds/536527230
with latest Linux kernel version.

In addition, this patch doesn't introduce failed tests for latest kernels
of Ubuntu (bionic, trusty, xenial), fedora, centos 73, rhel (74, 75, 76).

Reported-by: Ilya Maximets 
Signed-off-by: Yifeng Sun 
---
v1->v2: Fixed at bug that crashes certain kernels. Thanks YiHung!
v2->v3: Added HAVE_INET_FRAGS_RND, thanks YiHung.

 datapath/linux/compat/nf_conntrack_reasm.c | 23 ++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/datapath/linux/compat/nf_conntrack_reasm.c 
b/datapath/linux/compat/nf_conntrack_reasm.c
index 9d77d982712c..23904eef08d7 100644
--- a/datapath/linux/compat/nf_conntrack_reasm.c
+++ b/datapath/linux/compat/nf_conntrack_reasm.c
@@ -41,6 +41,7 @@
 #include 
 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -138,8 +139,12 @@ static void nf_ct_frag6_expire(unsigned long data)
 #ifdef HAVE_INET_FRAGS_RND
ip6_expire_frag_queue(net, fq, _frags);
 #else
+#ifdef HAVE_IPV6_FRAG_H
+   ip6frag_expire_frag_queue(net, fq);
+#else
ip6_expire_frag_queue(net, fq);
 #endif
+#endif
 }
 
 #ifdef HAVE_INET_FRAGS_RND
@@ -673,6 +678,16 @@ static struct pernet_operations nf_ct_net_ops = {
.exit = nf_ct_net_exit,
 };
 
+#ifdef HAVE_IPV6_FRAG_H
+static const struct rhashtable_params nfct_rhash_params = {
+   .head_offset= offsetof(struct inet_frag_queue, node),
+   .hashfn = ip6frag_key_hashfn,
+   .obj_hashfn = ip6frag_obj_hashfn,
+   .obj_cmpfn  = ip6frag_obj_cmpfn,
+   .automatic_shrinking= true,
+};
+#endif
+
 int rpl_nf_ct_frag6_init(void)
 {
int ret = 0;
@@ -683,10 +698,16 @@ int rpl_nf_ct_frag6_init(void)
 #ifdef HAVE_INET_FRAGS_RND
nf_frags.hashfn = nf_hashfn;
nf_frags.match = ip6_frag_match;
+   nf_frags.constructor = ip6_frag_init;
+#else
+#ifdef HAVE_IPV6_FRAG_H
+   nf_frags.rhash_params = nfct_rhash_params;
+   nf_frags.constructor = ip6frag_init;
 #else
nf_frags.rhash_params = ip6_rhash_params;
-#endif
nf_frags.constructor = ip6_frag_init;
+#endif
+#endif /* HAVE_INET_FRAGS_RND */
nf_frags.destructor = NULL;
nf_frags.qsize = sizeof(struct frag_queue);
nf_frags.frag_expire = nf_ct_frag6_expire;
-- 
2.7.4

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] datapath: Fix build errors for 4.9.172+ kernels

[Yifeng Sun]

Thanks Yi-Hung for the review.

4.9 kernel is using ip6frag_init instead of ip6_frag_init, so the
original change
is essential. Otherwise, there will be compiling errors:
https://travis-ci.org/yifsun/ovs-travis/builds/536900954

I will add /* HAVE_INET_FRAGS_RND */ and submit a new version.
Yifeng

On Fri, May 24, 2019 at 10:59 AM Yi-Hung Wei  wrote:
>
> On Thu, May 23, 2019 at 4:58 PM Yifeng Sun  wrote:
> >
> > 4.9.172+ kernel backported upstream patch 70b095c843266
> > ("ipv6: remove dependency of nf_defrag_ipv6 on ipv6 module")
> > and this caused compilation errors of OVS kernel module.
> >
> > This patch fixes it by checking and using new functions
> > introduced by the upstream patch.
> >
> > Travis tests passed at
> > https://travis-ci.org/yifsun/ovs-travis/builds/536527230
> > with latest Linux kernel version.
> >
> > In addition, this patch doesn't introduce failed tests for latest kernels
> > of Ubuntu (bionic, trusty, xenial), fedora, centos 73, rhel (74, 75, 76).
> >
> > Reported-by: Ilya Maximets 
> > Signed-off-by: Yifeng Sun 
> > ---
> > v1->v2: Fixed at bug that crashes certain kernels. Thanks YiHung!
> >
> >  datapath/linux/compat/nf_conntrack_reasm.c | 23 ++-
> >  1 file changed, 22 insertions(+), 1 deletion(-)
> >
> > diff --git a/datapath/linux/compat/nf_conntrack_reasm.c 
> > b/datapath/linux/compat/nf_conntrack_reasm.c
> > index 9d77d982712c..23904eef08d7 100644
> > --- a/datapath/linux/compat/nf_conntrack_reasm.c
> > +++ b/datapath/linux/compat/nf_conntrack_reasm.c
> >  int rpl_nf_ct_frag6_init(void)
> >  {
> > int ret = 0;
> > @@ -683,10 +698,16 @@ int rpl_nf_ct_frag6_init(void)
> >  #ifdef HAVE_INET_FRAGS_RND
> > nf_frags.hashfn = nf_hashfn;
> > nf_frags.match = ip6_frag_match;
> > +   nf_frags.constructor = ip6_frag_init;
> > +#else
> > +#ifdef HAVE_IPV6_FRAG_H
> > +   nf_frags.rhash_params = nfct_rhash_params;
> > +   nf_frags.constructor = ip6frag_init;
> >  #else
> > nf_frags.rhash_params = ip6_rhash_params;
> > -#endif
> > nf_frags.constructor = ip6_frag_init;
> > +#endif
> > +#endif
>
> Hi Yifeng,
>
> Thanks for the update. Are we replicating "nf_frags.constructor =
> ip6_frag_init;" for the 3 cases?
>
> How about the following change?
>
>  int rpl_nf_ct_frag6_init(void)
>  {
> int ret = 0;
> @@ -684,8 +699,12 @@ int rpl_nf_ct_frag6_init(void)
> nf_frags.hashfn = nf_hashfn;
> nf_frags.match = ip6_frag_match;
>  #else
> +#ifdef HAVE_IPV6_FRAG_H
> +   nf_frags.rhash_params = nfct_rhash_params;
> +#else
> nf_frags.rhash_params = ip6_rhash_params;
>  #endif
> +#endif /* HAVE_INET_FRAGS_RND */
> nf_frags.constructor = ip6_frag_init;
>
> -Yi-Hung
>
> > nf_frags.destructor = NULL;
> > nf_frags.qsize = sizeof(struct frag_queue);
> > nf_frags.frag_expire = nf_ct_frag6_expire;
> > --
> > 2.7.4
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCHv9] netdev-afxdp: add new netdev type for AF_XDP.

[William Tu]

The patch introduces experimental AF_XDP support for OVS netdev.
AF_XDP, the Address Family of the eXpress Data Path, is a new Linux socket
type built upon the eBPF and XDP technology.  It is aims to have comparable
performance to DPDK but cooperate better with existing kernel's networking
stack.  An AF_XDP socket receives and sends packets from an eBPF/XDP program
attached to the netdev, by-passing a couple of Linux kernel's subsystems
As a result, AF_XDP socket shows much better performance than AF_PACKET
For more details about AF_XDP, please see linux kernel's
Documentation/networking/af_xdp.rst. Note that by default, this feature is
not compiled in.

Signed-off-by: William Tu 
---
v1->v2:
- add a list to maintain unused umem elements
- remove copy from rx umem to ovs internal buffer
- use hugetlb to reduce misses (not much difference)
- use pmd mode netdev in OVS (huge performance improve)
- remove malloc dp_packet, instead put dp_packet in umem

v2->v3:
- rebase on the OVS master, 7ab4b0653784
  ("configure: Check for more specific function to pull in pthread library.")
- remove the dependency on libbpf and dpif-bpf.
  instead, use the built-in XDP_ATTACH feature.
- data structure optimizations for better performance, see[1]
- more test cases support
v3: https://mail.openvswitch.org/pipermail/ovs-dev/2018-November/354179.html

v3->v4:
- Use AF_XDP API provided by libbpf
- Remove the dependency on XDP_ATTACH kernel patch set
- Add documentation, bpf.rst

v4->v5:
- rebase to master
- remove rfc, squash all into a single patch
- add --enable-afxdp, so by default, AF_XDP is not compiled
- add options: xdpmode=drv,skb
- add multiple queue and multiple PMD support, with options: n_rxq
- improve documentation, rename bpf.rst to af_xdp.rst

v5->v6
- rebase to master, commit 0cdd5b13de91b98
- address errors from sparse and clang
- pass travis-ci test
- address feedback from Ben
- fix issues reported by 0-day robot
- improved documentation

v6-v7
- rebase to master, commit abf11558c1515bf3b1
- address feedbacks from Ilya, Ben, and Eelco, see:
  https://www.mail-archive.com/ovs-dev@openvswitch.org/msg32357.html
- add XDP mode change, implement get/set_config, reconfigure
- Fix reconfiguration/crash issue caused by libbpf, see patch:
  [PATCH bpf 0/2] libbpf: fixes for AF_XDP teardown
- perf optimization for batching umem_push/pop
- perf optimization for batching kick_tx
- test build with dpdk
- fix/refactor atomic operation
- make AF_XDP x86 specific, otherwise fail at build time
- lots of code refactoring
- add PVP setup in documentation

v7-v8:
- Address feedback from Ilya at:
  https://patchwork.ozlabs.org/patch/1095019/
- add netdev-linux-private.h
- fix afxdp reconfigure issue
- sort include headers
- remove unnecessary OVS_UNUSED
- coding style fixes
- error case handling and memory leak

v8-v9:
- rebase to master 180bbbed3a3867d52
- Address review feedback from Ben, Ilya and Eelco, at:
  https://patchwork.ozlabs.org/patch/1097740/
- == From Ilya ==
- Optimize the reconfiguration logic
- Implement .rxq_recv and .send for afxdp
- Remove system-afxdp-traffic.at, reuse existing code
- Use Ilya's rdtsc code
- remove --disable-system
- == From Eelco ==
- Fix bug when remove br0, util(revalidator49)|EMER|lib/poll-loop.c:111:
  assertion !fd != !wevent failed
- Fix bug and use default value from libbpf, ex: XSK_RING_PROD__DEFAULT...
- Clear xdp program when receive signal, ctrl+c
- Add options to vswitch.xml, set xdpmode default to skb-mode
- No support for ARM and PPC, now x86_64 only
- remove redundant header includes and function/macro definitions
- remove some ifdef HAVE_AF_XDP
- == From others/both about afxdp rx and tx ==
- Several umem push/pop error handling improvement/fixes
- add lock to address concurrent_txq case
- improve error handling
- add stats
- Things that are not done yet
- MTU limitation
- n_txq_desc/n_rxq_desc option.
---
 Documentation/automake.mk |   1 +
 Documentation/index.rst   |   1 +
 Documentation/intro/install/afxdp.rst | 433 +
 Documentation/intro/install/index.rst |   1 +
 acinclude.m4  |  35 ++
 configure.ac  |   1 +
 lib/automake.mk   |  14 +
 lib/dp-packet.c   |  28 ++
 lib/dp-packet.h   |  18 +-
 lib/dpif-netdev-perf.h|   6 +
 lib/netdev-afxdp.c| 858 ++
 lib/netdev-afxdp.h|  77 +++
 lib/netdev-linux-private.h| 139 ++
 lib/netdev-linux.c| 121 ++---
 lib/netdev-provider.h |   3 +
 lib/netdev.c  |  11 +
 lib/spinlock.h|  70 +++
 lib/xdpsock.c | 183 
 lib/xdpsock.h | 101 
 tests/automake.mk |  16 +
 tests/system-afxdp-macros.at  |  20 +
 

Re: [ovs-dev] [PATCH v2] datapath: Fix build errors for 4.9.172+ kernels

[Yi-Hung Wei]

On Thu, May 23, 2019 at 4:58 PM Yifeng Sun  wrote:
>
> 4.9.172+ kernel backported upstream patch 70b095c843266
> ("ipv6: remove dependency of nf_defrag_ipv6 on ipv6 module")
> and this caused compilation errors of OVS kernel module.
>
> This patch fixes it by checking and using new functions
> introduced by the upstream patch.
>
> Travis tests passed at
> https://travis-ci.org/yifsun/ovs-travis/builds/536527230
> with latest Linux kernel version.
>
> In addition, this patch doesn't introduce failed tests for latest kernels
> of Ubuntu (bionic, trusty, xenial), fedora, centos 73, rhel (74, 75, 76).
>
> Reported-by: Ilya Maximets 
> Signed-off-by: Yifeng Sun 
> ---
> v1->v2: Fixed at bug that crashes certain kernels. Thanks YiHung!
>
>  datapath/linux/compat/nf_conntrack_reasm.c | 23 ++-
>  1 file changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/datapath/linux/compat/nf_conntrack_reasm.c 
> b/datapath/linux/compat/nf_conntrack_reasm.c
> index 9d77d982712c..23904eef08d7 100644
> --- a/datapath/linux/compat/nf_conntrack_reasm.c
> +++ b/datapath/linux/compat/nf_conntrack_reasm.c
>  int rpl_nf_ct_frag6_init(void)
>  {
> int ret = 0;
> @@ -683,10 +698,16 @@ int rpl_nf_ct_frag6_init(void)
>  #ifdef HAVE_INET_FRAGS_RND
> nf_frags.hashfn = nf_hashfn;
> nf_frags.match = ip6_frag_match;
> +   nf_frags.constructor = ip6_frag_init;
> +#else
> +#ifdef HAVE_IPV6_FRAG_H
> +   nf_frags.rhash_params = nfct_rhash_params;
> +   nf_frags.constructor = ip6frag_init;
>  #else
> nf_frags.rhash_params = ip6_rhash_params;
> -#endif
> nf_frags.constructor = ip6_frag_init;
> +#endif
> +#endif

Hi Yifeng,

Thanks for the update. Are we replicating "nf_frags.constructor =
ip6_frag_init;" for the 3 cases?

How about the following change?

 int rpl_nf_ct_frag6_init(void)
 {
int ret = 0;
@@ -684,8 +699,12 @@ int rpl_nf_ct_frag6_init(void)
nf_frags.hashfn = nf_hashfn;
nf_frags.match = ip6_frag_match;
 #else
+#ifdef HAVE_IPV6_FRAG_H
+   nf_frags.rhash_params = nfct_rhash_params;
+#else
nf_frags.rhash_params = ip6_rhash_params;
 #endif
+#endif /* HAVE_INET_FRAGS_RND */
nf_frags.constructor = ip6_frag_init;

-Yi-Hung

> nf_frags.destructor = NULL;
> nf_frags.qsize = sizeof(struct frag_queue);
> nf_frags.frag_expire = nf_ct_frag6_expire;
> --
> 2.7.4
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovsdb-client.1: Fix typo.

[Justin Pettit]

> On May 23, 2019, at 2:42 PM, Ben Pfaff  wrote:
> 
> On Thu, May 23, 2019 at 02:10:08PM -0700, Justin Pettit wrote:
>> Signed-off-by: Justin Pettit 
> 
> Acked-by: Ben Pfaff 

Thanks.  I pushed this to master.

--Justin


___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC 1/3] OVN: introduce Controller_Event table

[Ben Pfaff]

On Thu, May 16, 2019 at 06:05:24PM +0200, Lorenzo Bianconi wrote:
> Add Controller_Event table to OVN SBDB in order to
> report CMS related event.
> Introduce event_table hashmap array and controller_event related
> structures to ovn-controller in order to track pending events
> forwarded by ovs-vswitchd. Moreover integrate event_table hashmap
> array with event_table ovn-sbdb table
> 
> Signed-off-by: Mark Michelson 
> Co-authored-by: Mark Michelson 
> Signed-off-by: Lorenzo Bianconi 

This seems like one potentially reasonable way for ovn-controller to
report events to the CMS.  I want to point out some design aspects that
might need to be considered:

1. Controller_Event doesn't include a column to attribute an event to a
   particular Chassis.  This might be important for accounting: how can
   an ovn-controller tell which events it owns and should eventually
   delete?  Currently it looks like every ovn-controller always iterates
   over every Controller_Event; is that really desirable and scalable?
   Such a column would also allow update_sb_monitors() to monitor only
   the rows associated with the current chassis, increasing scalability.

   Keep in mind that ovn-controller probably can't just keep in memory
   which events belong to it, because it might be restarted and needs to
   be able to recover that information.

2. Are these events (or each individual event, anyway) single-consumer?
   I believe that this design only works for single-consumer events
   because only one consumer can mark them as 'handled'.

3. Is there value in having 'handled', instead of just having the CMS
   delete rows that it has processed?  Using 'handled' requires an extra
   round-trip between ovn-controller and the database.

4. What is the tolerance for events that are never delivered or that are
   delivered more than once?  What can actually be guaranteed, given
   that the database can die and that ovn-controller can die?  (Also,
   OVSDB transactions cannot guarantee exactly-once semantics in corner
   cases unless the transactions are idempotent.)

5. How is the number of per-controller rows limited?

I think the workflow for these events should be clearly specified.  I
guess it's something like this:

1. ovn-controller detects that an event has occurred and adds a
   corresponding row to the Controller_Event table with false 'handled'.

2. CMS consumes the row and acts on it.

3. CMS changes 'handled' to true.

4. ovn-controller deletes row.

although I think that "3. CMS deletes row." seems more straightforward.

Did you consider the inter-version compatibility issues of making
event_type an enum?  It will force an upgrade order of first ovn-northd,
then the database schema, then ovn-controller, because any other order
will make it possible that ovn-controller tries to add an invalid event
type (which ovsdb-server will reject) or that an event that ovn-northd
doesn't understand nevertheless reaches it.  However,
Documentation/intro/install/ovn-upgrades.rst recommends a different
order.  In any case the upgrade implications need to be considered.

Some typo fixes:

diff --git a/ovn/ovn-sb.xml b/ovn/ovn-sb.xml
index ae5225e18fd0..a010c95b221b 100644
--- a/ovn/ovn-sb.xml
+++ b/ovn/ovn-sb.xml
@@ -3476,8 +3476,8 @@ tcp.flags = RST;
   
   
 
-  Database table used by ovn-controller to report CMS related
-  events
+  Database table used by ovn-controller to report CMS related
+  events.  The workflow for event processing is:
 
 
@@ -3485,7 +3485,7 @@ tcp.flags = RST;
 
 
 
-  Key-value pairs used to spcify evento info to the CMS.
+  Key-value pairs used to specify event info to the CMS.
   Possible values are:
 
   
@@ -3498,7 +3498,7 @@ tcp.flags = RST;
   empty_lb_backends event
 
 
-  load_balancer: UUID fo the load balancer reported for
+  load_balancer: UUID of the load balancer reported for
   the empty_lb_backends event
 
   
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] Add support for DHCP option 15 - domain name

[Numan Siddique]

On Fri, May 24, 2019 at 10:26 PM Ben Pfaff  wrote:

> On Tue, May 21, 2019 at 02:13:03PM +0200, mjoze...@redhat.com wrote:
> > From: Maciej Józefczyk 
> >
> > For Openstack Internal DNS functionality we need
> > to provide support for domain_name option.
> >
> > Signed-off-by: Maciej Józefczyk 
>
> This seems to be a little different from just adding support for a new
> DHCP option, because in some places it's renaming an existing "domain"
> option to "domain_name".  That may introduce a cross-version
> compatibility issue.  If it does, then it would be better to retain the
> existing name.
>
> What's the full story?
>

Oops. I never noticed the test code deleting the option - "domain" and
adding "domain_name".

I looked into the code now and this is what has happened. We never
supported the dhcp
option 15 (domain_name) earlier and this patch adds the support for this
option.
But the initial commit [1] which added the DHCP support in OVN has added
the option 15 with the
name - "domain" in the test-ovn.c. I think it was a mistake from my side to
add that option in the
test code even though we never supported that option. My mistake. I think I
might have added that
option in the test code (but not in the actual list) is probably to test
the action - put_dhcp_opts
with an option of type "str".

[1] -
https://github.com/openvswitch/ovs/commit/42814145d70c77462ce28b38841cd160f0486776#diff-e3335c87e2c9ad67b3d403e4f4361771R253

Thanks
Numan


> Thanks,
>
> Ben.
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] conntrack: add tcp_in_liberal option in userspace conntrack

[Darrell Ball]

Thanks for the patch Lidong/Zhike

I took an initial look and will send a response next week.

Darrell

On Tue, May 21, 2019 at 8:53 PM 姜立东  wrote:

> From 3ce112684921bca74839e109fda91848aa024a54 Mon Sep 17 00:00:00 2001
> From: Jiang Lidong 
> Date: Wed, 22 May 2019 11:21:34 +0800
> Subject: [PATCH] conntrack: add tcp_in_liberal option in userspace
> conntrack
>
> Adding similar cp_in_liberal option in userspace conntrack as
> kernel conntrack does to skip seq check on tcp connection.
> It prevents packet is marked as INVALID by stable seq
> info in conntrack connection.
>
> This option can help to make traffic survive in hardware
> offloading cases, especially when traffic is being moved
> back to software path from hardware forwarding engine.
>
> Signed-off-by: Lidong Jiang 
> Signed-off-by: Zhike Wang 
>
> ---
> lib/conntrack-private.h | 2 ++
> lib/conntrack-tcp.c | 5 +++--
> lib/conntrack.c | 6 ++
> lib/conntrack.h | 4 +++-
> lib/dpif-netdev.c   | 6 ++
> 5 files changed, 20 insertions(+), 3 deletions(-)
>
> diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h
> index 51b7d7f..9bc99cd 100644
> --- a/lib/conntrack-private.h
> +++ b/lib/conntrack-private.h
> @@ -172,6 +172,8 @@ struct conntrack {
>  /* Fragmentation handling context. */
>  struct ipf *ipf;
> +
> +bool tcp_be_liberal;
> };
>  /* Lock acquisition order:
> diff --git a/lib/conntrack-tcp.c b/lib/conntrack-tcp.c
> index 397aca1..61abafb 100644
> --- a/lib/conntrack-tcp.c
> +++ b/lib/conntrack-tcp.c
> @@ -272,7 +272,7 @@ tcp_conn_update(struct conntrack *ct, struct conn
> *conn_,
>  int ackskew = check_ackskew ? dst->seqlo - ack : 0;
> #define MAXACKWINDOW (0x + 1500)/* 1500 is an arbitrary fudge
> factor */
> -if (SEQ_GEQ(src->seqhi, end)
> +if ((SEQ_GEQ(src->seqhi, end)
>  /* Last octet inside other's window space */
>  && SEQ_GEQ(seq, src->seqlo - (dst->max_win << dws))
>  /* Retrans: not more than one window back */
> @@ -281,7 +281,8 @@ tcp_conn_update(struct conntrack *ct, struct conn
> *conn_,
>  && (ackskew <= (MAXACKWINDOW << sws))
>  /* Acking not more than one window forward */
>  && ((tcp_flags & TCP_RST) == 0 || orig_seq == src->seqlo
> -|| (orig_seq == src->seqlo + 1) || (orig_seq + 1 ==
> src->seqlo))) {
> +|| (orig_seq == src->seqlo + 1) || (orig_seq + 1 ==
> src->seqlo)))
> +|| (ct->tcp_be_liberal)) {
>  /* Require an exact/+1 sequence match on resets when possible */
>  /* update max window */
> diff --git a/lib/conntrack.c b/lib/conntrack.c
> index 6711f5e..bd92710 100644
> --- a/lib/conntrack.c
> +++ b/lib/conntrack.c
> @@ -2282,6 +2282,12 @@ conntrack_ipf_ctx(struct conntrack *ct)
>  return ct->ipf;
> }
> +void
> +conntrack_set_tcp_be_liberal(struct conntrack *ct, bool enabled)
> +{
> +ct->tcp_be_liberal = enabled;
> +}
> +
> int
> conntrack_dump_start(struct conntrack *ct, struct conntrack_dump *dump,
>   const uint16_t *pzone, int *ptot_bkts)
> diff --git a/lib/conntrack.h b/lib/conntrack.h
> index 2012150..b8d799d 100644
> --- a/lib/conntrack.h
> +++ b/lib/conntrack.h
> @@ -119,5 +119,7 @@ int conntrack_set_maxconns(struct conntrack *ct,
> uint32_t maxconns);
> int conntrack_get_maxconns(struct conntrack *ct, uint32_t *maxconns);
> int conntrack_get_nconns(struct conntrack *ct, uint32_t *nconns);
> struct ipf *conntrack_ipf_ctx(struct conntrack *ct);
> -
> +
> +void conntrack_set_tcp_be_liberal(struct conntrack *ct, bool enabled);
> +
> #endif /* conntrack.h */
> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> index 5a6f2ab..ae6a18e 100644
> --- a/lib/dpif-netdev.c
> +++ b/lib/dpif-netdev.c
> @@ -3823,6 +3823,12 @@ dpif_netdev_set_config(struct dpif *dpif, const
> struct smap *other_config)
>  uint32_t tx_flush_interval, cur_tx_flush_interval;
>  uint64_t rebalance_intvl;
> +bool tcp_be_liberal = smap_get_bool(other_config,
> +"conntrack_tcp_be_liberal",
> +false);
> +
> +conntrack_set_tcp_be_liberal(>conntrack, tcp_be_liberal);
> +
>  tx_flush_interval = smap_get_int(other_config, "tx-flush-interval",
>   DEFAULT_TX_FLUSH_INTERVAL);
>  atomic_read_relaxed(>tx_flush_interval, _tx_flush_interval);
> --
> 1.8.3.1
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] travis: Test with latest stable kernel releases.

[Ben Pfaff]

On Tue, May 21, 2019 at 03:46:22PM +0300, Ilya Maximets wrote:
> Instead of managing kernel minor versions manually we could always test
> with the most recent stable release of the desired branch.
> 
> With this patch applied Travis will always check with the most recent
> kernels, so we'll be notified about changes in upstream kernels that
> breaks the build of our kernel module. However, this will also break
> Travis checks on patches that doesn't touch the kernel parts until
> we fix the module.
> 
> Signed-off-by: Ilya Maximets 
> Tested-by: Yifeng Sun 
> Reviewed-by: Yifeng Sun 
> ---
> 
> Right now we have a broken build with recent longterm 4.9.177.
> So, this patch, should not be applied before the build with
> 4.9.177 fixed: https://patchwork.ozlabs.org/patch/1102480/
> 
> Version 2:
>   * Rebased on current master.
>   * No functional changes so I kept tags from v1.

Acked-by: Ben Pfaff 

but I guess you should wait for the broken build to be fixed before
applying it.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] Add support for DHCP option 15 - domain name

[Ben Pfaff]

On Tue, May 21, 2019 at 02:13:03PM +0200, mjoze...@redhat.com wrote:
> From: Maciej Józefczyk 
> 
> For Openstack Internal DNS functionality we need
> to provide support for domain_name option.
> 
> Signed-off-by: Maciej Józefczyk 

This seems to be a little different from just adding support for a new
DHCP option, because in some places it's renaming an existing "domain"
option to "domain_name".  That may introduce a cross-version
compatibility issue.  If it does, then it would be better to retain the
existing name.

What's the full story?

Thanks,

Ben.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v3 2/2] conntrack: add display support for sctp

[Ben Pfaff]

On Tue, May 21, 2019 at 02:16:31PM -0400, Aaron Conole wrote:
> Currently, only the netlink datapath supports SCTP connection tracking,
> but at least this removes the warning message that will pop up when
> running something like:
> 
>ovs-appctl dpctl/dump-conntrack
> 
> This doesn't impact any conntrack functionality, just the display.
> 
> Signed-off-by: Aaron Conole 
> ---
> v3: Moved the header detection to 1/2, and added a compat layer
> Changed to PRIu32 format specifier

I applied these to master.  Thanks!
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] Resolución de quejas y conflictos.

[Atención al cliente de alta calidad]

Cursos esenciales - Webinar Interactivo – Jueves 20 de Marzo

Atención al cliente de alta calidad

Todos somos clientes y, cuando actuamos como tales, queremos ser tratados como 
individuos especiales, no como un número.
 Sin embargo, como clientes, todos recordamos experiencias negativas que 
asociamos a un producto, una empresa o una persona que, de poder elegir, 
evitaremos en el futuro. 

Desarrollaremos en los participantes la capacidad de brindar una atención al
cliente de alta calidad. 
 

Objetivos Específicos:


• Mejorar la predisposición hacia la calidad en la atención al cliente.
• Fortalecer la comprensión de las necesidades del cliente.
• Proporcionar herramientas para la resolución de quejas y conflictos. 

Para mayor información, responder sobre este correo con la palabra Calidad + 
los siguientes datos:


NOMBRE:

TELÉFONO:

EMPRESA: 

Email Alterno:  

Llamanos al (045) 55 1554 6630
www.Innovalearn.mx 
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] AUTHORS: Add Liliia Butorina.

[Ilya Maximets]

On 24.05.2019 19:10, Ben Pfaff wrote:
> On Fri, May 24, 2019 at 04:32:50PM +0300, Ilya Maximets wrote:
>> Signed-off-by: Ilya Maximets 
> 
> Acked-by: Ben Pfaff 

Thanks! Applied to master.

> 
> I usually just push these, since they're self-evidently correct.

Seems reasonable. I'll do the same in the future.

Best regards, Ilya Maximets.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] 回复: [ovs-discuss] An issue that the deleted flow tables in kernel based datapath couldn't be established again

[Ben Pfaff]

On Fri, May 24, 2019 at 03:23:26AM +, pei Jikui wrote:
> Ben,
> 
> Much thanks for your reply.
> 1) I know this command will be rarely used by customer.  For me, the reason 
> why I used this command is trying to understand the full-cycle of packet 
> traveling in OVS system.
>And I do not know if there are some cases for the customer in the real 
> world to use this command.
> 
> 3) a) Not for every packets to the userspace. Just for the packets generated 
> the packet_miss upcall.
>  b) The initial thinking
>  i) for the case to delete all the flow tables, then cleanup all the 
> ufid keys in userspace.

But how would userspace find out?  "ovs-dpctl del-flows" goes directly
to the kernel.  It doesn't go through ovs-vswitchd at all.

>  ii) No much think for the case how to delete the according ufid for 
> the deleting specific flow entries.
> 
> Thanks
> 
> Jikui
> 
> 
> 发送自 Outlook
> 
> 
> 发件人: Ben Pfaff 
> 发送时间: 2019年5月24日 0:57
> 收件人: pei Jikui
> 抄送: ovs-dev@openvswitch.org; ovs-disc...@openvswitch.org
> 主题: Re: [ovs-discuss] An issue that the deleted flow tables in kernel based 
> datapath couldn't be established again
> 
> On Thu, May 23, 2019 at 11:57:05AM +, pei Jikui wrote:
> > 1)I found a case in the netlink based datapth, that if we delete the 
> > existing datapath flow tables via “ovs-dpctl del-flows”, the according 
> > datpath flow tables could not be created again and then  all the 
> > consecutive packets need to go through the user-space slow path.
> 
> OK.
> 
> One way to avoid this problem is to not run "ovs-dpctl del-flows".  Is
> there a reason you want to run that command?
> 
> > 2)I also found the potential root cause.  It is because when we delete 
> > the datapath flow tables, their according ufid keys stored in the userspace 
> > vswitchd are not deleted accordingly.  So, when the coming packets’ 
> > packet_missing upcall sent to vswitchd, they ufid key is still in the 
> > UKEY_OPERATIONAL status so that the DPIF_FP_CREATE message will not be sent 
> > to datapath anymore.
> 
> That does seem like a plausible root cause.
> 
> > 3)That will be causing the above case.  The possible fixinges are,
> >
> > a) Send the DPIF_FP_CREATE message to datapath regardless if the according 
> > ufid key exists or not. (I have verified this fix).
> 
> Do you mean, send the message on every packet that comes to userspace?
> This could have performance implications, but I don't know how much.
> 
> > b) More fine-graind fix is when we execute “ovs-dpctl del-flows **”, we 
> > also should clean up the according ufid cach in vswitch.
> 
> How do you suggest that userspace should detect that the flows were
> deleted?
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] dpif-netdev: Forwarding optimization for direct output flows.

[Ben Pfaff]

On Fri, May 24, 2019 at 03:18:50PM +0300, Ilya Maximets wrote:
> There are some cases where users want to have simple forwarding or drop
> rules for all packets received from particular port, i.e :
> 
>   "in_port=1,actions=2"
>   "in_port=1,actions=IN_PORT"
>   "in_port=1,actions=drop"
> 
> There are also cases where complex OF flows could be simplified down
> to simple forwarding/drop datapath flows.
> 
> In theory, we don't need to parse packets at all to follow these flows.
> "Direct output forwarding" optimization is intended to speed up above
> cases.
> 
> Design:
> 
> Due to various implementation restrictions userspace datapath has
> following flow fields always in exact match (i.e. it's required to
> match at least these fields of a packet even if the OF rule doesn't
> need that):
> 
>   - recirc_id
>   - in_port
>   - packet_type
>   - dl_type
>   - vlan_tci
>   - nw_frag (for ip packets)
> 
> Not all of these fields are related to packet itself. We already
> know the current 'recirc_id' and the 'in_port' before starting the
> packet processing. It also seems safe to assume that we're working
> with Ethernet packets. dpif-netdev sets exact match on 'vlan_tci'
> to avoid issues with flow format conversion and we don't really need
> to match with it until ofproto layer didn't ask us to.
> So, for the simple forwarding OF rule we need to match only with
> 'dl_type' and 'nw_frag'.
> 
> 'in_port', 'dl_type' and 'nw_frag' could be combined in a single
> 64bit integer that could be used as a hash in hash map.
> 
> New per-PMD flow table 'direct_output_table' introduced to store
> direct output flows only. 'dp_netdev_flow_add' adds flow to the
> usual 'flow_table' and to 'direct_output_table' if the flow meets
> following constraints:
> 
>   - 'recirc_id' in flow match is 0.
>   - Flow wildcards originally had wildcarded 'vlan_tci'.
>   - Flow has no actions (drop) or exactly one action equal to
> OVS_ACTION_ATTR_OUTPUT.
>   - Flow wildcards contains only minimal set of non-wildcarded fields
> (listed above).
> 
> If the number of flows for current 'in_port' in regular 'flow_table'
> equals number of flows for current 'in_port' in 'direct_output_table',
> we may use direct output optimization, because all the flows we have
> are direct output flows. This means that we only need to parse
> 'dl_type' and 'nw_frag' to perform packet matching.
> Now we making the unique flow mark from the 'in_port', 'dl_type' and
> 'nw_frag' and looking for it in 'direct_output_table'.
> On successful lookup we don't need to make full 'miniflow_extract()'.
> 
> Unsuccessful lookup technically means that we have no sufficient flow
> in datapath and upcall will be required. We may optimize this path
> in the future by bypassing the EMC, SMC and dpcls lookups in this case.
> 
> Performance improvement of this solution on a 'direct output' flows
> should be comparable with partial HW offloading, because it parses same
> packet fields and uses similar flow lookup scheme.
> However, unlike partial HW offloading, it works for all port types
> including virtual ones.
> 
> Signed-off-by: Ilya Maximets 
> ---
> 
> This patch was made as a point for "virtio-forwarder" discussion:
>https://mail.openvswitch.org/pipermail/ovs-dev/2019-May/358686.html
> However, it might be very useful by itself for usual cases too.
> 
> Testing is very welcome. I didn't run the performance tests on real
> systems, so I don't know the real performance impact yet.

The changes to parse_tcp_flags() look good to me.  It may be worth
updating the comments on that function to say that the output arguments
are set only if the function returns nonzero.

In general, I really support the idea that OVS is general-purpose but
that we should optimize it for important use cases.  That's basically
what we did at Nicira/VMware for network virtualization: we wanted to
make sure that OVS was as general as possible so that many people would
use it and adopt it, while at the same time making sure that it
performed well for NVP/NSX.  I mean, we didn't want it to run slowly in
other cases, obviously, but they weren't what we were internally
benchmarking.

In my view, this patch is entirely in line with that philosophy.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] AUTHORS: Add Liliia Butorina.

[Ben Pfaff]

On Fri, May 24, 2019 at 04:32:50PM +0300, Ilya Maximets wrote:
> Signed-off-by: Ilya Maximets 

Acked-by: Ben Pfaff 

I usually just push these, since they're self-evidently correct.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] AUTHORS: Add Liliia Butorina.

[Ilya Maximets]

Signed-off-by: Ilya Maximets 
---
 AUTHORS.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/AUTHORS.rst b/AUTHORS.rst
index 7d4713bee..345abbf92 100644
--- a/AUTHORS.rst
+++ b/AUTHORS.rst
@@ -222,6 +222,7 @@ Li RongQinglirongq...@baidu.com
 Lian-min Wang  liang-min.w...@intel.com
 Lilijunjerry.lili...@huawei.com
 Lili Huang huanglili.hu...@huawei.com
+Liliia Butorinal.butor...@partner.samsung.com
 Linda Sun  l...@vmware.com
 Lior Neudorfer l...@guardicore.com
 Liu Chang  txfh2...@aliyun.com
-- 
2.17.1

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2 0/2] netdev-dpdk: Post-copy Live Migration.

[Ilya Maximets]

CC: Liliia Butorina

On 24.05.2019 16:22, Ilya Maximets wrote:
> Thanks Liliia and Maxime! Applied to master.
> 
> Best regards, Ilya Maximets.
> 
> On 14.05.2019 16:08, Ilya Maximets wrote:
>> Version 2:
>>   * Minor NEWS rebase.
>>   * Disallowed postcopy if 'mlockall' specified.
>>
>> Ilya Maximets (1):
>>   vswitchd: Track status of memory locking.
>>
>> Liliia Butorina (1):
>>   netdev-dpdk: Post-copy Live Migration support for vhost-user-client.
>>
>>  Documentation/topics/dpdk/vhost-user.rst | 53 +++-
>>  NEWS |  1 +
>>  lib/dpdk-stub.c  |  6 +++
>>  lib/dpdk.c   | 18 
>>  lib/dpdk.h   |  1 +
>>  lib/netdev-dpdk.c|  5 +++
>>  lib/util.c   | 15 +++
>>  lib/util.h   |  3 ++
>>  vswitchd/ovs-vswitchd.c  |  2 +
>>  vswitchd/vswitch.xml | 16 +++
>>  10 files changed, 119 insertions(+), 1 deletion(-)
>>
> 
> 
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2 0/2] netdev-dpdk: Post-copy Live Migration.

[Ilya Maximets]

Thanks Liliia and Maxime! Applied to master.

Best regards, Ilya Maximets.

On 14.05.2019 16:08, Ilya Maximets wrote:
> Version 2:
>   * Minor NEWS rebase.
>   * Disallowed postcopy if 'mlockall' specified.
> 
> Ilya Maximets (1):
>   vswitchd: Track status of memory locking.
> 
> Liliia Butorina (1):
>   netdev-dpdk: Post-copy Live Migration support for vhost-user-client.
> 
>  Documentation/topics/dpdk/vhost-user.rst | 53 +++-
>  NEWS |  1 +
>  lib/dpdk-stub.c  |  6 +++
>  lib/dpdk.c   | 18 
>  lib/dpdk.h   |  1 +
>  lib/netdev-dpdk.c|  5 +++
>  lib/util.c   | 15 +++
>  lib/util.h   |  3 ++
>  vswitchd/ovs-vswitchd.c  |  2 +
>  vswitchd/vswitch.xml | 16 +++
>  10 files changed, 119 insertions(+), 1 deletion(-)
> 
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC V2] netdev-rte-offloads: HW offload virtio-forwarder

[Ilya Maximets]

On 22.05.2019 15:10, Simon Horman wrote:
> Hi,
> 
> On Thu, May 16, 2019 at 08:44:31AM +, Roni Bar Yanai wrote:
>>> -Original Message-
>>> From: Ilya Maximets 
>>> Sent: Wednesday, May 15, 2019 4:37 PM
>>> To: Roni Bar Yanai ; ovs-dev@openvswitch.org; Ian
>>> Stokes ; Kevin Traynor 
>>> Cc: Eyal Lavee ; Oz Shlomo ; Eli
>>> Britstein ; Rony Efraim ; Asaf
>>> Penso 
>>> Subject: Re: [RFC V2] netdev-rte-offloads: HW offload virtio-forwarder
>>>
>>> On 15.05.2019 16:01, Roni Bar Yanai wrote:
 Hi Ilya,

 Thanks for the comment.

 I think the suggested arch is very good and has many advantages, and
 in fact I had something very similar as my internally first approach.

 However, I had one problem: it doesn't solves the kernel case. It make
 sense doing forwarding using dpdk also when OVS is kernel (port
 representor and rule offloads are done with kernel OVS). It makes
 sense because we can have one solution and because DPDK has better
 performance.
>>>
>>> I'm not sure if it makes practical sense to run separate userpace
>>> datapath just to pass packets between vhost and VF. This actually
>>> matches with some of your own disadvantages of separate DPDK apps.
>>> Separate userspace datapath will need its own complex start,
>>> configuration and maintenance. Also it will consume additional cpu cores
>>> which will not be shared with kernel packet processing.  I think that
>>> just move everything to userspace in this case would be much more simple
>>> for user than maintaining such configuration.
>>
>> Maybe It doesn't make sense for OVS-DPDK but for OVS users it does.  When
>> you run offload with OVS-kernel, and for some vendors this is the current
>> status, and virtio is a requirement, you now have millions of packets
>> that should be forwarded. Basically you have two options:
>>
>> 1. use external application (we discussed that).
>>
>> 2. create user space data plane and configure forwarding (OVS), but then
>> you have performance issues as OVS is not optimized for this. And for
>> kernel data plane much worse off course.
>>
>> Regarding burning a core. In case of HW offload you will do it either
>> way, and there is no benefit for adding FW functionality for kernel data
>> path, mainly because of kernel performance limitations.
>>
>> I agree that in such case moving to user space is a solution for some,
>> but keep in mind that some doesn't have such support for DPDK and for
>> others they have their own OVS based data path with their adjustments, so
>> it will be a hard transition.
>>
>> While arch is good for the two DPDK use cases, it leaves the kernel one
>> out.  Any thoughts how we can add this use case as well and still keep
>> the suggested arch?
> 
> ...
> 
> At Netronome we have an Open Source standalone application,
> called virtio-forwarder (https://github.com/Netronome/virtio-forwarder).
> The reason that we provide this solution is that we see this as a
> requirement for some customers. This includes customers using OVS
> with the kernel based HW offload (OVS-TC).
> 
> In general I agree that integration with OVS has some advantages and
> I'm happy to see this issue being discussed. But as we see demand
> for use of virtio-forwarder in conjunction with OVS-TC I see that
> as a requirement for a solution that is integrated with OVS, which leads
> me to lean towards the proposal put forward by Roni.
> 
> I also feel that the proposal put forward by Roni is likely to prove more
> flexible that a port-based approach, as proposed by Ilya. For one thing
> such a design ought to allow for arbitrary combinations of port types.
> In fact, it would be entirely feasible to run this in conjunction with a
> non-OVS offload aware NIC (SR-IOV in VEB mode).
> 
> Returning to the stand-alone Netronome implementation, I would welcome
> discussion of how any useful portions of this could be reused.
> 

Hi Simon. Thanks for link. It's very interesting.

My key point about the proposal put forward by Roni is that Open vSwitch
is an *OF switch* first of all and *not the multitool*. This proposal adds
some parasite work to the main OVS workflow which doesn't connected
with its main purpose. If you really want this implemented, this should
probably be done inside DPDK. You may implement a virtual device in DPDK
(like bonding) that will forward traffic between subports while calling
receive function. Adding this vdev to OVS as a usual DPDK port you will
be able to achieve your goal. DPDK as a development kit (an actual multitool)
is much more appropriate place for such solutions.

BTW, the root cause of this approach is the slow packet forwarding in OVS
in compare with direct rx + tx without parsing.
OVS performance improvement is probably the right direction where we
can move to achieve reasonably effective packet forwarding. I prepared
a patch that should allow much faster packet forwarding for direct
output flows like "in_port=1,actions=output:2". Take 

[ovs-dev] [PATCH] dpif-netdev: Forwarding optimization for direct output flows.

[Ilya Maximets]

There are some cases where users want to have simple forwarding or drop
rules for all packets received from particular port, i.e :

  "in_port=1,actions=2"
  "in_port=1,actions=IN_PORT"
  "in_port=1,actions=drop"

There are also cases where complex OF flows could be simplified down
to simple forwarding/drop datapath flows.

In theory, we don't need to parse packets at all to follow these flows.
"Direct output forwarding" optimization is intended to speed up above
cases.

Design:

Due to various implementation restrictions userspace datapath has
following flow fields always in exact match (i.e. it's required to
match at least these fields of a packet even if the OF rule doesn't
need that):

  - recirc_id
  - in_port
  - packet_type
  - dl_type
  - vlan_tci
  - nw_frag (for ip packets)

Not all of these fields are related to packet itself. We already
know the current 'recirc_id' and the 'in_port' before starting the
packet processing. It also seems safe to assume that we're working
with Ethernet packets. dpif-netdev sets exact match on 'vlan_tci'
to avoid issues with flow format conversion and we don't really need
to match with it until ofproto layer didn't ask us to.
So, for the simple forwarding OF rule we need to match only with
'dl_type' and 'nw_frag'.

'in_port', 'dl_type' and 'nw_frag' could be combined in a single
64bit integer that could be used as a hash in hash map.

New per-PMD flow table 'direct_output_table' introduced to store
direct output flows only. 'dp_netdev_flow_add' adds flow to the
usual 'flow_table' and to 'direct_output_table' if the flow meets
following constraints:

  - 'recirc_id' in flow match is 0.
  - Flow wildcards originally had wildcarded 'vlan_tci'.
  - Flow has no actions (drop) or exactly one action equal to
OVS_ACTION_ATTR_OUTPUT.
  - Flow wildcards contains only minimal set of non-wildcarded fields
(listed above).

If the number of flows for current 'in_port' in regular 'flow_table'
equals number of flows for current 'in_port' in 'direct_output_table',
we may use direct output optimization, because all the flows we have
are direct output flows. This means that we only need to parse
'dl_type' and 'nw_frag' to perform packet matching.
Now we making the unique flow mark from the 'in_port', 'dl_type' and
'nw_frag' and looking for it in 'direct_output_table'.
On successful lookup we don't need to make full 'miniflow_extract()'.

Unsuccessful lookup technically means that we have no sufficient flow
in datapath and upcall will be required. We may optimize this path
in the future by bypassing the EMC, SMC and dpcls lookups in this case.

Performance improvement of this solution on a 'direct output' flows
should be comparable with partial HW offloading, because it parses same
packet fields and uses similar flow lookup scheme.
However, unlike partial HW offloading, it works for all port types
including virtual ones.

Signed-off-by: Ilya Maximets 
---

This patch was made as a point for "virtio-forwarder" discussion:
   https://mail.openvswitch.org/pipermail/ovs-dev/2019-May/358686.html
However, it might be very useful by itself for usual cases too.

Testing is very welcome. I didn't run the performance tests on real
systems, so I don't know the real performance impact yet.

 lib/dpif-netdev.c | 257 +++---
 lib/flow.c|  10 +-
 lib/flow.h|   3 +-
 3 files changed, 251 insertions(+), 19 deletions(-)

diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index 5a6f2abac..b455bdc7b 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -34,6 +34,7 @@
 
 #include "bitmap.h"
 #include "cmap.h"
+#include "ccmap.h"
 #include "conntrack.h"
 #include "coverage.h"
 #include "ct-dpif.h"
@@ -530,6 +531,8 @@ struct dp_netdev_flow {
 /* Hash table index by unmasked flow. */
 const struct cmap_node node; /* In owning dp_netdev_pmd_thread's */
  /* 'flow_table'. */
+const struct cmap_node direct_output_node; /* In dp_netdev_pmd_thread's
+ 'direct_output_table'. */
 const struct cmap_node mark_node; /* In owning flow_mark's mark_to_flow */
 const ovs_u128 ufid; /* Unique flow identifier. */
 const ovs_u128 mega_ufid;/* Unique mega flow identifier. */
@@ -543,7 +546,8 @@ struct dp_netdev_flow {
 struct ovs_refcount ref_cnt;
 
 bool dead;
-uint32_t mark;   /* Unique flow mark assigned to a flow */
+uint32_t mark;   /* Unique flow mark for netdev offloading. */
+uint64_t direct_output_mark; /* Unique flow mark for direct output. */
 
 /* Statistics. */
 struct dp_netdev_flow_stats stats;
@@ -658,12 +662,19 @@ struct dp_netdev_pmd_thread {
 
 /* Flow-Table and classifiers
  *
- * Writers of 'flow_table' must take the 'flow_mutex'.  Corresponding
- * changes to 'classifiers' must be made while still holding the
- * 'flow_mutex'.
+ * Writers of 

Re: [ovs-dev] [PATCH v9] Improved Packet Drop Statistics in OVS

[Eelco Chaudron]

Hi Anju,

Was there ever a follow up on this patch? I only see one response from 
Ilya on this asking about his v8 comments.


Thanks,

Eelco


On 27 Feb 2019, at 10:22, Anju Thomas wrote:

Currently OVS maintains explicit packet drop/error counters only on 
port
level. Packets that are dropped as part of normal OpenFlow processing 
are
counted in flow stats of “drop” flows or as table misses in table 
stats.
These can only be interpreted by controllers that know the semantics 
of
the configured OpenFlow pipeline. Without that knowledge, it is 
impossible
for an OVS user to obtain e.g. the total number of packets dropped due 
to

OpenFlow rules.

Furthermore, there are numerous other reasons for which packets can be
dropped by OVS slow path that are not related to the OpenFlow 
pipeline.
The generated datapath flow entries include a drop action to avoid 
further
expensive upcalls to the slow path, but subsequent packets dropped by 
the

datapath are not accounted anywhere.

Finally, the datapath itself drops packets in certain error 
situations.

Also, these drops are today not accounted for.

This makes it difficult for OVS users to monitor packet drop in an OVS
instance and to alert a management system in case of a unexpected 
increase
of such drops. Also OVS trouble-shooters face difficulties in 
analysing

packet drops.

With this patch we implement following changes to address the issues
mentioned above.

1. Identify and account all the silent packet drop scenarios

2. Display these drops in ovs-appctl coverage/show

A detailed presentation on this was presented at OvS conference 2017 
and

link for the corresponding presentation is available at:

https://www.slideshare.net/LF_OpenvSwitch/lfovs17troubleshooting-the-data-plane-in-ovs-82280329

Co-authored-by: Rohith Basavaraja 
Co-authored-by: Keshav Gupta 
Signed-off-by: Anju Thomas 
Signed-off-by: Rohith Basavaraja 
Signed-off-by: Keshav Gupta 
---
 datapath/linux/compat/include/linux/openvswitch.h |   1 +
 lib/dpif-netdev.c |  44 -
 lib/dpif.c|   7 +
 lib/dpif.h|   3 +
 lib/odp-execute.c |  81 -
 lib/odp-util.c|   9 +
 ofproto/ofproto-dpif-ipfix.c  |   1 +
 ofproto/ofproto-dpif-sflow.c  |   1 +
 ofproto/ofproto-dpif-xlate.c  | 103 +++
 ofproto/ofproto-dpif-xlate.h  |   3 +
 ofproto/ofproto-dpif.c|   8 +
 ofproto/ofproto-dpif.h|   7 +-
 tests/automake.mk |   3 +-
 tests/dpif-netdev.at  |   8 +
 tests/drop-stats.at   | 197 
++

 tests/ofproto-dpif.at |   2 +-
 tests/testsuite.at|   1 +
 tests/tunnel-push-pop.at  |  28 ++-
 tests/tunnel.at   |  14 +-
 19 files changed, 475 insertions(+), 46 deletions(-)
 create mode 100644 tests/drop-stats.at

diff --git a/datapath/linux/compat/include/linux/openvswitch.h 
b/datapath/linux/compat/include/linux/openvswitch.h

index d5aa09d..e77e9c8 100644
--- a/datapath/linux/compat/include/linux/openvswitch.h
+++ b/datapath/linux/compat/include/linux/openvswitch.h
@@ -946,6 +946,7 @@ enum ovs_action_attr {
OVS_ACTION_ATTR_POP_NSH,  /* No argument. */
OVS_ACTION_ATTR_METER,/* u32 meter number. */
OVS_ACTION_ATTR_CLONE,/* Nested OVS_CLONE_ATTR_*.  */
+   OVS_ACTION_ATTR_DROP, /* Drop action. */

 #ifndef __KERNEL__
OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index 77ac1d2..acc7913 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -101,6 +101,17 @@ enum { MAX_METERS = 65536 };/* Maximum number 
of meters. */
 enum { MAX_BANDS = 8 }; /* Maximum number of bands / meter. 
*/

 enum { N_METER_LOCKS = 64 };/* Maximum number of meters. */

+COVERAGE_DEFINE(datapath_drop_meter);
+COVERAGE_DEFINE(datapath_drop_upcall_error);
+COVERAGE_DEFINE(datapath_drop_lock_error);
+COVERAGE_DEFINE(datapath_drop_userspace_action_error);
+COVERAGE_DEFINE(datapath_drop_tunnel_push_error);
+COVERAGE_DEFINE(datapath_drop_tunnel_pop_error);
+COVERAGE_DEFINE(datapath_drop_recirc_error);
+COVERAGE_DEFINE(datapath_drop_invalid_port);
+COVERAGE_DEFINE(datapath_drop_invalid_tnl_port);
+COVERAGE_DEFINE(datapath_drop_rx_invalid_packet);
+
 /* Protects against changes to 'dp_netdevs'. */
 static struct ovs_mutex dp_netdev_mutex = OVS_MUTEX_INITIALIZER;

@@ -5647,6 +5658,7 @@ dp_netdev_run_meter(struct dp_netdev *dp, struct 
dp_packet_batch *packets_,

 band->packet_count += 1;
 band->byte_count += dp_packet_size(packet);

+

[ovs-dev] Descuentos en Viajes en Argentina

[TRAVEL CUPON Argentina]

( https://www.travelcupon.com )
 
Disfruta de viajar al mejor precio
 
LA ANGOSTURA
  
VILLA GESELL
  
VILLA GESELL
  
VILLA GESELL
  
( http://www.travelcupon.com/delcaballero.html )    
( http://www.travelcupon.com/TEQUENDAMA%203.html )    
( http://www.travelcupon.com/TEQUENDAMA%203.html )    
( http://www.travelcupon.com/TEQUENDAMA%203.html )   
               
Paga $ 9105.- Antes $ 16554.- 7 dias para 2 con desayuno !!

 
Paga $ 11937.- 3 noches para 3 con desayuno !!
 
Paga $ 13542.- 3 noches para 4 con desayuno !!
 
Paga $ 10563.- 3 noches para 2 con desayuno !!
  
.              
VER MAS ( http://www.travelcupon.com/delcaballero.html )
  
VER MAS ( http://www.travelcupon.com/TEQUENDAMA%202.html )
  
VER MAS ( http://www.travelcupon.com/TEQUENDAMA%203.html )
  
VER MAS ( http://www.travelcupon.com/TEQUENDAMA.html )
  
.              
VILLA GESELL
  
PLAYA
  
PLAYA
  
VICTORIA
  
( http://www.travelcupon.com/TEQUENDAMA%203.html )    
( http://www.travelcupon.com/Mar%20Abierto.html )    
( http://www.travelcupon.com/Mar%20Abierto.html )    
( http://www.travelcupon.com/Hotel%20America%20Victoria.html )   
               
Paga $ 10563.- 3 noches para 2 con desayuno en Hab. Superior !!
 
Santa Clara del mar Promocion para 2-3 Pasajeros !!
 
Santa Clara del mar Promocion para 4-5 Pasajeros !!
 
Victoria Promocion para 2-3 o 4 Pasajeros Disfruta todo el año!!
  
.              
VER MAS ( http://www.travelcupon.com/TEQUENDAMA%204.html )
  
VER MAS ( http://www.travelcupon.com/Mar%20Abierto.html )
  
VER MAS ( http://www.travelcupon.com/Mar%20Abierto%202.html )
  
VER MAS ( http://www.travelcupon.com/Hotel%20America%20Victoria.html )
  
.              
FEDERACION
  
SALTA
  
SALTA
  
CORDOBA
  
( http://www.travelcupon.com/Federacion%20Termal.html )    
( http://www.travelcupon.com/Posada%20de%20las%20Nubes2.html )    
( http://www.travelcupon.com/Posada%20de%20las%20Nubes2.html )    
( http://www.travelcupon.com/El%20Pungo.html )   
.              
Federacion 35 % para 2-3 o 4 Pasajeros Disfruta todo el año!!
 
Salta 20% Off para 4 o 5 Pasajeros Cupon Especial!!
 
Salta 20% Off para 2 o 3 Pasajeros Cupon Especial!!
  
Cabaña Los Troncos 4 personas 5-7 dias desde $ 15246.- Todo el Año
  
.              
VER MAS ( http://www.travelcupon.com/Federacion%20Termal.html )
  
VER MAS ( http://www.travelcupon.com/Posada%20de%20las%20Nubes2.html )
  
VER MAS ( http://www.travelcupon.com/Posada%20de%20las%20Nubes.html )
  
VER MAS ( http://www.travelcupon.com/El%20Pungo.html )
  
               
CORDOBA
  
CORDOBA
  
CORDOBA
  
CORDOBA
  
( http://www.travelcupon.com/El%20Pungo.html )    
( http://www.travelcupon.com/El%20Pungo.html )    
( http://www.travelcupon.com/El%20Pungo.html )    
( http://www.travelcupon.com/El%20Pungo.html )   
               
Cabaña del Mirador 4 personas 5-7 dias desde $ 13248.- Todo el Año
 
Cabaña Alpina 4 personas 5-7 dias desde $ 13248.- Todo el Año
 
Apart Pink 2 personas 5-7 dias desde $ 8712.- Todo el Año
 
Apart Aromo 2 personas 5-7 dias desde $ 8112.- Todo el Año
  
               
VER MAS ( http://www.travelcupon.com/El%20Pungo%202.html )
 
VER MAS ( http://www.travelcupon.com/El%20Pungo%203.html )
 
VER MAS ( http://www.travelcupon.com/El%20Pungo%204.html )
 
VER MAS ( http://www.travelcupon.com/El%20Pungo%205.html )
  
               
Registrate en nuestro sitio y obtene importantes Descuentos en Turismo

.
REGISTRATE ( https://www.travelcupon.com )
.
( https://www.facebook.com/Travel-Cupon-1103950346425310/?modal=admin_todo_tour 
)
( https://www.instagram.com/travel_cupon/?hl=es-la )
( https://twitter.com/viajes72 )
.
www.travelcupon.com ( https://www.travelcupon.com )
Este es un e-mail legal,  y contiene informacion de servicios y productos 
que consideramos pueden ser de su interes.
De acuerdo con la nueva Ley argentina Nro. 26.032, la libre distribucion de 
este e-mail esta
autorizada por tratarse de propositos de informacion, sin embargo, si le 
hemos causado alguna molestia por el mismo, le rogamos acepte nuestras 
disculpas.

Si lo desea puede ser quitado de nuestra lista de correos
solicitandolo a baj...@yopmail.com
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] Libretas Personalizadas con tu Marca desde $19,90

[FINEART EMPRESAS]

 
 
 
  LIBRETAS PERSONALIZADAS   
 
 
desde $19.90 +IVA c/u
 
 
 
 
 
 
 
 
 
PEDÍ LAS TUYAS ( https://www.fineart.com.ar/notas-personalizadas/ )
 
 
Considere nuestras libretas para su próximo evento, conferencia o 
iniciativa de marketing.

 
 
CARACTERÍSTICAS
TAMAÑOS: 14.5cm x 9cm | 21cm x 15cm
 
48 PÁGINAS LISAS BLANCAS (80grs)
 
ACABALLADO PARA APERTURA 180º
 
TAPAS IMPRESAS: ILUSTRACIÓN O KRAFT
 
TERMINACIÓN EN POLIPROPILENO MATE
 
 
 
CONTACTO
 
Balcarce 225 · Ramos Mejía
(54 11) 4653.6260
 
 
( https://www.fineart.com.ar/ )
 
 
 
Este es un e-mail legal y contiene informacion de servicios y productos que 
consideramos pueden ser de su interes.
De acuerdo con la nueva Ley argentina Nro. 26.032, la libre distribucion de 
este e-mail esta
autorizada por tratarse de propositos de informacion, sin embargo, si le 
hemos causado alguna molestia por el mismo, le rogamos acepte nuestras 
disculpas.

Si lo desea puede ser quitado de nuestra lista de correos solicitandolo a 
baj...@yopmail.com
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] Vacaciones en Brasil 2019

[Hotel Parador da Cachoeira]

 
 

Haga su Reserva Directo al Hotel

Descuentos en Reservas Anticipadas


 
( http://www.hotelparador.com.br/contato/ )
Hotel Parador da Cachoeira - Florianópolis
El Hotel Parador da Cachoeira fue cuidadosamente pensado para proporcionar 
confort y bienestar en un ambiente armonioso y bien decorado.

Localizado en la playa de Cachoeira do Bom Jesus, una de las más bellas del 
norte de la isla de Florianópolis, el Hotel Parador da Cachoeira, a través 
de su infraestructura y equipo de profesionales, ofrece lo que tiene de 
mejor: hospitalidad, descanso y entretenimiento.

 
 

( http://www.hotelparador.com.br/contato/ )
 
SERVICIOS
* Bar (alta temporada)
* Piscinas adulto e infantil
* Salas de estar, TV y juegos (metegol y ping pong)
* Playground externo para chicos
* Desayuno buffet
* Sillas y sombrilla
* Internet wi fi
* Reservas de paseos, transfer y servicios
* Garage cubierto (lugares rotativos) y estacionamiento

 
+ Info ( http://www.hotelparador.com.br/contato/ )
 

( http://www.hotelparador.com.br/apartamentos/standard/ )
STANDAR
El apartamento estándar tiene una habitación y un baño que alberga con 
confort hasta dos personas y garantiza excelentes momentos de descanso 
durante su estancia en negocios o placer..
 
 
+ Info ( http://www.hotelparador.com.br/apartamentos/standard/ )
 
( http://www.hotelparador.com.br/apartamentos/suite/ )
SUITE
El apartamento Suite tiene dormitorio, baño, living/comedor y mini cocina. 
Hospeda hasta 4 personas. La mini cocina equipada con vajilla basica, 
cocina eléctrica, microondas y refrigerador posibilita la libertad de 
preparar comidas en cualquier momento y la barbacoa en el balcón 
proporciona momentos de ocio y confraternización.
+ Info ( http://www.hotelparador.com.br/apartamentos/suite/ )
 

( http://www.hotelparador.com.br/apartamentos/suite-familia/ )
SUITE FAMILIA
El apartamento de la familia es perfecto para alojar a una familia 
completa. Con dos dormitorios, dos baños, living/comedor y mini cocina, 
ofrece confort para hasta 6 personas. La mini cocina equipada con vajilla 
basica, cocina eléctrica, microondas y refrigerador posibilita la libertad 
de preparar comidas en cualquier momento y la barbacoa en el balcón 
proporciona momentos de ocio y confraternización.
 
+ Info ( http://www.hotelparador.com.br/apartamentos/suite-familia/ )
 
( http://www.hotelparador.com.br/apartamentos/suite-cobertura/ )
SUITE COBERTURA
El apartamento Suite Cobertura es perfecto para aquellos que desean vistas 
panorámicas. Con una habitación, baño,living/comedor y mini cocina, alberga 
con confort hasta 4 personas. La mini cocina equipada con vajilla basica, 
cocina eléctrica, microondas y refrigerador permite la libertad de preparar 
comidas en cualquier momento. La amplia terraza tiene tumbonas, amaca, mesa 
y sillas externa junto a la barbacoa para proporcionar momentos de ocio y 
confraternización.
+ Info ( http://www.hotelparador.com.br/apartamentos/suite-cobertura/ )
 

 

 

( http://www.hotelparador.com.br/contato/ )
 
( http://www.hotelparador.com.br/contato/ )
 
CONSULTAS ( http://www.hotelparador.com.br/contato/ )
( http://www.hotelparador.com.br/contato/ )

 
 

 

 

Este es un e-mail legal y contiene informacion de servicios y productos que 
consideramos pueden ser de su interes.
De acuerdo con la nueva Ley argentina Nro. 26.032, la libre distribucion de 
este e-mail esta
autorizada por tratarse de propositos de informacion, sin embargo, si le 
hemos causado alguna molestia por el mismo, le rogamos acepte nuestras 
disculpas.

Si lo desea puede ser quitado de nuestra lista de correos solicitandolo a 
baj...@yopmail.com

 

 

 

 

 
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev