Re: [ovs-dev] Scale testing OVN with ovn-heater for OpenStack use cases

[Daniel Alvarez Sanchez]

Hey Frode, Dumitru, thanks a lot for initiating this

On Thu, May 11, 2023 at 6:05 PM Frode Nordahl 
wrote:

> Hello, Dumitru, Daniel and team,
>
> On Thu, May 11, 2023 at 5:23 PM Dumitru Ceara  wrote:
> >
> > Hi Frode,
> >
> > During an internal discussion with RedHat OpenStack folks (Daniel in cc)
> > about scalability of OVN in OpenStack deployments I remembered that you
> > mentioned that you're thinking of enhancing ovn-heater in order to test
> > your types of deployments [0].  I assumed that those are also OpenStack
> > based deployments so I was wondering if there's an opportunity for
> > collaboration here.
>
> Thank you for reaching out to me on this topic. We do indeed have scale
> testing in the context of OpenStack on our roadmap and welcome the
> invitation to collaborate on this.
>
> As you know we did some preparation work together already as you refer
> to in [0], and we thank you for your help in upstreaming ovn-heater.
>
> We have yet to schedule exactly when in the cycle to work on this, I
> guess a next step could be to set up a meet to share some ideas and
> see how it aligns?


> If that resonates with you, I'd be happy to organize.
>

Sounds good, please do :)


>
> > Having OpenStack-like scenarios in ovn-heater would really allow OVN
> > developers to profile/troubleshoot/optimize OVN for those use cases too.
> >
> > Well, in general, the above is true for any other CMS too.
>
> Yes, this is what we have been thinking as well. We are supporting other
> products in their journey with OVN as well, so similar projects may come
> from those in the future.
>

Yes, we can try to model OpenStack-like scenarios for ovn-heater where the
resources are generally more distributed across the hypervisors than in k8s
for example, perhaps larger L2 domain sizes, etcetera.
Happy to discuss further!

daniel


> --
> Frode Nordahl
>
> > Thanks,
> > Dumitru
> >
> > [0]
> https://mail.openvswitch.org/pipermail/ovs-dev/2023-March/402680.html
> >
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v3] db-ctl-base: Partially revert b8bf410a5

[Daniel Alvarez Sanchez]

The commit b8bf410a5 [0] broke the `ovs-vsctl add` command
which now overwrites the value if it existed already.

This patch reverts the code around the `cmd_add` function
to restore the previous behavior. It also adds testing coverage
for this functionality.

[0]
https://github.com/openvswitch/ovs/commit/b8bf410a5c94173da02279b369d75875c4035959

Fixes: b8bf410a5 ("db-ctl-base: Use partial map/set updates for last
add/set commands")
Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2182767
Signed-off-by: Daniel Alvarez Sanchez 
---

Notes:
  * v2: fix failing test
  * v3: fix formatting and adding 'Fixes' to the commit message

 lib/db-ctl-base.c  | 42 +-
 tests/ovs-vsctl.at |  8 +++-
 2 files changed, 12 insertions(+), 38 deletions(-)

diff --git a/lib/db-ctl-base.c b/lib/db-ctl-base.c
index 134496ef3..5d2635946 100644
--- a/lib/db-ctl-base.c
+++ b/lib/db-ctl-base.c
@@ -1492,7 +1492,7 @@ cmd_add(struct ctl_context *ctx)
 const struct ovsdb_idl_column *column;
 const struct ovsdb_idl_row *row;
 const struct ovsdb_type *type;
-struct ovsdb_datum new;
+struct ovsdb_datum old;
 int i;

 ctx->error = get_table(table_name, );
@@ -1516,13 +1516,7 @@ cmd_add(struct ctl_context *ctx)
 }

 type = >type;
-
-if (ctx->last_command) {
-ovsdb_datum_init_empty();
-} else {
-ovsdb_datum_clone(, ovsdb_idl_read(row, column));
-}
-
+ovsdb_datum_clone(, ovsdb_idl_read(row, column));
 for (i = 4; i < ctx->argc; i++) {
 struct ovsdb_type add_type;
 struct ovsdb_datum add;
@@ -1533,41 +1527,23 @@ cmd_add(struct ctl_context *ctx)
 ctx->error = ovsdb_datum_from_string(, _type, ctx->argv[i],
  ctx->symtab);
 if (ctx->error) {
-ovsdb_datum_destroy(, >type);
+ovsdb_datum_destroy(, >type);
 return;
 }
-ovsdb_datum_union(, , type);
+ovsdb_datum_union(, , type);
 ovsdb_datum_destroy(, type);
 }
-
-if (!ctx->last_command && new.n > type->n_max) {
+if (old.n > type->n_max) {
 ctl_error(ctx, "\"add\" operation would put %u %s in column %s of "
   "table %s but the maximum number is %u",
-  new.n,
+  old.n,
   type->value.type == OVSDB_TYPE_VOID ? "values" : "pairs",
   column->name, table->name, type->n_max);
-ovsdb_datum_destroy(, >type);
+ovsdb_datum_destroy(, >type);
 return;
 }
-
-if (ctx->last_command) {
-/* Partial updates can only be made one by one. */
-for (i = 0; i < new.n; i++) {
-struct ovsdb_datum *datum = xmalloc(sizeof *datum);
-
-ovsdb_datum_init_empty(datum);
-ovsdb_datum_add_from_index_unsafe(datum, , i, type);
-if (ovsdb_type_is_map(type)) {
-ovsdb_idl_txn_write_partial_map(row, column, datum);
-} else {
-ovsdb_idl_txn_write_partial_set(row, column, datum);
-}
-}
-ovsdb_datum_destroy(, >type);
-} else {
-ovsdb_idl_txn_verify(row, column);
-ovsdb_idl_txn_write(row, column, );
-}
+ovsdb_idl_txn_verify(row, column);
+ovsdb_idl_txn_write(row, column, );

 invalidate_cache(ctx);
 }
diff --git a/tests/ovs-vsctl.at b/tests/ovs-vsctl.at
index a92156f00..a368bff6e 100644
--- a/tests/ovs-vsctl.at
+++ b/tests/ovs-vsctl.at
@@ -425,6 +425,7 @@ AT_CHECK([RUN_OVS_VSCTL_ONELINE(
   [add-port a a1],
   [add-bond a bond0 a2 a3],
   [br-set-external-id a key0 value0],
+  [add Bridge a external_ids key0=value1],
   [set port a1 external-ids:key1=value1],
   [set interface a2 external-ids:key2=value2],
   [set interface a2 external-ids:key3=value3],
@@ -446,6 +447,7 @@ AT_CHECK([RUN_OVS_VSCTL_ONELINE(



+
 key0=value0
 value0

@@ -1071,13 +1073,9 @@ AT_CHECK([RUN_OVS_VSCTL([set controller br1
'connection-mode=xyz'])],
 AT_CHECK([RUN_OVS_VSCTL([set controller br1 connection-mode:x=y])],
   [1], [], [ovs-vsctl: cannot specify key to set for non-map column
connection_mode
 ])
-AT_CHECK([RUN_OVS_VSCTL([add bridge br1 datapath_id x y -- show])],
+AT_CHECK([RUN_OVS_VSCTL([add bridge br1 datapath_id x y])],
   [1], [], [ovs-vsctl: "add" operation would put 2 values in column
datapath_id of table Bridge but the maximum number is 1
 ])
-AT_CHECK([RUN_OVS_VSCTL([add bridge br1 datapath_id x y])], [1], [],
[stderr])
-AT_CHECK([sed "/^.*|WARN|.*/d" < stderr], [0], [dnl
-ovs-vsctl: transaction error: {"details":"set must have 0 to 1 members but
2 are present","error":"syntax error","syntax":"[[\"set\",[\"x\",\

[ovs-dev] [PATCH v2] db-ctl-base: Partially revert b8bf410a5

[Daniel Alvarez Sanchez]

The commit b8bf410a5 [0] broke the `ovs-vsctl add` command
which now overwrites the value if it existed already.

This patch reverts the code around the `cmd_add` function
to restore the previous behavior. It also adds testing coverage
for this functionality.

[0]
https://github.com/openvswitch/ovs/commit/b8bf410a5c94173da02279b369d75875c4035959

Signed-off-by: Daniel Alvarez Sanchez 
Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2182767
---
Notes:
  v1 -> v2
  - Fix failing test

 lib/db-ctl-base.c  | 42 +-
 tests/ovs-vsctl.at |  8 +++-
 2 files changed, 12 insertions(+), 38 deletions(-)

diff --git a/lib/db-ctl-base.c b/lib/db-ctl-base.c
index 134496ef3..5d2635946 100644
--- a/lib/db-ctl-base.c
+++ b/lib/db-ctl-base.c
@@ -1492,7 +1492,7 @@ cmd_add(struct ctl_context *ctx)
 const struct ovsdb_idl_column *column;
 const struct ovsdb_idl_row *row;
 const struct ovsdb_type *type;
-struct ovsdb_datum new;
+struct ovsdb_datum old;
 int i;

 ctx->error = get_table(table_name, );
@@ -1516,13 +1516,7 @@ cmd_add(struct ctl_context *ctx)
 }

 type = >type;
-
-if (ctx->last_command) {
-ovsdb_datum_init_empty();
-} else {
-ovsdb_datum_clone(, ovsdb_idl_read(row, column));
-}
-
+ovsdb_datum_clone(, ovsdb_idl_read(row, column));
 for (i = 4; i < ctx->argc; i++) {
 struct ovsdb_type add_type;
 struct ovsdb_datum add;
@@ -1533,41 +1527,23 @@ cmd_add(struct ctl_context *ctx)
 ctx->error = ovsdb_datum_from_string(, _type, ctx->argv[i],
  ctx->symtab);
 if (ctx->error) {
-ovsdb_datum_destroy(, >type);
+ovsdb_datum_destroy(, >type);
 return;
 }
-ovsdb_datum_union(, , type);
+ovsdb_datum_union(, , type);
 ovsdb_datum_destroy(, type);
 }
-
-if (!ctx->last_command && new.n > type->n_max) {
+if (old.n > type->n_max) {
 ctl_error(ctx, "\"add\" operation would put %u %s in column %s of "
   "table %s but the maximum number is %u",
-  new.n,
+  old.n,
   type->value.type == OVSDB_TYPE_VOID ? "values" : "pairs",
   column->name, table->name, type->n_max);
-ovsdb_datum_destroy(, >type);
+ovsdb_datum_destroy(, >type);
 return;
 }
-
-if (ctx->last_command) {
-/* Partial updates can only be made one by one. */
-for (i = 0; i < new.n; i++) {
-struct ovsdb_datum *datum = xmalloc(sizeof *datum);
-
-ovsdb_datum_init_empty(datum);
-ovsdb_datum_add_from_index_unsafe(datum, , i, type);
-if (ovsdb_type_is_map(type)) {
-ovsdb_idl_txn_write_partial_map(row, column, datum);
-} else {
-ovsdb_idl_txn_write_partial_set(row, column, datum);
-}
-}
-ovsdb_datum_destroy(, >type);
-} else {
-ovsdb_idl_txn_verify(row, column);
-ovsdb_idl_txn_write(row, column, );
-}
+ovsdb_idl_txn_verify(row, column);
+ovsdb_idl_txn_write(row, column, );

 invalidate_cache(ctx);
 }
diff --git a/tests/ovs-vsctl.at b/tests/ovs-vsctl.at
index a92156f00..a368bff6e 100644
--- a/tests/ovs-vsctl.at
+++ b/tests/ovs-vsctl.at
@@ -425,6 +425,7 @@ AT_CHECK([RUN_OVS_VSCTL_ONELINE(
   [add-port a a1],
   [add-bond a bond0 a2 a3],
   [br-set-external-id a key0 value0],
+  [add Bridge a external_ids key0=value1],
   [set port a1 external-ids:key1=value1],
   [set interface a2 external-ids:key2=value2],
   [set interface a2 external-ids:key3=value3],
@@ -446,6 +447,7 @@ AT_CHECK([RUN_OVS_VSCTL_ONELINE(



+
 key0=value0
 value0

@@ -1071,13 +1073,9 @@ AT_CHECK([RUN_OVS_VSCTL([set controller br1
'connection-mode=xyz'])],
 AT_CHECK([RUN_OVS_VSCTL([set controller br1 connection-mode:x=y])],
   [1], [], [ovs-vsctl: cannot specify key to set for non-map column
connection_mode
 ])
-AT_CHECK([RUN_OVS_VSCTL([add bridge br1 datapath_id x y -- show])],
+AT_CHECK([RUN_OVS_VSCTL([add bridge br1 datapath_id x y])],
   [1], [], [ovs-vsctl: "add" operation would put 2 values in column
datapath_id of table Bridge but the maximum number is 1
 ])
-AT_CHECK([RUN_OVS_VSCTL([add bridge br1 datapath_id x y])], [1], [],
[stderr])
-AT_CHECK([sed "/^.*|WARN|.*/d" < stderr], [0], [dnl
-ovs-vsctl: transaction error: {"details":"set must have 0 to 1 members but
2 are present","error":"syntax error","syntax":"[[\"set\",[\"x\",\"y\"]]]"}
-])
 AT_CHECK([RUN_OVS_VSCTL([remove netflow `cat netflow-uuid` targets '"
1.2.3.4:567"'])],
   [1], [], [ovs-vsctl: "remove" operation would put 0 values in column
targets of table NetFlow but the minimum number is 1
 ])
--
2.39.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] db-ctl-base: Partially revert b8bf410a5

[Daniel Alvarez Sanchez]

The commit b8bf410a5 [0] broke the `ovs-vsctl add` command
which now overwrites the value if it existed already.

This patch reverts the code around the `cmd_add` function
to restore the previous behavior. It also adds testing coverage
for this functionality.

[0]
https://github.com/openvswitch/ovs/commit/b8bf410a5c94173da02279b369d75875c4035959

Signed-off-by: Daniel Alvarez Sanchez 
Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2182767
---
 lib/db-ctl-base.c  | 42 +-
 tests/ovs-vsctl.at |  2 ++
 2 files changed, 11 insertions(+), 33 deletions(-)

diff --git a/lib/db-ctl-base.c b/lib/db-ctl-base.c
index 134496ef3..5d2635946 100644
--- a/lib/db-ctl-base.c
+++ b/lib/db-ctl-base.c
@@ -1492,7 +1492,7 @@ cmd_add(struct ctl_context *ctx)
 const struct ovsdb_idl_column *column;
 const struct ovsdb_idl_row *row;
 const struct ovsdb_type *type;
-struct ovsdb_datum new;
+struct ovsdb_datum old;
 int i;

 ctx->error = get_table(table_name, );
@@ -1516,13 +1516,7 @@ cmd_add(struct ctl_context *ctx)
 }

 type = >type;
-
-if (ctx->last_command) {
-ovsdb_datum_init_empty();
-} else {
-ovsdb_datum_clone(, ovsdb_idl_read(row, column));
-}
-
+ovsdb_datum_clone(, ovsdb_idl_read(row, column));
 for (i = 4; i < ctx->argc; i++) {
 struct ovsdb_type add_type;
 struct ovsdb_datum add;
@@ -1533,41 +1527,23 @@ cmd_add(struct ctl_context *ctx)
 ctx->error = ovsdb_datum_from_string(, _type, ctx->argv[i],
  ctx->symtab);
 if (ctx->error) {
-ovsdb_datum_destroy(, >type);
+ovsdb_datum_destroy(, >type);
 return;
 }
-ovsdb_datum_union(, , type);
+ovsdb_datum_union(, , type);
 ovsdb_datum_destroy(, type);
 }
-
-if (!ctx->last_command && new.n > type->n_max) {
+if (old.n > type->n_max) {
 ctl_error(ctx, "\"add\" operation would put %u %s in column %s of "
   "table %s but the maximum number is %u",
-  new.n,
+  old.n,
   type->value.type == OVSDB_TYPE_VOID ? "values" : "pairs",
   column->name, table->name, type->n_max);
-ovsdb_datum_destroy(, >type);
+ovsdb_datum_destroy(, >type);
 return;
 }
-
-if (ctx->last_command) {
-/* Partial updates can only be made one by one. */
-for (i = 0; i < new.n; i++) {
-struct ovsdb_datum *datum = xmalloc(sizeof *datum);
-
-ovsdb_datum_init_empty(datum);
-ovsdb_datum_add_from_index_unsafe(datum, , i, type);
-if (ovsdb_type_is_map(type)) {
-ovsdb_idl_txn_write_partial_map(row, column, datum);
-} else {
-ovsdb_idl_txn_write_partial_set(row, column, datum);
-}
-}
-ovsdb_datum_destroy(, >type);
-} else {
-ovsdb_idl_txn_verify(row, column);
-ovsdb_idl_txn_write(row, column, );
-}
+ovsdb_idl_txn_verify(row, column);
+ovsdb_idl_txn_write(row, column, );

 invalidate_cache(ctx);
 }
diff --git a/tests/ovs-vsctl.at b/tests/ovs-vsctl.at
index a92156f00..03786a6ff 100644
--- a/tests/ovs-vsctl.at
+++ b/tests/ovs-vsctl.at
@@ -425,6 +425,7 @@ AT_CHECK([RUN_OVS_VSCTL_ONELINE(
   [add-port a a1],
   [add-bond a bond0 a2 a3],
   [br-set-external-id a key0 value0],
+  [add Bridge a external_ids key0=value1],
   [set port a1 external-ids:key1=value1],
   [set interface a2 external-ids:key2=value2],
   [set interface a2 external-ids:key3=value3],
@@ -446,6 +447,7 @@ AT_CHECK([RUN_OVS_VSCTL_ONELINE(



+
 key0=value0
 value0

--
2.39.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] northd: Make the use of common zone in NAT configurable

[Daniel Alvarez Sanchez]

Thanks a lot Ales!

On Tue, Feb 7, 2023 at 9:08 AM Ales Musil  wrote:

> There are essentially three problems with the current
> combination of DGP + SNAT + LB:
>
> 1) The first packet is being SNATed in common zone due
> to a problem with pinctrl not preserving ct_mark/ct_label.
> The commit would create a SNAT entry within the same with DNAT
> entry.
>
> 2) The unSNAT for reply always happened in common zone because of
> the loopback check which would be triggered only when we loop
> the packet through the LR. Now there are two possibilities how
> the reply packet would be handled:
>
> a) If the entry for SNAT in common zone did not time out yet, the
> packet would be passed through unSNAT in common zone which would
> be fine and continue on. However, the unDNAT wouldn't work due to
> the limitation of CT not capable of doing unSNAT/unDNAT on the same
> packet twice in the same zone. So the reply would arrive to
> the original interface, but with wrong source address.
>
> b) If the entry for SNAT timed out it would loop and do unSNAT correctly
> in separate zone and then also unDNAT. This is not possible anymore with
> a recent change 8c341b9d (northd: Drop packets destined to router owned
> NAT IP for DGP).
> The reply would be dropped before looping after that change co the traffic
> would never arrive to the original interface.
>
> 3) The unDNAT was happening only if the DGP was outport meaning
> the reply traffic was routed out, but in the opposite case
> the unDNAT was happening only because of the looping which made
> outport=inport. That's why it worked before introduction of explicit drop.
>
> In order to fix all those issues do two changes:
>
> 1) Include inport in the unDNAT match, so that we have both
> routing directions covered e.g. (inport == "dgp_port" || outport ==
> "dpg_port").
>
> 2) Always use the separate zone for SNAT and DNAT. As the common
> zone was needed for HWOL make the common zone optional with
> configuration option called "use_common_zone". This option is
> by default "false" and can be specified per LR. Use of separate
> zones also eliminates the need for the flag propagation
> in "lr_out_chk_dnat_local" stage, removing the match on ct_mark/ct_label.
>

I can't think of a scenario - at least in OpenStack - where we would have
LRs with different values of this setting.
Would it be easier to have a global setting instead?
For the specific case of OpenStack it would be also way easier as it is a
setting we can turn on/off (likely at deployment time) and not have Neutron
having to take care of updating each LR; ie. we can hide this from Neutron.

What do you think?
Thanks a lot!
daniel

>
> Reported-at: https://bugzilla.redhat.com/2161281
> Signed-off-by: Ales Musil 
> ---
>  northd/northd.c | 509 +---
>  northd/ovn-northd.8.xml |  90 +--
>  ovn-nb.xml  |  10 +
>  tests/ovn-northd.at | 217 -
>  tests/ovn.at|   3 +
>  tests/system-ovn.at |  76 +-
>  6 files changed, 509 insertions(+), 396 deletions(-)
>
> diff --git a/northd/northd.c b/northd/northd.c
> index 77e105b86..902f21d77 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -10519,7 +10519,13 @@ build_distr_lrouter_nat_flows_for_lb(struct
> lrouter_nat_lb_flows_ctx *ctx,
>   enum lrouter_nat_lb_flow_type type,
>   struct ovn_datapath *od)
>  {
> -char *gw_action = od->is_gw_router ? "ct_dnat;" : "ct_dnat_in_czone;";
> +bool use_common_zone =
> +smap_get_bool(>nbr->options, "use_common_zone", false);
> +char *gw_action = !od->is_gw_router && use_common_zone
> +  ? "ct_dnat_in_czone;"
> +  : "ct_dnat;";
> +struct ovn_port *dgp = od->l3dgw_ports[0];
> +
>  /* Store the match lengths, so we can reuse the ds buffer. */
>  size_t new_match_len = ctx->new_match->length;
>  size_t est_match_len = ctx->est_match->length;
> @@ -10556,10 +10562,9 @@ build_distr_lrouter_nat_flows_for_lb(struct
> lrouter_nat_lb_flows_ctx *ctx,
>  char *action = type == LROUTER_NAT_LB_FLOW_NORMAL
> ? gw_action : ctx->est[type].action;
>
> -ds_put_format(ctx->undnat_match,
> -  ") && outport == %s && is_chassis_resident(%s)",
> -  od->l3dgw_ports[0]->json_key,
> -  od->l3dgw_ports[0]->cr_port->json_key);
> +ds_put_format(ctx->undnat_match, ") && (inport == %s || outport ==
> %s)"
> +  " && is_chassis_resident(%s)", dgp->json_key,
> dgp->json_key,
> +  dgp->cr_port->json_key);
>  ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_OUT_UNDNAT, 120,
>  ds_cstr(ctx->undnat_match), action,
>  >lb->nlb->header_);
> @@ -11025,13 +11030,8 @@ copy_ra_to_sb(struct ovn_port *op, const char
> *address_mode)
>  static inline bool
>  

Re: [ovs-dev] [PATCH ovn 0/5] Add MAC binding aging mechanism

[Daniel Alvarez Sanchez]

On Wed, Jun 15, 2022 at 1:54 PM Ales Musil  wrote:

> Hi Daniel,
>
> thank you for the suggestions, please see my replies inline.
>
> On Wed, Jun 15, 2022 at 1:00 PM Daniel Alvarez Sanchez <
> dalva...@redhat.com> wrote:
>
>> Hey Ales, first of all thanks a lot for this series!! Great job and good
>> approach.
>>
>> I however have some concerns but maybe they're not much of a deal, please
>> let me know what you think:
>>
>> - Broadcast gARPs will cause all ovn-controller instances in the cluster
>> to attempt to write into the SB database at the same time as they're racing
>> for the ownership of the MAC_Binding entry. This can cause performance
>> problems if they're frequent and/or the number of nodes is "large".
>> What I'd suggest is probably a small random delay to this 'race' so that
>> ovsdb-server won't need to process all the write txns at once. Also with
>> this delay there's a chance that some of the nodes will receive the update
>> of the ownership and abort their write attempt.
>>
>
> This is definitely a potential problem, thank you for bringing it up. It
> seems like this race was there always, even without the chassis that "owns"
> the binding controllers would still try to add the same MAC binding when
> the gARP arrived at multiple of them. So it is actually a broader issue,
> still definitely something that would deserve to be addressed.
>

Indeed, I suspected that this problem existed before. It can be addressed
as a follow up perhaps but thanks for the inputs.


>
>
>>
>> - Similar write 'storm' can happen (at smaller scale) if many of those
>> MAC_Binding entries are added at the same time (as a consequence of a node
>> with a lot of workloads booting up or similar scenario). In this case,
>> there's a chance that ovn-controller(s) will attempt to delete those
>> entries after they age out. Perhaps adding a random number of seconds to
>> the timeout would alleviate this. IMO this is less of a problem.
>>
>
> This might indeed be an issue, however in a real scenario it does not seem
> likely that all binding from this "storm" would be removed at once. Also
> the 60 sec timeout by default is probably not the best choice
> so considering something like 300 secs, it would be a burst of removals
> but still spaced enough apart IMO.
>

You're probably right. The scenario that I had in mind is a node hosting a
lot of workloads with VIPs for example (could be an OpenShift/kubernetes
deployed on top of OpenStack) and announcing their location with gARPs,
upon reboot. All those entries may be added to the cluster right at the
same time and, therefore, expire at the same time *if* they are not used.
Again, more unlikely but experience tells me that these things happen :p
In any case, the problem is much smaller than the first one I described,
even in the worst case.



>
>
>>
>> - The 'racing' for taking ownership can lead to scenarios where the node
>> that wins is not really the one communicating with that MAC anymore, so
>> once it ages out, the other nodes will have to send ARP requests again.
>> This is probably complicated but perhaps, before deleting the entry, we can
>> mark it somehow as 'expired' for a while and other nodes can take ownership
>> during this state if their idle_age < TIMEOUT.
>>
>
> That's probably the biggest downside of this approach. I would like to
> measure if it is causing any issues, if so communicating to other
> ovn-controllers if someone wants to take ownership might be the way to go.
>

I'd say it is hard to tell what issues will cause aside from a few
broadcast packets and some delays in the dataplane due to having to
re-learn the MAC address. I believe that it is quite likely that the
ovn-controller instance that owns the entry is not really the one
communicating in unicast with that particular endpoint. The larger the
cluster, the more likely this scenario is.

What I was suggesting is perhaps marking it as 'expired'
(external_ids:expired=timestamp). If the external_id is set, then other
ovn-controllers can take ownership if idle_age < some_threshold. Taking
ownership would clear the external_id:expired and change the 'chassis'. If
nobody takes ownership, then the previous owner (as the 'chassis' column is
still populated), will delete the entry after a certain amount of time.
This complicates things a bit but saves some broadcast traffic in the
network and avoid dataplane delays.

>
>
>>
>> Again, thanks a lot for this!
>> daniel
>>
>
> Thanks,
> Ales
>
>
>>
>> On Tue, Jun 14, 2022 at 3:49 PM Ales Musil  wrote:
>>
>>> Add MAC binding aging mechanism, that
>>> s

Re: [ovs-dev] [PATCH ovn 0/5] Add MAC binding aging mechanism

[Daniel Alvarez Sanchez]

Hey Ales, first of all thanks a lot for this series!! Great job and good
approach.

I however have some concerns but maybe they're not much of a deal, please
let me know what you think:

- Broadcast gARPs will cause all ovn-controller instances in the cluster to
attempt to write into the SB database at the same time as they're racing
for the ownership of the MAC_Binding entry. This can cause performance
problems if they're frequent and/or the number of nodes is "large".
What I'd suggest is probably a small random delay to this 'race' so that
ovsdb-server won't need to process all the write txns at once. Also with
this delay there's a chance that some of the nodes will receive the update
of the ownership and abort their write attempt.

- Similar write 'storm' can happen (at smaller scale) if many of those
MAC_Binding entries are added at the same time (as a consequence of a node
with a lot of workloads booting up or similar scenario). In this case,
there's a chance that ovn-controller(s) will attempt to delete those
entries after they age out. Perhaps adding a random number of seconds to
the timeout would alleviate this. IMO this is less of a problem.

- The 'racing' for taking ownership can lead to scenarios where the node
that wins is not really the one communicating with that MAC anymore, so
once it ages out, the other nodes will have to send ARP requests again.
This is probably complicated but perhaps, before deleting the entry, we can
mark it somehow as 'expired' for a while and other nodes can take ownership
during this state if their idle_age < TIMEOUT.

Again, thanks a lot for this!
daniel

On Tue, Jun 14, 2022 at 3:49 PM Ales Musil  wrote:

> Add MAC binding aging mechanism, that
> should take care of stale MAC bindings.
>
> The mechanism works on "ownership" of the
> MAC binding row. The chassis that creates
> the row is then checking if the "idle_age"
> of the flow is over the aging threshold.
> In that case the MAC binding is removed
> from database. The "owner" might change
> when another chassis saw an update of the
> MAC address.
>
> This approach has downside, the chassis
> that "owns" the MAC binding might not actually be
> the one that is using it actively. This
> might lead some delays in packet flow when
> the row is removed.
>
> The threshold can be configured in
> NB_global table with key "mac_binding_age_threshold"
> in seconds with default value being 60.
>
> The test case is present in the last patch of the series.
>
> Ales Musil (5):
>   Add chassis column to MAC_Binding table
>   Add MAC binding aging mechanism
>   Add stopwatch for MAC binding aging
>   Allow the MAC binding age threshold to be configurable
>   ovn.at: Add test case covering the MAC binding aging
>
>  controller/automake.mk |   4 +-
>  controller/mac-binding-aging.c | 230 +
>  controller/mac-binding-aging.h |  32 +
>  controller/ovn-controller.c|  32 +
>  controller/pinctrl.c   |  35 +++--
>  northd/northd.c|  12 ++
>  northd/ovn-northd.c|   2 +-
>  ovn-nb.xml |   5 +
>  ovn-sb.ovsschema   |   6 +-
>  ovn-sb.xml |   5 +
>  tests/ovn.at   | 164 +--
>  11 files changed, 499 insertions(+), 28 deletions(-)
>  create mode 100644 controller/mac-binding-aging.c
>  create mode 100644 controller/mac-binding-aging.h
>
> --
> 2.35.3
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn v2] pinctrl: Don't send gARPs for localports

[Daniel Alvarez Sanchez]

On Thu, Mar 25, 2021 at 8:58 AM Dumitru Ceara  wrote:

> On 3/24/21 6:23 PM, Daniel Alvarez Sanchez wrote:
> > Ports of type 'localport' are present on every hypervisor and
> > ovn-controller is sending gARPs for them which makes upstream
> > switches to see its MAC address flapping.
> >
> > In order to avoid this behavior, the current patch is skipping
> > localports when sending gARP/RARP packets.
> >
> > Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1939470
> >
> > Signed-off-by: Daniel Alvarez Sanchez 
>
> 0-day robot was complaining because this misses:
>
> Co-authored-by: Dumitru Ceara 
>
> But I guess that, if the patch is accepted, this can be added at commit
> time.
>

Right, if you think it's best I can send a v3 replacing your 'Signed-off'
by 'Co-authored'.

thanks!
daniel

>
> Regards,
> Dumitru
>
> > Signed-off-by: Dumitru Ceara 
> > ---
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH ovn v2] pinctrl: Don't send gARPs for localports

[Daniel Alvarez Sanchez]

Ports of type 'localport' are present on every hypervisor and
ovn-controller is sending gARPs for them which makes upstream
switches to see its MAC address flapping.

In order to avoid this behavior, the current patch is skipping
localports when sending gARP/RARP packets.

Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1939470

Signed-off-by: Daniel Alvarez Sanchez 
Signed-off-by: Dumitru Ceara 
---

v1 -> v2: added test 

 controller/pinctrl.c |  6 +
 tests/ovn.at | 55 
 2 files changed, 61 insertions(+)

diff --git a/controller/pinctrl.c b/controller/pinctrl.c
index b42288ea5..523a45b9a 100644
--- a/controller/pinctrl.c
+++ b/controller/pinctrl.c
@@ -4240,6 +4240,12 @@ send_garp_rarp_update(struct ovsdb_idl_txn 
*ovnsb_idl_txn,
   struct shash *nat_addresses)
 {
 volatile struct garp_rarp_data *garp_rarp = NULL;
+
+/* Skip localports as they don't need to be announced */
+if (!strcmp(binding_rec->type, "localport")) {
+return;
+}
+
 /* Update GARP for NAT IP if it exists.  Consider port bindings with type
  * "l3gateway" for logical switch ports attached to gateway routers, and
  * port bindings with type "patch" for logical switch ports attached to
diff --git a/tests/ovn.at b/tests/ovn.at
index b751d6db2..8c0146c2d 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -11596,6 +11596,61 @@ OVN_CLEANUP([hv1],[hv2])
 AT_CLEANUP
 ])
 
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([ovn -- localport suppress gARP])
+ovn_start
+
+net_add n1
+sim_add hv1
+as hv1
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+
+check ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys
+
+check ovn-nbctl ls-add ls \
+-- lsp-add ls lp \
+-- lsp-set-type lp localport \
+-- lsp-set-addresses lp "00:00:00:00:00:01 10.0.0.1" \
+-- lsp-add ls ln \
+-- lsp-set-type ln localnet \
+-- lsp-set-options ln network_name=phys \
+-- lsp-add ls lsp \
+-- lsp-set-addresses lsp "00:00:00:00:00:02 10.0.0.2"
+
+dnl First bind the localport.
+check ovs-vsctl add-port br-int vif1 \
+-- set Interface vif1 external-ids:iface-id=lp
+check ovn-nbctl --wait=hv sync
+
+dnl Then bind the regular vif.
+check ovs-vsctl add-port br-int vif2 \
+-- set Interface vif2 external-ids:iface-id=lsp \
+options:tx_pcap=hv1/vif2-tx.pcap \
+options:rxq_pcap=hv1/vif2-rx.pcap
+
+wait_for_ports_up lsp
+check ovn-nbctl --wait=hv sync
+
+dnl Wait for at least two gARPs from lsp (10.0.0.2).
+lsp_garp=00020806000108000604000100020a020a02
+OVS_WAIT_UNTIL([
+garps=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/br-phys-tx.pcap | 
grep ${lsp_garp} -c`
+test $garps -ge 2
+])
+
+dnl At this point it's safe to assume that ovn-controller skipped sending gARP
+dnl for the localport.  Check that there are no other packets than the gARPs
+dnl for the regular vif.
+AT_CHECK([
+pkts=`$PYTHON "$ovs_srcdir/utilities/ovs-pcap.in" hv1/br-phys-tx.pcap | 
grep -v ${lsp_garp} -c`
+test 0 -eq $pkts
+])
+
+OVN_CLEANUP([hv1])
+AT_CLEANUP
+])
+
 OVN_FOR_EACH_NORTHD([
 AT_SETUP([ovn -- 1 LR with HA distributed router gateway port])
 ovn_start
-- 

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH ovn] pinctrl: Don't send gARPs for localports

[Daniel Alvarez Sanchez]

Ports of type 'localport' are present on every hypervisor and
ovn-controller is sending gARPs for them which makes upstream
switches to see its MAC address flapping.

In order to avoid this behavior, the current patch is skipping
localports when sending gARP/RARP packets.

Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1939470

Signed-off-by: Daniel Alvarez Sanchez 
---
 controller/pinctrl.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/controller/pinctrl.c b/controller/pinctrl.c
index b42288ea5..523a45b9a 100644
--- a/controller/pinctrl.c
+++ b/controller/pinctrl.c
@@ -4240,6 +4240,12 @@ send_garp_rarp_update(struct ovsdb_idl_txn 
*ovnsb_idl_txn,
   struct shash *nat_addresses)
 {
 volatile struct garp_rarp_data *garp_rarp = NULL;
+
+/* Skip localports as they don't need to be announced */
+if (!strcmp(binding_rec->type, "localport")) {
+return;
+}
+
 /* Update GARP for NAT IP if it exists.  Consider port bindings with type
  * "l3gateway" for logical switch ports attached to gateway routers, and
  * port bindings with type "patch" for logical switch ports attached to
-- 

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] ovn-controller: Always run the I-P OVS Interface change handler.

[Daniel Alvarez Sanchez]

On Thu, Dec 17, 2020 at 10:23 AM Dumitru Ceara  wrote:

> The incremental processing engine implementation is such that if an
> input node gets updated but the output node doesn't have a change
> handler for it then the output node is immediately recomputed.  That is,
> no other input change handlers are executed.
>
> In case of the en_physical_flow_changes node, if a ct-zone changes we
> were also skipping the OVS interface change handler.  That is incorrect
> as there is an implicit requirement that flows for OVS interfaces that
> got deleted should be cleared before physical_run() is called.
>
> Instead, we now add an explicit change handler for ct-zone changes.
> This change handler never processes ct-zone updates incrementally but
> ensures that the OVS interface change handler is also called.
>
> Moreover, OVS interface changes (including deletes) are now processed
> before physical_run() is called in the flow_output
> physical_flow_changes_handler.  This is a requirement because otherwise
> physical_run() will fail to add flows for new OVS interfaces that
> correspond to unchanged Port_Bindings.
>
> Reported-by: Daniel Alvarez 
> Reported-at: https://bugzilla.redhat.com/1908391
> Fixes: a3005f0dc777 ("ovn-controller: I-P for ct zone and OVS interface
> changes in flow output stage.")
> Signed-off-by: Dumitru Ceara 
> ---
>  controller/ovn-controller.c | 30 ++-
>  tests/ovn.at| 72
> +
>  2 files changed, 94 insertions(+), 8 deletions(-)
>
> diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c
> index b5f0c13..5708b7b 100644
> --- a/controller/ovn-controller.c
> +++ b/controller/ovn-controller.c
> @@ -1714,6 +1714,16 @@ en_physical_flow_changes_run(struct engine_node
> *node, void *data)
>  engine_set_node_state(node, EN_UPDATED);
>  }
>
> +/* ct_zone changes are not handled incrementally but a handler is required
> + * to avoid skipping the ovs_iface incremental change handler.
> + */
> +static bool
> +physical_flow_changes_ct_zones_handler(struct engine_node *node
> OVS_UNUSED,
> +   void *data OVS_UNUSED)
> +{
> +return false;
> +}
> +
>  /* There are OVS interface changes. Indicate to the flow_output engine
>   * to handle these OVS interface changes for physical flow computations.
> */
>  static bool
> @@ -2256,6 +2266,13 @@ flow_output_physical_flow_changes_handler(struct
> engine_node *node, void *data)
>  struct ed_type_pfc_data *pfc_data =
>  engine_get_input_data("physical_flow_changes", node);
>
> +/* If there are OVS interface changes. Try to handle them
> incrementally. */
> +if (pfc_data->ovs_ifaces_changed) {
> +if (!physical_handle_ovs_iface_changes(_ctx, >flow_table)) {
> +return false;
> +}
> +}
> +
>  if (pfc_data->recompute_physical_flows) {
>  /* This indicates that we need to recompute the physical flows. */
>  physical_clear_unassoc_flows_with_db(>flow_table);
> @@ -2265,12 +2282,6 @@ flow_output_physical_flow_changes_handler(struct
> engine_node *node, void *data)
>  return true;
>  }
>
> -if (pfc_data->ovs_ifaces_changed) {
> -/* There are OVS interface changes. Try to handle them
> - * incrementally. */
> -return physical_handle_ovs_iface_changes(_ctx, >flow_table);
> -}
> -
>  return true;
>  }
>
> @@ -2549,11 +2560,14 @@ main(int argc, char *argv[])
>  /* Engine node physical_flow_changes indicates whether
>   * we can recompute only physical flows or we can
>   * incrementally process the physical flows.
> + *
> + * Note: The order of inputs is important, all OVS interface changes
> must
> + * be handled before any ct_zone changes.
>   */
> -engine_add_input(_physical_flow_changes, _ct_zones,
> - NULL);
>  engine_add_input(_physical_flow_changes, _ovs_interface,
>   physical_flow_changes_ovs_iface_handler);
> +engine_add_input(_physical_flow_changes, _ct_zones,
> + physical_flow_changes_ct_zones_handler);
>
>  engine_add_input(_flow_output, _addr_sets,
>   flow_output_addr_sets_handler);
> diff --git a/tests/ovn.at b/tests/ovn.at
> index 80bc099..66088a7 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -22059,6 +22059,78 @@ OVS_WAIT_UNTIL([test x$(as hv1 ovn-appctl -t
> ovn-controller debug/status) = "xru
>  OVN_CLEANUP([hv1])
>  AT_CLEANUP
>
> +AT_SETUP([ovn -- Multiple OVS port changes Incremental Processing])

Re: [ovs-dev] Scaling of Logical_Flows and MAC_Binding tables

[Daniel Alvarez Sanchez]

On Wed, Nov 25, 2020 at 7:59 PM Dumitru Ceara  wrote:

> On 11/25/20 7:06 PM, Numan Siddique wrote:
> > On Wed, Nov 25, 2020 at 10:24 PM Renat Nurgaliyev 
> wrote:
> >>
> >>
> >>
> >> On 25.11.20 16:14, Dumitru Ceara wrote:
> >>> On 11/25/20 3:30 PM, Renat Nurgaliyev wrote:
>  Hello folks,
> 
> >>> Hi Renat,
> >>>
>  we run a lab where we try to evaluate scalability potential of OVN
> with
>  OpenStack as CMS.
>  Current lab setup is following:
> 
>  500 networks
>  500 routers
>  1500 VM ports (3 per network/router)
>  1500 Floating IPs (one per VM port)
> 
>  There is an external network, which is bridged to br-provider on
> gateway
>  nodes. There are 2000 ports
>  connected to this external network (1500 Floating IPs + 500 SNAT
> router
>  ports). So the setup is not
>  very big we'd say, but after applying this configuration via ML2/OVN
>  plugin, northd kicks in and does
>  its job, and after its done, Logical_Flow table gets 645877 entries,
>  which is way too much. But ok,
>  we move on and start one controller on the gateway chassis, and here
>  things get really messy.
>  MAC_Binding table grows from 0 to 999088 entries in one moment, and
>  after its done, the size of SB
>  biggest tables look like this:
> 
>  999088 MAC_Binding
>  645877 Logical_Flow
>  4726 Port_Binding
>  1117 Multicast_Group
>  1068 Datapath_Binding
>  1046 Port_Group
>  551 IP_Multicast
>  519 DNS
>  517 HA_Chassis_Group
>  517 HA_Chassis
>  ...
> 
>  MAC binding table gets huge, basically it now has an entry for every
>  port that is connected to external
>  network * number of datapaths, which roughly makes it one million
>  entries. This table by itself increases
>  the size of the SB by 200 megabytes. Logical_Flow table also gets very
>  heavy, we have already played a bit
>  with logical datapath patches that Ilya Maximets submitted, and it
> looks
>  much better, but the size of
>  the MAC_Binding table still feels inadequate.
> 
>  We would like to start to work at least on MAC_Binding table
>  optimisation, but it is a bit difficult
>  to start working from scratch. Can someone help us with ideas how this
>  could be optimised?
> 
>  Maybe it would also make sense to group entries in MAC_Binding table
> in
>  the same way like it is proposed
>  for logical flows in Ilya's patch?
> 
> >>> Maybe it would work but I'm not really sure how, right now.  However,
> >>> what if we change the way MAC_Bindings are created?
> >>>
> >>> Right now a MAC Binding is created for each logical router port but in
> >>> your case there are a lot of logical router ports connected to the
> >>> single provider logical switch and they all learn the same ARPs.
> >>>
> >>> What if we instead store MAC_Bindings per logical switch?  Basically
> >>> sharing all these MAC_Bindings between all router ports connected to
> the
> >>> same LS.
> >>>
> >>> Do you see any problem with this approach?
> >>>
> >>> Thanks,
> >>> Dumitru
> >>>
> >>>
> >> I believe that this approach is way to go, at least nothing comes to my
> mind
> >> that could go wrong here. We will try to make a patch for that.
> However, if
> >> someone is familiar with the code and knows how to do it fast, it would
> also
> >> be very nice.
> >
> > This approach should work.
> >
> > I've another idea (I won't call it a solution yet). What if we drop
> > the usage of MAC_Binding altogether ?
>
> This would be great!
>
> >
> > - When ovn-controller learns a mac_binding, it will not create a row
> > into the SB MAC_binding table
> > - Instead it will maintain the learnt mac binding in its memory.
> > - ovn-controller will still program the table 66 with the flow to set
> > the eth.dst (for the get_arp() action)
> >
> > This has a couple of advantages
> >   - Right now we never flush the old/stale mac_binding entries.
> >   - If suppose the mac of an external IP has changed, but OVN has an
> > entry for that IP with old mac in the mac_binding table,
> > we will use the old mac, causing the packet to be sent out to the
> > wrong destination and the packet might get lost.
> >   - So we will get rid of this problem
> >   - We will also save SB DB space.
> >
> > There are few disadvantages
> >   -  Other ovn-controllers will not add the flows in table 66. I guess
> > this should be fine as each ovn-controller
> > can generate the ARP request and learn the mac.
> >   - When ovn-controller restarts we lose the learnt macs and would
> > need to learn again.
> >
> > Any thoughts on this ?
>

It'd be great to have some sort of local ARP cache but I'm concerned about
the performance implications.

- How are you going to determine when an entry is stale?
If you slow path the packets to reset the timeout everytime a pkt with
source mac is received, it doesn't look good. 

Re: [ovs-dev] [PATCH] ovs-ctl: Don't set hostname as external-id

[Daniel Alvarez Sanchez]

Thanks a lot!
I proposed the new approach at:

https://patchwork.ozlabs.org/project/openvswitch/patch/20200525152821.19838-1-dalva...@redhat.com/


On Sat, May 23, 2020 at 8:38 PM Han Zhou  wrote:
>
>
>
> On Sat, May 23, 2020 at 12:06 AM Daniel Alvarez  wrote:
> >
> >
> > Thanks a lot Terry!
> >
> >
> > > On 22 May 2020, at 23:28, Terry Wilson  wrote:
> > >
> > > 
> > >
> > >
> > >> On Wed, May 20, 2020 at 10:52 AM Daniel Alvarez  
> > >> wrote:
> > >> ovs-ctl started to add the hostname as external-id [0] at some point.
> > >>
> > >> However, this can be problematic as if it's already set by an external
> > >> entity it will get overwritten. In RHEL systems, systemd will invoke
> > >> ovs-ctl to start OVS and that will overwrite it to the hostname of the
> > >> machine.
> > > If the problem is just ovs-ctl *overwriting* an existing entity then can 
> > > we just change
> > >
> > >> -ovs_vsctl set Open_vSwitch . external-ids:hostname="$hn"
> > >
> > > to ovs_vsctl add Open_vSwitch . external_ids:hostname="$hn"
> > >
> > > since add doesn't overwrite existing values if the key is set[1].
> >
> > This sounds great to me!
> > Han, it looks like Terry’s suggestion would work for the HV onboarding 
> > scenario that you mentioned and also fit in the model where an external 
> > entity decides what name to give to a particular HV. What do you think?
>
> Yes, SGTM, too :)
>
> > >
> > > [1] From man ovs-vsctl
> > >   [--if-exists] add table record column [key=]value...
> > >   Adds the specified value or key-value pair to column in 
> > > record in table.  If column is a map, then key is required, otherwise it 
> > > is  prohib‐
> > >   ited.  If key already exists in a map column, then the 
> > > current value is not replaced (use the set command to replace an existing 
> > > value).
> > >
> > >   Without --if-exists, it is an error if record does not 
> > > exist.  With --if-exists, this command does nothing if record does not 
> > > exist.
> > >
> > > Terry
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] ovs-ctl: Don't overwrite external-id hostname

[Daniel Alvarez]

ovs-ctl started to add the hostname as external-id [0] at some point.

However, this can be problematic as if it's already set by an external
entity it will get overwritten. In RHEL systems, systemd will invoke
ovs-ctl to start OVS and that will overwrite it to the hostname of the
machine.

For OVN this can have a big impact because if, for whatever reason the
hostname changes and the host gets restarted, ovn-controller won't
claim the ports back leaving the workloads unaccessible.

Also, it makes sense to not overwrite it as 1) it's an external_id,
so it will actually let external entities to configure it (unlike now),
and 2) it's optional. In the case that some systems were relying on
ovs-ctl to set the external-id for the first time (e.g onboarding
of a new hypervisor), this patch is not changing such behavior.

For more details, see discussion at [1].

[0] https://mail.openvswitch.org/pipermail/ovs-dev/2016-March/312054.html
[1] https://mail.openvswitch.org/pipermail/ovs-dev/2020-May/370813.html

Signed-off-by: Daniel Alvarez 
---
 utilities/ovs-ctl.in | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/utilities/ovs-ctl.in b/utilities/ovs-ctl.in
index 8c5cd7032..9be9c9871 100644
--- a/utilities/ovs-ctl.in
+++ b/utilities/ovs-ctl.in
@@ -43,7 +43,8 @@ set_hostname () {
 else
 hn="$(uname -n)"
 fi
-ovs_vsctl set Open_vSwitch . external-ids:hostname="$hn"
+# Set the hostname if it wasn't set before
+ovs_vsctl add Open_vSwitch . external-ids hostname="$hn"
 }

 set_system_ids () {
--

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovs-ctl: Don't set hostname as external-id

[Daniel Alvarez]

Thanks a lot Terry!


> On 22 May 2020, at 23:28, Terry Wilson  wrote:
> 
> 
> 
> 
>> On Wed, May 20, 2020 at 10:52 AM Daniel Alvarez  wrote:
>> ovs-ctl started to add the hostname as external-id [0] at some point.
>> 
>> However, this can be problematic as if it's already set by an external
>> entity it will get overwritten. In RHEL systems, systemd will invoke
>> ovs-ctl to start OVS and that will overwrite it to the hostname of the
>> machine.
> If the problem is just ovs-ctl *overwriting* an existing entity then can we 
> just change
> 
>> -ovs_vsctl set Open_vSwitch . external-ids:hostname="$hn"
> 
> to ovs_vsctl add Open_vSwitch . external_ids:hostname="$hn" 
> 
> since add doesn't overwrite existing values if the key is set[1].

This sounds great to me!
Han, it looks like Terry’s suggestion would work for the HV onboarding scenario 
that you mentioned and also fit in the model where an external entity decides 
what name to give to a particular HV. What do you think?
> 
> [1] From man ovs-vsctl
>   [--if-exists] add table record column [key=]value...
>   Adds the specified value or key-value pair to column in record 
> in table.  If column is a map, then key is required, otherwise it is  prohib‐
>   ited.  If key already exists in a map column, then the current 
> value is not replaced (use the set command to replace an existing value).
> 
>   Without --if-exists, it is an error if record does not exist.  
> With --if-exists, this command does nothing if record does not exist.
> 
> Terry
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovs-ctl: Don't set hostname as external-id

[Daniel Alvarez Sanchez]

On Fri, May 22, 2020 at 2:17 AM Han Zhou  wrote:
>
>
>
> On Thu, May 21, 2020 at 3:19 AM Daniel Alvarez Sanchez  
> wrote:
> >
> > On Thu, May 21, 2020 at 9:22 AM Han Zhou  wrote:
> > >
> > >
> > >
> > > On Wed, May 20, 2020 at 8:52 AM Daniel Alvarez  
> > > wrote:
> > > >
> > > > ovs-ctl started to add the hostname as external-id [0] at some point.
> > > >
> > > > However, this can be problematic as if it's already set by an external
> > > > entity it will get overwritten. In RHEL systems, systemd will invoke
> > > > ovs-ctl to start OVS and that will overwrite it to the hostname of the
> > > > machine.
> > > >
> > > > For OVN this can have a big impact because if, for whatever reason the
> > > > hostname changes and the host gets restarted, ovn-controller won't
> > > > claim the ports back leaving the workloads unaccessible.
> > > >
> > > > Also, it makes sense to leave this untouched as 1) it's an external_id,
> > > > so it will actually let external entities to configure it (unlike now),
> > > > and 2) it's optional. In the case of OVN, if the external-id doesn't
> > > > exist, it'll default to its hostname so nothing should get broken by
> > > > this change.
> > > >
> > > > [0] 
> > > > https://mail.openvswitch.org/pipermail/ovs-dev/2016-March/312054.html
> > > >
> > > > Signed-off-by: Daniel Alvarez 
> > > > ---
> > > >  utilities/ovs-ctl.in | 12 
> > > >  1 file changed, 12 deletions(-)
> > > >
> > > > diff --git a/utilities/ovs-ctl.in b/utilities/ovs-ctl.in
> > > > index 8c5cd7032..87fc4934f 100644
> > > > --- a/utilities/ovs-ctl.in
> > > > +++ b/utilities/ovs-ctl.in
> > > > @@ -35,17 +35,6 @@ insert_mod_if_required () {
> > > >  ovs_kmod_ctl insert
> > > >  }
> > > >
> > > > -set_hostname () {
> > > > -# 'hostname -f' needs network connectivity to work.  So we should
> > > > -# call this only after ovs-vswitchd is running.
> > > > -if test X$FULL_HOSTNAME = Xyes; then
> > > > -hn="$(hostname -f)" || hn="$(uname -n)"
> > > > -else
> > > > -hn="$(uname -n)"
> > > > -fi
> > > > -ovs_vsctl set Open_vSwitch . external-ids:hostname="$hn"
> > > > -}
> > > > -
> > > >  set_system_ids () {
> > > >  set ovs_vsctl set Open_vSwitch .
> > > >
> > > > @@ -225,7 +214,6 @@ start_forwarding () {
> > > >  if test X"$OVS_VSWITCHD" = Xyes; then
> > > >  do_start_forwarding || return 1
> > > >  fi
> > > > -set_hostname &
> > > >  return 0
> > > >  }
> > > >
> > > > --
> > > >
> > > > ___
> > > > dev mailing list
> > > > d...@openvswitch.org
> > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> > >
> > > Hi Daniel,
> > >
> > > I believe there are OpenStack based OVN users already depends on this. 
> > > (And we had also add the --no-full-hostname option so that it will set 
> > > the short name, so that it matches with openstack's "requested-chassis" 
> > > setting which uses nova compute node name.) For the scenarios that this 
> > > behavior is not desired, I think it is better to add a new option to 
> > > override it, such as "--no-hostname", so that existing environment won't 
> > > get impacted. What do you think?
> >
> > Hi Han, thanks a lot for your feedback!
> > I thought of adding a --no-hostname option. However I still see that
> > this patch has value. Let me try to explain.
> >
> > Existing OpenStack deployments will have their 'requested-chassis' set
> > to the current name of the hosts at the time the VMs were created.
> > This hostname may or may not match the machine hostname as it's a
> > string consumed from the external_ids, hence external and expected to
> > be set by CMS (for example, it's configurable by puppet at OpenStack
> > deployment time).
> > Now let's imagine you restart one node so ovs-ctl will overwrite
> > whatever the CMS relied upon and set it to the machine hostname. From
> > this time on, the workloads

Re: [ovs-dev] [PATCH] ovs-ctl: Don't set hostname as external-id

[Daniel Alvarez Sanchez]

On Thu, May 21, 2020 at 9:22 AM Han Zhou  wrote:
>
>
>
> On Wed, May 20, 2020 at 8:52 AM Daniel Alvarez  wrote:
> >
> > ovs-ctl started to add the hostname as external-id [0] at some point.
> >
> > However, this can be problematic as if it's already set by an external
> > entity it will get overwritten. In RHEL systems, systemd will invoke
> > ovs-ctl to start OVS and that will overwrite it to the hostname of the
> > machine.
> >
> > For OVN this can have a big impact because if, for whatever reason the
> > hostname changes and the host gets restarted, ovn-controller won't
> > claim the ports back leaving the workloads unaccessible.
> >
> > Also, it makes sense to leave this untouched as 1) it's an external_id,
> > so it will actually let external entities to configure it (unlike now),
> > and 2) it's optional. In the case of OVN, if the external-id doesn't
> > exist, it'll default to its hostname so nothing should get broken by
> > this change.
> >
> > [0] https://mail.openvswitch.org/pipermail/ovs-dev/2016-March/312054.html
> >
> > Signed-off-by: Daniel Alvarez 
> > ---
> >  utilities/ovs-ctl.in | 12 
> >  1 file changed, 12 deletions(-)
> >
> > diff --git a/utilities/ovs-ctl.in b/utilities/ovs-ctl.in
> > index 8c5cd7032..87fc4934f 100644
> > --- a/utilities/ovs-ctl.in
> > +++ b/utilities/ovs-ctl.in
> > @@ -35,17 +35,6 @@ insert_mod_if_required () {
> >  ovs_kmod_ctl insert
> >  }
> >
> > -set_hostname () {
> > -# 'hostname -f' needs network connectivity to work.  So we should
> > -# call this only after ovs-vswitchd is running.
> > -if test X$FULL_HOSTNAME = Xyes; then
> > -hn="$(hostname -f)" || hn="$(uname -n)"
> > -else
> > -hn="$(uname -n)"
> > -fi
> > -ovs_vsctl set Open_vSwitch . external-ids:hostname="$hn"
> > -}
> > -
> >  set_system_ids () {
> >  set ovs_vsctl set Open_vSwitch .
> >
> > @@ -225,7 +214,6 @@ start_forwarding () {
> >  if test X"$OVS_VSWITCHD" = Xyes; then
> >  do_start_forwarding || return 1
> >  fi
> > -set_hostname &
> >  return 0
> >  }
> >
> > --
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
> Hi Daniel,
>
> I believe there are OpenStack based OVN users already depends on this. (And 
> we had also add the --no-full-hostname option so that it will set the short 
> name, so that it matches with openstack's "requested-chassis" setting which 
> uses nova compute node name.) For the scenarios that this behavior is not 
> desired, I think it is better to add a new option to override it, such as 
> "--no-hostname", so that existing environment won't get impacted. What do you 
> think?

Hi Han, thanks a lot for your feedback!
I thought of adding a --no-hostname option. However I still see that
this patch has value. Let me try to explain.

Existing OpenStack deployments will have their 'requested-chassis' set
to the current name of the hosts at the time the VMs were created.
This hostname may or may not match the machine hostname as it's a
string consumed from the external_ids, hence external and expected to
be set by CMS (for example, it's configurable by puppet at OpenStack
deployment time).
Now let's imagine you restart one node so ovs-ctl will overwrite
whatever the CMS relied upon and set it to the machine hostname. From
this time on, the workloads on that VM will not be claimed by OVN and
are left inaccessible until manual intervention happens.

I think it's fundamentally wrong for OVS (ovs-ctl in this case) to set
an external id itself as its default behavior (as they're meant for
external entities IIUC). Can you please elaborate on how existing
environments can get impacted by this change? Also keep in mind that
if the external_id is not set, ovn-controller is taking the hostname
of the machine as the default [0]. I might be overlooking something,
for sure.

Thanks again!
Daniel

[0] https://github.com/ovn-org/ovn/blob/master/controller/chassis.c#L133

>
> Thanks,
> Han

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] ovs-ctl: Don't set hostname as external-id

[Daniel Alvarez]

ovs-ctl started to add the hostname as external-id [0] at some point.

However, this can be problematic as if it's already set by an external
entity it will get overwritten. In RHEL systems, systemd will invoke
ovs-ctl to start OVS and that will overwrite it to the hostname of the
machine.

For OVN this can have a big impact because if, for whatever reason the
hostname changes and the host gets restarted, ovn-controller won't
claim the ports back leaving the workloads unaccessible.

Also, it makes sense to leave this untouched as 1) it's an external_id,
so it will actually let external entities to configure it (unlike now),
and 2) it's optional. In the case of OVN, if the external-id doesn't
exist, it'll default to its hostname so nothing should get broken by
this change.

[0] https://mail.openvswitch.org/pipermail/ovs-dev/2016-March/312054.html

Signed-off-by: Daniel Alvarez 
---
 utilities/ovs-ctl.in | 12 
 1 file changed, 12 deletions(-)

diff --git a/utilities/ovs-ctl.in b/utilities/ovs-ctl.in
index 8c5cd7032..87fc4934f 100644
--- a/utilities/ovs-ctl.in
+++ b/utilities/ovs-ctl.in
@@ -35,17 +35,6 @@ insert_mod_if_required () {
 ovs_kmod_ctl insert
 }

-set_hostname () {
-# 'hostname -f' needs network connectivity to work.  So we should
-# call this only after ovs-vswitchd is running.
-if test X$FULL_HOSTNAME = Xyes; then
-hn="$(hostname -f)" || hn="$(uname -n)"
-else
-hn="$(uname -n)"
-fi
-ovs_vsctl set Open_vSwitch . external-ids:hostname="$hn"
-}
-
 set_system_ids () {
 set ovs_vsctl set Open_vSwitch .

@@ -225,7 +214,6 @@ start_forwarding () {
 if test X"$OVS_VSWITCHD" = Xyes; then
 do_start_forwarding || return 1
 fi
-set_hostname &
 return 0
 }

--

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] ovn-controller to no longer monitor Chassis' external_ids

[Daniel Alvarez Sanchez]

Thanks Lucas!

The only problem I see is that we still need something like the
Private_Chassis approach that we discussed or otherwise bumping nb_cfg will
still be generating N notifications (N == num_chassis) on every write.
Still, this solves part of the problem.

I was wondering if we could leverage the Controller_Event table [0] for
this purpose and generate an event to health check the chassis. Looks like
ovn-controller monitors only its own chassis so we could be good here. What
do you think?

[0] https://mail.openvswitch.org/pipermail/ovs-dev/2019-June/359826.html

On Thu, May 7, 2020 at 12:12 PM  wrote:

> From: Lucas Alvares Gomes 
>
> Prior to this patch, the external_ids column from the Chassis table were
> being used for two purposes:
>
> 1. Holding configuration for the OVN (copied by ovn-controller from the
> local OVS database)
>
> 2. Holding information from external systems (the main purpose of the
>external_ids column across other resources).
>
> The problem with mixing these two use cases is that, ovn-controller
> should be aware of changes to the configuration and it would trigger
> flow re-computations upon those changes. It shouldn't care tho about the
> information from external systems but, since these two things were put
> together, CMSs writing things to the external_ids column of the Chassis
> were waking up ovn-controller to recompute flows.
>
> This patch is separating these two things by creating another column in
> the Chassis table called "other_config". This new table holds the OVN
> configuration that is copied over from the local OVS db leaving the
> external_ids column unmonitored and free for other systems to make use
> of it.
>
> This change also keeps things backward compatible by continuing to
> write the OVN configuration to the external_ids column for external
> systems that may be reading them from there but, that column is no
> longer monitored so it won't generate any events. This behavior should
> be temporary and removed in the future. The note in the NEWS file and
> comments in the code itself points to the future removal of this
> behavior.
>
> Reported-At: https://bugzilla.redhat.com/show_bug.cgi?id=1824220
> Signed-off-by: Lucas Alvares Gomes 
> ---
>  NEWS| 10 ++-
>  controller/bfd.c|  2 +-
>  controller/chassis.c| 48 ++---
>  controller/encaps.c |  4 +--
>  controller/ovn-controller.8.xml |  2 +-
>  controller/ovn-controller.c |  9 ---
>  controller/physical.c   |  4 +--
>  ic/ovn-ic.c |  6 ++---
>  northd/ovn-northd.c |  4 +--
>  ovn-sb.ovsschema|  7 +++--
>  ovn-sb.xml  | 14 +-
>  tests/ovn-controller.at | 16 +--
>  12 files changed, 71 insertions(+), 55 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index c2b7945df..1b33be249 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -1,7 +1,15 @@
>  Post-v20.03.0
>  --
> - Added support for external_port_range in NAT table.
> -
> +   - ovn-controller no longer monitor the external_ids column from
> + the Chassis table. This was done to avoid having to do a flow
> + recalculation every time external systems wrote to this column. The
> + chassis configuration has now being moved to a new column called
> + "other_config". As a note, the configurations are still be written to
> + the external_ids column (but no longer triggers an alert) to
> + keep the backward compatibility with current systems that may be
> + reading it from that column but, this behavior will be removed
> + in the future.
>
>  OVN v20.03.0 - xx xxx 
>  --
> diff --git a/controller/bfd.c b/controller/bfd.c
> index 2b1e87f6d..b305eb158 100644
> --- a/controller/bfd.c
> +++ b/controller/bfd.c
> @@ -152,7 +152,7 @@ bfd_calculate_chassis(
>  /* It's an HA chassis. So add the ref_chassis to the bfd set.
> */
>  for (size_t i = 0; i < ha_chassis_grp->n_ref_chassis; i++) {
>  struct sbrec_chassis *ref_ch =
> ha_chassis_grp->ref_chassis[i];
> -if (smap_get_bool(_ch->external_ids, "is-remote",
> false)) {
> +if (smap_get_bool(_ch->other_config, "is-remote",
> false)) {
>  continue;
>  }
>  sset_add(_chassis, ref_ch->name);
> diff --git a/controller/chassis.c b/controller/chassis.c
> index 522893ead..c1c609028 100644
> --- a/controller/chassis.c
> +++ b/controller/chassis.c
> @@ -299,24 +299,24 @@ chassis_parse_ovs_config(const struct
> ovsrec_open_vswitch_table *ovs_table,
>  }
>
>  static void
> -chassis_build_external_ids(struct smap *ext_ids, const char
> *bridge_mappings,
> +chassis_build_other_config(struct smap *config, const char
> *bridge_mappings,
> const char *datapath_type, const char
> *cms_options,
>

Re: [ovs-dev] [PATCH v3 ovn] Documentation: Change 'Open vSwitch' for 'OVN and logo

[Daniel Alvarez Sanchez]

On Fri, Mar 20, 2020 at 1:38 PM Mark Michelson  wrote:

> Thanks for this. Getting the conversion from "Open vSwitch" to OVN has
> been a challenge.
>
> Everything looks good here, but I have one further suggestion. When you
> click the links to manpages on ovn.org, the header on the manpages still
> reads "Open vSwitch Manual" and the footer shows "Open vSwitch
> 20.03.90". These are added by the buildaux/xml2nroff script. If that can
> be updated as part of this patch, that would be great.
>

Thanks, done in v4. It'll require uploading the new static manpages to the
website though.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2 ovn] Documentation: Change 'Open vSwitch' for 'OVN and logo

[Daniel Alvarez Sanchez]

On Tue, Mar 17, 2020 at 6:57 PM Ben Pfaff  wrote:

> On Tue, Mar 17, 2020 at 06:22:07PM +0100, Daniel Alvarez wrote:
> > The current version of the documentation is still using the
> > Open vSwitch logo and includes some references to OVS that
> > should be changed by OVN.
> >
> > Signed-off-by: Daniel Alvarez 
>
> Here's the original SVG for the logo.
>

Thanks Ben, I picked the OVN log and resized to 200x200 as per the Sphinx
doc [0].
I'm a bit puzzled as the OVS logo [1] is 502x334 but somehow it works well
in the OVS website.

Do you have any suggestions? Thanks a lot!

[0]
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html-logo
[1]
https://github.com/openvswitch/ovs/blob/master/Documentation/_static/logo.png
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] Documentation: Change 'Open vSwitch' for 'OVN' and logo

[Daniel Alvarez Sanchez]

On Tue, Mar 17, 2020 at 5:11 PM Ilya Maximets  wrote:

> On 3/17/20 4:17 PM, Daniel Alvarez wrote:
> > The current version of the documentation is still using the
> > Open vSwitch logo and includes some references to OVS that
> > should be changed by OVN.
> >
> > Signed-off-by: Daniel Alvarez 
> > ---
> >
> > diff --git a/Documentation/automake.mk b/Documentation/automake.mk
> > index 2f337532c..ca1fb1222 100644
> > --- a/Documentation/automake.mk
> > +++ b/Documentation/automake.mk
> > @@ -1,7 +1,6 @@
> >  DOC_SOURCE = \
> >   Documentation/group-selection-method-property.txt \
> >   Documentation/_static/logo.png \
> > - Documentation/_static/overview.png \
> >   Documentation/conf.py \
> >   Documentation/index.rst \
> >   Documentation/contents.rst \
> > diff --git a/Documentation/conf.py b/Documentation/conf.py
> > index 2c85dcf4d..c7949b73e 100644
> > --- a/Documentation/conf.py
> > +++ b/Documentation/conf.py
> > @@ -47,9 +47,9 @@ source_suffix = '.rst'
> >  master_doc = 'contents'
> >
> >  # General information about the project.
> > -project = u'Open vSwitch'
> > -copyright = u'2016, The Open vSwitch Development Community'
> > -author = u'The Open vSwitch Development Community'
> > +project = u'OVN'
> > +copyright = u'2020, The OVN Development Community'
> > +author = u'The OVN Development Community'
> >
> >  # The version info for the project you're documenting, acts as
> replacement for
> >  # |version| and |release|, also used in various other places throughout
> the
> > diff --git a/Documentation/contents.rst b/Documentation/contents.rst
> > index a754bd225..469dfc34d 100644
> > --- a/Documentation/contents.rst
> > +++ b/Documentation/contents.rst
> > @@ -22,7 +22,7 @@
> >Avoid deeper levels because they do not render well.
> >
> >  ===
> > -Open vSwitch Documentation Contents
> > +OVN Documentation Contents
> >  ===
>
> Upper/underlines should match the text length, i.e.
>
> ===
> My text
> ===
>
> Longer test
> ---
>
> rST format requires that and there will be a warning.
>

Oh, my bad dalvarez-- :(
Sending a v2 shortly

>
>
> Same for all ther changes here.
>
> >
> >  .. toctree::
> > diff --git a/Documentation/faq/index.rst b/Documentation/faq/index.rst
> > index 7ad71aabe..f2b3afba8 100644
> > --- a/Documentation/faq/index.rst
> > +++ b/Documentation/faq/index.rst
> > @@ -24,7 +24,7 @@
> >Avoid deeper levels because they do not render well.
> >
> >  
> > -Open vSwitch FAQ
> > +OVN FAQ
> >  
> >
> >  .. toctree::
> > diff --git a/Documentation/index.rst b/Documentation/index.rst
> > index c0c6c374d..b026fe4fd 100644
> > --- a/Documentation/index.rst
> > +++ b/Documentation/index.rst
> > @@ -24,13 +24,13 @@
> >Avoid deeper levels because they do not render well.
> >
> >  ==
> > -Open vSwitch Documentation
> > +OVN Documentation
> >  ==
> >
> >  How the Documentation is Organised
> >  --
> >
> > -The Open vSwitch documentation is organised into multiple sections:
> > +The OVN documentation is organised into multiple sections:
> >
> >  - :doc:`Installation guides ` guide you through
> >installing Open vSwitch (OVS) and Open Virtual Network (OVN) on a
> variety of
> > @@ -68,10 +68,10 @@ Deeper Dive
> >:doc:`intro/install/rhel` |
> >:doc:`intro/install/fedora`
> >
> > -The Open vSwitch Project
> > +The OVN Project
> >  
> >
> > -Learn more about the Open vSwitch project and about how you can
> contribute:
> > +Learn more about the OVN project and about how you can contribute:
> >
> >  - **Community:** :doc:`internals/release-process` |
> >:doc:`internals/authors` |
> > @@ -90,7 +90,7 @@ Learn more about the Open vSwitch project and about
> how you can contribute:
> >:doc:`internals/committer-emeritus-status`
> >
> >  - **Documentation:** :doc:`internals/contributing/documentation-style` |
> > -  :doc:`Building Open vSwitch Documentation
> ` |
> > +  :doc:`Building OVN Documentation ` |
> >:doc:`internals/documentation`
> >
> >  Getting Help
> > --
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> >
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] ovn-northd: Address scale issues with DNAT flows.

[Daniel Alvarez Sanchez]

Can we please include this patch as part of the next release?

This is a really important bug fix especially on systems that have a
considerable number of floating IPs where otherwise the cloud becomes
unusable.

On Tue, Feb 4, 2020 at 2:28 PM Daniel Alvarez Sanchez 
wrote:

> Hi Numan,
>
> Thanks for this. I have deployed a setup with master branch and this patch
> applied. The patch LGTM and I can confirm that it works. Moreover, the
> number of lflows has been dramatically reduced from around 50
>
> With 150 FIPs:
>
> Before: ~70K lflows  https://imgur.com/a/OaVelCb
> After (in the same setup): 2K lflows
>
> [root@central ~]# ovn-sbctl list logical_flow | grep _uuid -c
> 2092
> [root@central ~]# ovn-sbctl list logical_flow | grep lr_out_egr_loop -c
> 155
> [root@central ~]# ovn-sbctl list logical_flow | grep lr_in_ip_routing -c
> 7
>
> And I verified that FIP to FIP traffic works via the localnet port so it
> is really distributed and does not traverse any tunnels.
>
> Thanks!
> Daniel
>
> Tested-By: Daniel Alvarez Sanchez 
> Acked-By: Daniel Alvarez Sanchez 
>
> On Tue, Feb 4, 2020 at 1:59 PM Dumitru Ceara  wrote:
>
>> On 2/1/20 11:53 AM, num...@ovn.org wrote:
>> > From: Numan Siddique 
>> >
>> > When the commit [1] added Distributed NAT support in OVN, it didn't
>> address
>> > the requirement of making East/West NAT traffic distributed. The E/W NAT
>> > traffic was still centralized. Later a couple of patches [2], addressed
>> this
>> > requirement. But the approach taken in [2] resulted in a lot of logical
>> flows
>> > as number of dnat_and_snat entries increase, as reported in
>> @Reported-at.
>> >
>> > This patch
>> >   - reverts the approch taken in [2].
>> >   - removing the flows which does the NAT direct (REGBIT_NAT_REDIRECT)
>> to
>> > the gateway chassis.
>> >   - and to solve the E/W centralized NAT it does the following:
>> >  * Since for each NAT entry we know the MAC binding to be used for
>> the
>> >external_ip - either the external_mac if set or the MAC of the
>> >distributed gateway router port, this patch adds the flows in the
>> >S_ROUTER_IN_ARP_RESOLVE stage to set the eth.dst to the MAC if
>> the
>> >IP destination is external_ip.
>> >  * The existing flows in the S_ROUTER_OUT_EGR_LOOP are now added by
>> additional
>> >match -  is_chassis_resident('P') - where 'P' is logical_port of
>> the NAT entry
>> >if set, otherwise it is the chassis resident port of distributed
>> router port.
>> >With this additional match, the packet will be loopbacked to
>> apply the unSNAT/DNAT
>> >rules on the relevant chassis.
>> >
>> > Suppose if a logical port 'P' with IP 'A' has a dnat_and_snat entry
>> with external_mac/logical_port
>> > set, and if the packet's IP destination is one of the DNAT IP - then
>> the packet will be sent out
>> > of the local chassis, since eth.dst is resolved in the
>> S_ROUTER_IN_ARP_RESOLVE stage.
>> > If the external_mac/logical_port is not in NAT entry, then the packet
>> will be redirected to
>> > the gateway chassis.
>> >
>> > With this patch, for the logical resource reported in @Reported-at, the
>> number of logical
>> > flows come down to around 45k from 650k.
>> >
>> > [1] - ceacd9d49316("ovn: distributed NAT flows")
>> >
>> > [2] - 551e3d989557("OVN: fix DVR Floating IP support")
>> >   8244c6b6bd88("OVN: do not distribute traffic for local FIP")
>> >
>> > Reported-at:
>> https://mail.openvswitch.org/pipermail/ovs-discuss/2020-January/049714.html
>> > Reported-by: Daniel Alvarez Sanchez 
>> > Signed-off-by: Numan Siddique 
>> > ---
>> >  northd/ovn-northd.8.xml | 191 +
>> >  northd/ovn-northd.c | 264 ++--
>> >  tests/ovn-northd.at |   8 +-
>> >  3 files changed, 99 insertions(+), 364 deletions(-)
>> >
>>
>> Hi Numan,
>>
>> The patch looks ok to me and it passes unit tests.
>>
>> Hi Daniel,
>> As this is quite a significant change it might be nice if you could also
>> try it out on your setup.
>>
>> Otherwise:
>>
>> Acked-by: Dumitru Ceara 
>>
>> Regards,
>> Dumitru
>>
>>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] ovn-northd: Address scale issues with DNAT flows.

[Daniel Alvarez Sanchez]

Hi Numan,

Thanks for this. I have deployed a setup with master branch and this patch
applied. The patch LGTM and I can confirm that it works. Moreover, the
number of lflows has been dramatically reduced from around 50

With 150 FIPs:

Before: ~70K lflows  https://imgur.com/a/OaVelCb
After (in the same setup): 2K lflows

[root@central ~]# ovn-sbctl list logical_flow | grep _uuid -c
2092
[root@central ~]# ovn-sbctl list logical_flow | grep lr_out_egr_loop -c
155
[root@central ~]# ovn-sbctl list logical_flow | grep lr_in_ip_routing -c
7

And I verified that FIP to FIP traffic works via the localnet port so it is
really distributed and does not traverse any tunnels.

Thanks!
Daniel

Tested-By: Daniel Alvarez Sanchez 
Acked-By: Daniel Alvarez Sanchez 

On Tue, Feb 4, 2020 at 1:59 PM Dumitru Ceara  wrote:

> On 2/1/20 11:53 AM, num...@ovn.org wrote:
> > From: Numan Siddique 
> >
> > When the commit [1] added Distributed NAT support in OVN, it didn't
> address
> > the requirement of making East/West NAT traffic distributed. The E/W NAT
> > traffic was still centralized. Later a couple of patches [2], addressed
> this
> > requirement. But the approach taken in [2] resulted in a lot of logical
> flows
> > as number of dnat_and_snat entries increase, as reported in @Reported-at.
> >
> > This patch
> >   - reverts the approch taken in [2].
> >   - removing the flows which does the NAT direct (REGBIT_NAT_REDIRECT) to
> > the gateway chassis.
> >   - and to solve the E/W centralized NAT it does the following:
> >  * Since for each NAT entry we know the MAC binding to be used for
> the
> >external_ip - either the external_mac if set or the MAC of the
> >distributed gateway router port, this patch adds the flows in the
> >S_ROUTER_IN_ARP_RESOLVE stage to set the eth.dst to the MAC if the
> >IP destination is external_ip.
> >  * The existing flows in the S_ROUTER_OUT_EGR_LOOP are now added by
> additional
> >match -  is_chassis_resident('P') - where 'P' is logical_port of
> the NAT entry
> >if set, otherwise it is the chassis resident port of distributed
> router port.
> >With this additional match, the packet will be loopbacked to
> apply the unSNAT/DNAT
> >rules on the relevant chassis.
> >
> > Suppose if a logical port 'P' with IP 'A' has a dnat_and_snat entry with
> external_mac/logical_port
> > set, and if the packet's IP destination is one of the DNAT IP - then the
> packet will be sent out
> > of the local chassis, since eth.dst is resolved in the
> S_ROUTER_IN_ARP_RESOLVE stage.
> > If the external_mac/logical_port is not in NAT entry, then the packet
> will be redirected to
> > the gateway chassis.
> >
> > With this patch, for the logical resource reported in @Reported-at, the
> number of logical
> > flows come down to around 45k from 650k.
> >
> > [1] - ceacd9d49316("ovn: distributed NAT flows")
> >
> > [2] - 551e3d989557("OVN: fix DVR Floating IP support")
> >   8244c6b6bd88("OVN: do not distribute traffic for local FIP")
> >
> > Reported-at:
> https://mail.openvswitch.org/pipermail/ovs-discuss/2020-January/049714.html
> > Reported-by: Daniel Alvarez Sanchez 
> > Signed-off-by: Numan Siddique 
> > ---
> >  northd/ovn-northd.8.xml | 191 +
> >  northd/ovn-northd.c | 264 ++--
> >  tests/ovn-northd.at |   8 +-
> >  3 files changed, 99 insertions(+), 364 deletions(-)
> >
>
> Hi Numan,
>
> The patch looks ok to me and it passes unit tests.
>
> Hi Daniel,
> As this is quite a significant change it might be nice if you could also
> try it out on your setup.
>
> Otherwise:
>
> Acked-by: Dumitru Ceara 
>
> Regards,
> Dumitru
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] ovn-controller: Add missing port group lflow references.

[Daniel Alvarez Sanchez]

Thanks for this patch. This can be a security issue as ACLs applied to a
Port Group may not be taking effect. Tested this patch on an OpenStack
environment that recreated the issue and I confirm that it fixes the
problem.

On Mon, Dec 2, 2019 at 1:40 PM Dumitru Ceara  wrote:

> The commit that adds incremental processing for port-group changes
> doesn't store logical flow references for port groups. If a port group
> is updated (e.g., a port is added) no logical flow recalculation will be
> performed.
>
> To fix this, when parsing the flow expression also store the referenced
> port groups and bind them to the logical flows that depend on them. If
> the port group is updated then the logical flows referring them will
> also be reinstalled.
>
> Reported-by: Daniel Alvarez 
> Reported-at: https://bugzilla.redhat.com/1778164
> CC: Han Zhou 
> Fixes: 978f5e90af0a ("ovn-controller: Incremental processing for
> port-group changes.")
> Signed-off-by: Dumitru Ceara 
>
> Change-Id: Id39695f022f16b598fd8416cd2494e7dab3bf11b
> ---
>  controller/lflow.c|  9 -
>  include/ovn/expr.h|  4 +++-
>  lib/actions.c |  4 ++--
>  lib/expr.c| 24 +---
>  tests/test-ovn.c  |  8 
>  utilities/ovn-trace.c |  2 +-
>  6 files changed, 35 insertions(+), 16 deletions(-)
>
> diff --git a/controller/lflow.c b/controller/lflow.c
> index 36150bd..a689320 100644
> --- a/controller/lflow.c
> +++ b/controller/lflow.c
> @@ -616,14 +616,21 @@ consider_logical_flow(
>  struct expr *expr;
>
>  struct sset addr_sets_ref = SSET_INITIALIZER(_sets_ref);
> +struct sset port_groups_ref = SSET_INITIALIZER(_groups_ref);
>  expr = expr_parse_string(lflow->match, , addr_sets,
> port_groups,
> - _sets_ref, );
> + _sets_ref, _groups_ref, );
>  const char *addr_set_name;
>  SSET_FOR_EACH (addr_set_name, _sets_ref) {
>  lflow_resource_add(lfrr, REF_TYPE_ADDRSET, addr_set_name,
> >header_.uuid);
>  }
> +const char *port_group_name;
> +SSET_FOR_EACH (port_group_name, _groups_ref) {
> +lflow_resource_add(lfrr, REF_TYPE_PORTGROUP, port_group_name,
> +   >header_.uuid);
> +}
>  sset_destroy(_sets_ref);
> +sset_destroy(_groups_ref);
>
>  if (!error) {
>  if (prereqs) {
> diff --git a/include/ovn/expr.h b/include/ovn/expr.h
> index 22f633e..21bf51c 100644
> --- a/include/ovn/expr.h
> +++ b/include/ovn/expr.h
> @@ -390,11 +390,13 @@ void expr_print(const struct expr *);
>  struct expr *expr_parse(struct lexer *, const struct shash *symtab,
>  const struct shash *addr_sets,
>  const struct shash *port_groups,
> -struct sset *addr_sets_ref);
> +struct sset *addr_sets_ref,
> +struct sset *port_groups_ref);
>  struct expr *expr_parse_string(const char *, const struct shash *symtab,
> const struct shash *addr_sets,
> const struct shash *port_groups,
> struct sset *addr_sets_ref,
> +   struct sset *port_groups_ref,
> char **errorp);
>
>  struct expr *expr_clone(struct expr *);
> diff --git a/lib/actions.c b/lib/actions.c
> index 586d7b7..051e6c8 100644
> --- a/lib/actions.c
> +++ b/lib/actions.c
> @@ -240,8 +240,8 @@ add_prerequisite(struct action_context *ctx, const
> char *prerequisite)
>  struct expr *expr;
>  char *error;
>
> -expr = expr_parse_string(prerequisite, ctx->pp->symtab, NULL, NULL,
> NULL,
> - );
> +expr = expr_parse_string(prerequisite, ctx->pp->symtab, NULL, NULL,
> + NULL, NULL, );
>  ovs_assert(!error);
>  ctx->prereqs = expr_combine(EXPR_T_AND, ctx->prereqs, expr);
>  }
> diff --git a/lib/expr.c b/lib/expr.c
> index 71de615..e5e4d21 100644
> --- a/lib/expr.c
> +++ b/lib/expr.c
> @@ -480,7 +480,8 @@ struct expr_context {
>  const struct shash *symtab;/* Symbol table. */
>  const struct shash *addr_sets; /* Address set table. */
>  const struct shash *port_groups; /* Port group table. */
> -struct sset *addr_sets_ref;   /* The set of address set
> referenced. */
> +struct sset *addr_sets_ref;  /* The set of address set
> referenced. */
> +struct sset *port_groups_ref;/* The set of port groups
> referenced. */
>  bool not;/* True inside odd n

Re: [ovs-dev] [PATCH ovn] ovn-controller: Add command to trigger an I-P full recompute.

[Daniel Alvarez Sanchez]

This is very handy! Can you please add the command to [0]?

[0]
https://github.com/ovn-org/ovn/blob/master/controller/ovn-controller.8.xml#L403

On Mon, Dec 2, 2019 at 5:19 PM Dumitru Ceara  wrote:

> Incremental processing tries to minimize the number of times
> ovn-controller has to fully reprocess the contents of the southbound
> database. However, if a bug in the I-P code causes ovn-controller to
> end up in an inconsistent state, we have no easy way to force a full
> recalculation of the openflow entries.
>
> This commit adds a new command to ovn-controller, "recompute", which
> allows users to force a full recompute of the database. It can be
> triggered by the user in the following way:
>
> ovn-appctl -t ovn-controller recompute
>
> Signed-off-by: Dumitru Ceara 
> ---
>  controller/ovn-controller.c | 14 ++
>  1 file changed, 14 insertions(+)
>
> diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c
> index c56190f..04d9dea 100644
> --- a/controller/ovn-controller.c
> +++ b/controller/ovn-controller.c
> @@ -73,6 +73,7 @@ static unixctl_cb_func meter_table_list;
>  static unixctl_cb_func group_table_list;
>  static unixctl_cb_func inject_pkt;
>  static unixctl_cb_func ovn_controller_conn_show;
> +static unixctl_cb_func engine_recompute_cmd;
>
>  #define DEFAULT_BRIDGE_NAME "br-int"
>  #define DEFAULT_PROBE_INTERVAL_MSEC 5000
> @@ -1941,6 +1942,9 @@ main(int argc, char *argv[])
>  unixctl_command_register("inject-pkt", "MICROFLOW", 1, 1, inject_pkt,
>   _pkt);
>
> +unixctl_command_register("recompute", "", 0, 0, engine_recompute_cmd,
> + NULL);
> +
>  uint64_t engine_run_id = 0;
>  bool engine_run_done = true;
>
> @@ -2442,3 +2446,13 @@ ovn_controller_conn_show(struct unixctl_conn *conn,
> int argc OVS_UNUSED,
>  }
>  unixctl_command_reply(conn, result);
>  }
> +
> +static void
> +engine_recompute_cmd(struct unixctl_conn *conn OVS_UNUSED, int argc
> OVS_UNUSED,
> + const char *argv[] OVS_UNUSED, void *arg OVS_UNUSED)
> +{
> +VLOG_INFO("User triggered force recompute.");
> +engine_set_force_recompute(true);
> +poll_immediate_wake();
> +unixctl_command_reply(conn, NULL);
> +}
> --
> 1.8.3.1
>

Reviewed-by: Daniel Alvarez 

>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] Exclude inport and outport symbol tables from conjunction

[Daniel Alvarez Sanchez]

Acked-by: Daniel Alvarez 


On Fri, Sep 13, 2019 at 11:02 PM Mark Michelson  wrote:
>
> Acked-by: Mark Michelson
>
> It sucks that we lose the efficiency of the conjunctive match altogether
> on port groups because of this error, but I understand this is a huge
> bug and needs fixing.
If I'm not mistaken, from OpenStack standpoint conjunction was *only*
being used when using port groups and ACLs that matched on port ranges
( e.g tcp.dst >= X && tcp.dst <=Y) which was not working. Therefore
we're not losing performance because it was already broken (given that
there was more than one ACL like that).

>
> Perhaps it would be good to start up a discussion on this list about a
> more longterm solution that would allow for conjunctive matches with no
> ambiguity.
Agreed! We already discussed some ideas on IRC but it'd be awesome to
have a thread and brainstorm there.

Thanks a lot everyone!
Daniel
>
> On 9/13/19 4:49 PM, nusid...@redhat.com wrote:
> > From: Numan Siddique 
> >
> > If there are multiple ACLs associated with a port group and they
> > match on a range of some field, then ovn-controller doesn't install
> > the flows properly and this results in broken ACL functionality.
> >
> > For example, if there is a port group - pg1 with logical ports - [p1, p2]
> > and if there are below ACLs (only match condition is shown)
> >
> > 1 -  outport == @pg1 && ip4 && tcp.dst >= 500 && tcp.dst <= 501
> > 2 -  outport == @pg1 && ip4 && tcp.dst >= 600 && tcp.dst <= 601
> >
> > The first ACL will result in the below OF flows
> >
> > 1.  conj_id=1,tcp
> > 2.  tcp,reg15=0x11: conjunction(1, 1/2)
> > 3.  tcp,reg15=0x12: conjunction(1, 1/2)
> > 5.  tcp,tp_dst=500: conjunction(1, 2/2)
> > 6.  tcp,tp_dst=501: conjunction(1, 2/2)
> >
> > The second ACL will result in the below OF flows
> > 7.  conj_id=2,tcp
> > 8.  tcp,reg15=0x11: conjunction(2, 1/2)
> > 9.  tcp,reg15=0x12: conjunction(2, 1/2)
> > 11. tcp,tp_dst=600: conjunction(2, 2/2)
> > 12. tcp,tp_dst=601: conjunction(2, 3/2)
> >
> > The OF flows (2) and (8) have the exact match but with different action.
> > This results in only one of the flows getting installed. The same goes
> > for the flows (3) and (9). And this completely breaks the ACL functionality
> > for such scenarios.
> >
> > In order to fix this issue, this patch excludes the 'inport' and 'outport' 
> > symbols
> > from conjunction. With this patch we will have the below flows.
> >
> > tcp,reg15=0x11,tp_dst=500
> > tcp,reg15=0x11,tp_dst=501
> > tcp,reg15=0x12,tp_dst=500
> > tcp,reg15=0x12,tp_dst=501
> > tcp,reg15=0x13,tp_dst=500
> > tcp,reg15=0x13,tp_dst=501
> > tcp,reg15=0x11,tp_dst=600
> > tcp,reg15=0x11,tp_dst=601
> > tcp,reg15=0x12,tp_dst=600
> > tcp,reg15=0x12,tp_dst=601
> > tcp,reg15=0x13,tp_dst=600
> > tcp,reg15=0x13,tp_dst=601
> >
> > Signed-off-by: Numan Siddique 
> > ---
> >   lib/expr.c   |  2 +-
> >   tests/ovn.at | 26 ++
> >   2 files changed, 27 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/expr.c b/lib/expr.c
> > index e4c650f7c..c0871e1e8 100644
> > --- a/lib/expr.c
> > +++ b/lib/expr.c
> > @@ -1499,7 +1499,7 @@ expr_symtab_add_string(struct shash *symtab, const 
> > char *name,
> >   const struct mf_field *field = mf_from_id(id);
> >   struct expr_symbol *symbol;
> >
> > -symbol = add_symbol(symtab, name, 0, prereqs, EXPR_L_NOMINAL, false,
> > +symbol = add_symbol(symtab, name, 0, prereqs, EXPR_L_NOMINAL, true,
> >   field->writable);
> >   symbol->field = field;
> >   return symbol;
> > diff --git a/tests/ovn.at b/tests/ovn.at
> > index 2a35b4e15..14d9f59b0 100644
> > --- a/tests/ovn.at
> > +++ b/tests/ovn.at
> > @@ -589,6 +589,24 @@ ip,reg14=0x6
> >   ipv6,reg14=0x5
> >   ipv6,reg14=0x6
> >   ])
> > +AT_CHECK([expr_to_flow 'inport == {"eth0", "eth1", "eth2"} && ip4 && tcp 
> > && tcp.dst == {500, 501}'], [0], [dnl
> > +tcp,reg14=0x5,tp_dst=500
> > +tcp,reg14=0x5,tp_dst=501
> > +tcp,reg14=0x6,tp_dst=500
> > +tcp,reg14=0x6,tp_dst=501
> > +])
> > +AT_CHECK([expr_to_flow 'outport == {"eth0", "eth1", "eth2"} && ip4 && tcp 
> > && tcp.src == {400, 401} && tcp.dst == {500, 501}'], [0], [dnl
> > +conj_id=1,tcp,reg15=0x5
> > +conj_id=2,tcp,reg15=0x6
> > +tcp,reg15=0

Re: [ovs-dev] [ovs-discuss] ovn-controller is taking 100% CPU all the time in one deployment

[Daniel Alvarez Sanchez]

Hi Han,

On Fri, Aug 30, 2019 at 10:37 PM Han Zhou  wrote:
>
> On Fri, Aug 30, 2019 at 1:25 PM Numan Siddique  wrote:
> >
> > Hi Han,
> >
> > I am thinking of this approach to solve this problem. I still need to
> test it.
> > If you have any comments or concerns do let me know.
> >
> >
> > **
> > diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c
> > index 9a282..a83b56362 100644
> > --- a/northd/ovn-northd.c
> > +++ b/northd/ovn-northd.c
> > @@ -6552,6 +6552,41 @@ build_lrouter_flows(struct hmap *datapaths, struct
> hmap *ports,
> >
> >  }
> >
> > +/* Handle GARP reply packets received on a distributed router
> gateway
> > + * port. GARP reply broadcast packets could be sent by external
> > + * switches. We don't want them to be handled by all the
> > + * ovn-controllers if they receive it. So add a priority-92 flow
> to
> > + * apply the put_arp action on a redirect chassis and drop it on
> > + * other chassis.
> > + * Note that we are already adding a priority-90 logical flow in
> the
> > + * table S_ROUTER_IN_IP_INPUT to apply the put_arp action if
> > + * arp.op == 2.
> > + * */
> > +if (op->od->l3dgw_port && op == op->od->l3dgw_port
> > +&& op->od->l3redirect_port) {
> > +for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) {
> > +ds_clear();
> > +ds_put_format(,
> > +  "inport == %s && is_chassis_resident(%s)
> && "
> > +  "eth.bcast && arp.op == 2 && arp.spa ==
> %s/%u",
> > +  op->json_key,
> op->od->l3redirect_port->json_key,
> > +  op->lrp_networks.ipv4_addrs[i].network_s,
> > +  op->lrp_networks.ipv4_addrs[i].plen);
> > +ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 92,
> > +  ds_cstr(),
> > +  "put_arp(inport, arp.spa, arp.sha);");
> > +ds_clear();
> > +ds_put_format(,
> > +  "inport == %s && !is_chassis_resident(%s)
> && "
> > +  "eth.bcast && arp.op == 2 && arp.spa ==
> %s/%u",
> > +  op->json_key,
> op->od->l3redirect_port->json_key,
> > +  op->lrp_networks.ipv4_addrs[i].network_s,
> > +  op->lrp_networks.ipv4_addrs[i].plen);
> > +ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 92,
> > +  ds_cstr(), "drop;");
> > +}
> > +}
> > +
> >  /* A set to hold all load-balancer vips that need ARP responses.
> */
> >  struct sset all_ips = SSET_INITIALIZER(_ips);
> >  int addr_family;
> > *
> >
> > If a physical switch sends GARP request packets we have existing logical
> flows
> > which handle them only on the gateway chassis.
> >
> > But if the physical switch sends GARP reply packets, then these packets
> > are handled by ovn-controllers where bridge mappings are configured.
> > I think its good enough if the gateway chassis handles these packet.
> >
> > In the deployment where we are seeing this issue, the physical switch
> sends GARP reply
> > packets.
> >
> > Thanks
> > Numan
> >
> >
> Hi Numan,
>
> I think both GARP request and reply should be handled on all chassises. It
> should work not only for physical switch, but also for virtual workloads.
> At least our current use cases relies on that.

I believe that Numan's patch will not change the behavior for virtual
(OVN) workloads, does it?

Although I'm in favor of this patch, I still think that it's not
enough for non-Incremental Processing versions of OVS because even
we're going to release pressure on the compute nodes, still on loaded
systems, the gateway nodes are going to be hogging the CPU. Plus, I
think there's value even from a security standpoint in having it on
stable branches as it looks like a simple attack vector.

>
> Thanks,
> Han
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [ovs-discuss] ovn-controller is taking 100% CPU all the time in one deployment

[Daniel Alvarez Sanchez]

On Fri, Aug 30, 2019 at 8:18 PM Han Zhou  wrote:
>
>
>
> On Fri, Aug 30, 2019 at 6:46 AM Mark Michelson  wrote:
> >
> > On 8/30/19 5:39 AM, Daniel Alvarez Sanchez wrote:
> > > On Thu, Aug 29, 2019 at 10:01 PM Mark Michelson  
> > > wrote:
> > >>
> > >> On 8/29/19 2:39 PM, Numan Siddique wrote:
> > >>> Hello Everyone,
> > >>>
> > >>> In one of the OVN deployments, we are seeing 100% CPU usage by
> > >>> ovn-controllers all the time.
> > >>>
> > >>> After investigations we found the below
> > >>>
> > >>>- ovn-controller is taking more than 20 seconds to complete full loop
> > >>> (mainly in lflow_run() function)
> > >>>
> > >>>- The physical switch is sending GARPs periodically every 10 seconds.
> > >>>
> > >>>- There is ovn-bridge-mappings configured and these GARP packets
> > >>> reaches br-int via the patch port.
> > >>>
> > >>>- We have a flow in router pipeline which applies the action - 
> > >>> put_arp
> > >>> if it is arp packet.
> > >>>
> > >>>- ovn-controller pinctrl thread receives these garps, stores the
> > >>> learnt mac-ips in the 'put_mac_bindings' hmap and notifies the
> > >>> ovn-controller main thread by incrementing the seq no.
> > >>>
> > >>>- In the ovn-controller main thread, after lflow_run() finishes,
> > >>> pinctrl_wait() is called. This function calls - poll_immediate_wake() as
> > >>> 'put_mac_bindings' hmap is not empty.
> > >>>
> > >>> - This causes the ovn-controller poll_block() to not sleep at all and
> > >>> this repeats all the time resulting in 100% cpu usage.
> > >>>
> > >>> The deployment has OVS/OVN 2.9.  We have back ported the pinctrl_thread
> > >>> patch.
> > >>>
> > >>> Some time back I had reported an issue about lflow_run() taking lot of
> > >>> time - 
> > >>> https://mail.openvswitch.org/pipermail/ovs-dev/2019-July/360414.html
> > >>>
> > >>> I think we need to improve the logical processing sooner or later.
> > >>
> > >> I agree that this is very important. I know that logical flow processing
> > >> is the biggest bottleneck for ovn-controller, but 20 seconds is just
> > >> ridiculous. In your scale testing, you found that lflow_run() was taking
> > >> 10 seconds to complete.
> > > I support this statement 100% (20 seconds is just ridiculous). To be
> > > precise, in this deployment we see over 23 seconds for the main loop
> > > to process and I've seen even 30 seconds some times. I've been talking
> > > to Numan these days about this issue and I support profiling this
> > > actual deployment so that we can figure out how incremental processing
> > > would help.
> > >
> > >>
> > >> I'm curious if there are any factors in this particular deployment's
> > >> configuration that might contribute to this. For instance, does this
> > >> deployment have a glut of ACLs? Are they not using port groups?
> > > They're not using port groups because it's 2.9 and it is not there.
> > > However, I don't think port groups would make a big difference in
> > > terms of ovn-controller computation. I might be wrong but Port Groups
> > > help reduce the number of ACLs in the NB database while the # of
> > > Logical Flows would still remain the same. We'll try to get the
> > > contents of the NB database and figure out what's killing it.
> > >
> >
> > You're right that port groups won't reduce the number of logical flows.
>
> I think port-group reduces number of logical flows significantly, and also 
> reduces OVS flows when conjunctive matches are effective.

Right, definitely the number of lflows will be much lower. My bad as I
was directly involved in this! :) I was just thinking that the number
of OVS flows will remain the same so the computation for
ovn-controller would be similar but I missed the conjunctive matches
part in my statement.


> Please see my calculation here: 
> https://www.slideshare.net/hanzhou1978/large-scale-overlay-networks-with-ovn-problems-and-solutions/30
>
> > However, it can reduce the computation in ovn-controller. The reason is
> > that the logical flows generated by ACLs that use port groups may result
> > in conjunc

Re: [ovs-dev] [ovs-discuss] ovn-controller is taking 100% CPU all the time in one deployment

[Daniel Alvarez Sanchez]

On Thu, Aug 29, 2019 at 10:01 PM Mark Michelson  wrote:
>
> On 8/29/19 2:39 PM, Numan Siddique wrote:
> > Hello Everyone,
> >
> > In one of the OVN deployments, we are seeing 100% CPU usage by
> > ovn-controllers all the time.
> >
> > After investigations we found the below
> >
> >   - ovn-controller is taking more than 20 seconds to complete full loop
> > (mainly in lflow_run() function)
> >
> >   - The physical switch is sending GARPs periodically every 10 seconds.
> >
> >   - There is ovn-bridge-mappings configured and these GARP packets
> > reaches br-int via the patch port.
> >
> >   - We have a flow in router pipeline which applies the action - put_arp
> > if it is arp packet.
> >
> >   - ovn-controller pinctrl thread receives these garps, stores the
> > learnt mac-ips in the 'put_mac_bindings' hmap and notifies the
> > ovn-controller main thread by incrementing the seq no.
> >
> >   - In the ovn-controller main thread, after lflow_run() finishes,
> > pinctrl_wait() is called. This function calls - poll_immediate_wake() as
> > 'put_mac_bindings' hmap is not empty.
> >
> > - This causes the ovn-controller poll_block() to not sleep at all and
> > this repeats all the time resulting in 100% cpu usage.
> >
> > The deployment has OVS/OVN 2.9.  We have back ported the pinctrl_thread
> > patch.
> >
> > Some time back I had reported an issue about lflow_run() taking lot of
> > time - https://mail.openvswitch.org/pipermail/ovs-dev/2019-July/360414.html
> >
> > I think we need to improve the logical processing sooner or later.
>
> I agree that this is very important. I know that logical flow processing
> is the biggest bottleneck for ovn-controller, but 20 seconds is just
> ridiculous. In your scale testing, you found that lflow_run() was taking
> 10 seconds to complete.
I support this statement 100% (20 seconds is just ridiculous). To be
precise, in this deployment we see over 23 seconds for the main loop
to process and I've seen even 30 seconds some times. I've been talking
to Numan these days about this issue and I support profiling this
actual deployment so that we can figure out how incremental processing
would help.

>
> I'm curious if there are any factors in this particular deployment's
> configuration that might contribute to this. For instance, does this
> deployment have a glut of ACLs? Are they not using port groups?
They're not using port groups because it's 2.9 and it is not there.
However, I don't think port groups would make a big difference in
terms of ovn-controller computation. I might be wrong but Port Groups
help reduce the number of ACLs in the NB database while the # of
Logical Flows would still remain the same. We'll try to get the
contents of the NB database and figure out what's killing it.

>
> This particular deployment's configuration may give us a good scenario
> for our testing to improve lflow processing time.
Absolutely!
>
> >
> > But to fix this issue urgently, we are thinking of the below approach.
> >
> >   - pinctrl_thread will locally cache the mac_binding entries (just like
> > it caches the dns entries). (Please note pinctrl_thread can not access
> > the SB DB IDL).
>
> >
> > - Upon receiving any arp packet (via the put_arp action), pinctrl_thread
> > will check the local mac_binding cache and will only wake up the main
> > ovn-controller thread only if the mac_binding update is required.
> >
> > This approach will solve the issue since the MAC sent by the physical
> > switches will not change. So there is no need to wake up ovn-controller
> > main thread.
>
> I think this can work well. We have a lot of what's needed already in
> pinctrl at this point. We have the hash table of mac bindings already.
> Currently, we flush this table after we write the data to the southbound
> database. Instead, we would keep the bindings in memory. We would need
> to ensure that the in-memory MAC bindings eventually get deleted if they
> become stale.
>
> >
> > In the present master/2.12 these GARPs will not cause this 100% cpu loop
> > issue because incremental processing will not recompute flows.
>
> Another mitigating factor for master is something I'm currently working
> on. I've got the beginnings of a patch series going where I am
> separating pinctrl into a separate process from ovn-controller:
> https://github.com/putnopvut/ovn/tree/pinctrl_process
>
> It's in the early stages right now, so please don't judge :)
>
> Separating pinctrl to its own process means that it cannot directly
> cause ovn-controller to wake up like it currently might.
>
> >
> > Even though the above approach is not really required for master/2.12, I
> > think it is still Ok to have this as there is no harm.
> >
> > I would like to know your comments and any concerns if any.
>
> Hm, I don't really understand why we'd want to put this in master/2.12
> if the problem doesn't exist there. The main concern I have is with
> regards to cache lifetime. I don't want to introduce potential memory
> growth concerns 

Re: [ovs-dev] [PATCH v2, ovn] Make pid_exists() more robust against empty pid argument

[Daniel Alvarez Sanchez]

On Wed, Aug 14, 2019 at 5:49 PM Michele Baldessari  wrote:
>
> In some of our destructive testing of ovn-dbs inside containers managed
> by pacemaker we reached a situation where /var/run/openvswitch had
> empty .pid files. The current code does not deal well with them
> and pidfile_is_running() returns true in such a case and this confuses
> the OCF resource agent.
>
> - Before this change:
> Inside a container run:
>   killall ovsdb-server;
>   echo -n '' > /var/run/openvswitch/ovnnb_db.pid; echo -n '' > 
> /var/run/openvswitch/ovnsb_db.pid
>
> We will observe that the cluster is unable to ever recover because
> it believes the ovn processes to be running when they really aren't and
> eventually just fails:
>  podman container set: ovn-dbs-bundle 
> [192.168.24.1:8787/rhosp15/openstack-ovn-northd:pcmklatest]
>ovn-dbs-bundle-0 (ocf::ovn:ovndb-servers):   Master controller-0
>ovn-dbs-bundle-1 (ocf::ovn:ovndb-servers):   Stopped controller-1
>ovn-dbs-bundle-2 (ocf::ovn:ovndb-servers):   Slave controller-2
>
> Let's make sure pid_exists() returns false when the pid is an empty
> string.
>
> - After this change the cluster is able to recover from this state and
> correctly start the resource:
>  podman container set: ovn-dbs-bundle 
> [192.168.24.1:8787/rhosp15/openstack-ovn-northd:pcmklatest]
>ovn-dbs-bundle-0 (ocf::ovn:ovndb-servers):   Master controller-0
>ovn-dbs-bundle-1 (ocf::ovn:ovndb-servers):   Slave controller-1
>ovn-dbs-bundle-2 (ocf::ovn:ovndb-servers):   Slave controller-2
>
> Fixes: 3028ce2595c8 ("ovs-lib: Allow "status" command to work as non-root.")
>
> Signed-off-by: Michele Baldessari 
> ---
> v1 -> v2
> 
> - Implemented Ilya's suggestion and moved the check from
>   pidfile_is_running() to pid_exists() and re-run my tests
> ---
>  utilities/ovs-lib.in | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/utilities/ovs-lib.in b/utilities/ovs-lib.in
> index fa840ec637f5..dc485413ef0c 100644
> --- a/utilities/ovs-lib.in
> +++ b/utilities/ovs-lib.in
> @@ -127,7 +127,7 @@ fi
>  pid_exists () {
>  # This is better than "kill -0" because it doesn't require permission to
>  # send a signal (so daemon_status in particular works as non-root).
> -test -d /proc/"$1"
> +test -n "$1" && test -d /proc/"$1"
>  }
>
>  pid_comm_check () {
> --
> 2.21.0

Acked-By:  Daniel Alvarez 
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH branch-2.12] [ovn-northd] Don't emit ICMP need to frag on LRP with no IPv4 address

[Daniel Alvarez]

Prior to this patch, when a LRP has only IPv6 addresses, ovn-northd
will crash (SIGSEV) because the current code injects a flow to
emit the ICMP need-to-frag from its IPv4 address which does not
exist.

This patch is adding a check to skip the flow installation in case
the port does not have any IPv4 address.

Signed-off-by: Daniel Alvarez 
---
 ovn/northd/ovn-northd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
index ae09cf338..21e546d38 100644
--- a/ovn/northd/ovn-northd.c
+++ b/ovn/northd/ovn-northd.c
@@ -7855,7 +7855,8 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap 
*ports,
 for (size_t i = 0; i < od->nbr->n_ports; i++) {
 struct ovn_port *rp = ovn_port_find(ports,
 od->nbr->ports[i]->name);
-if (!rp || rp == od->l3dgw_port) {
+if (!rp || rp == od->l3dgw_port ||
+!rp->lrp_networks.ipv4_addrs) {
 continue;
 }
 ds_clear();
--
1.8.3.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2 branch-2.11] ovsdb-server: drop all connections on read/write status change

[Daniel Alvarez]

Prior to this patch, only db change aware connections were dropped
on a read/write status change. However, current schema in OVN does
not allow clients to monitor whether a particular DB changes this
status. In order to accomplish this, we'd need to change the schema
and adapting ovsdb-server and existing clients.

Before tackling that, this patch is changing ovsdb-server to drop
*all* the existing connections upon a read/write status change. This
will force clients to reconnect and honor the change.

Reported-at:
https://mail.openvswitch.org/pipermail/ovs-discuss/2019-July/048981.html
Signed-off-by: Daniel Alvarez 
Signed-off-by: Ben Pfaff 
---
 ovsdb/jsonrpc-server.c |  2 +-
 tests/ovn.at   | 21 +
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/ovsdb/jsonrpc-server.c b/ovsdb/jsonrpc-server.c
index 7c7a277f0..739e0e72e 100644
--- a/ovsdb/jsonrpc-server.c
+++ b/ovsdb/jsonrpc-server.c
@@ -365,7 +365,7 @@ ovsdb_jsonrpc_server_set_read_only(struct 
ovsdb_jsonrpc_server *svr,
 {
 if (svr->read_only != read_only) {
 svr->read_only = read_only;
-ovsdb_jsonrpc_server_reconnect(svr, false,
+ovsdb_jsonrpc_server_reconnect(svr, true,
xstrdup(read_only
? "making server read-only"
: "making server read/write"));
diff --git a/tests/ovn.at b/tests/ovn.at
index 4b14720f3..291330b3b 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -12090,3 +12090,24 @@ ovn-nbctl list logical_switch_port
 ovn-nbctl list logical_router_port

 AT_CLEANUP
+
+# Run ovn-nbctl in daemon mode, change to a backup database and verify that
+# an insert operation is not allowed.
+AT_SETUP([ovn -- can't write to a backup database server instance])
+ovn_start
+on_exit 'kill $(cat ovn-nbctl.pid)'
+export OVN_NB_DAEMON=$(ovn-nbctl --pidfile --detach)
+
+AT_CHECK([ovn-nbctl ls-add sw0])
+as ovn-nb
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep active | 
wc -l], [0], [1
+])
+ovs-appctl -t ovsdb-server ovsdb-server/set-active-ovsdb-server 
tcp:192.0.2.2:6641
+ovs-appctl -t ovsdb-server ovsdb-server/connect-active-ovsdb-server
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep -c 
backup], [0], [1
+])
+AT_CHECK([ovn-nbctl ls-add sw1], [1], [ignore],
+[ovn-nbctl: transaction error: {"details":"insert operation not allowed when 
database server is in read only mode","error":"not allowed"}
+])
+
+AT_CLEANUP
--
2.21.0 (Apple Git-120)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] [ovn-northd] Don't emit ICMP need to frag on LRP with no IPv4 address

[Daniel Alvarez]

Prior to this patch, when a LRP has only IPv6 addresses, ovn-northd
will crash (SIGSEV) because the current code injects a flow to
emit the ICMP need-to-frag from its IPv4 address which does not
exist.

This patch is adding a check to skip the flow installation in case
the port does not have any IPv4 address.

Signed-off-by: Daniel Alvarez 
---
 ovn/northd/ovn-northd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
index eb6c47cad..3542ba72f 100644
--- a/ovn/northd/ovn-northd.c
+++ b/ovn/northd/ovn-northd.c
@@ -7705,7 +7705,8 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap 
*ports,
 for (size_t i = 0; i < od->nbr->n_ports; i++) {
 struct ovn_port *rp = ovn_port_find(ports,
 od->nbr->ports[i]->name);
-if (!rp || rp == od->l3dgw_port) {
+if (!rp || rp == od->l3dgw_port ||
+!rp->lrp_networks.ipv4_addrs) {
 continue;
 }
 ds_clear();
-- 
2.21.0 (Apple Git-120)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] Shutdown SSL connection before closing socket

[Daniel Alvarez Sanchez]

Acked-By: Daniel Alvarez 

On Thu, Jul 11, 2019 at 3:08 PM Terry Wilson  wrote:
>
> Without shutting down the SSL connection, log messages like:
>
> stream_ssl|WARN|SSL_read: unexpected SSL connection close
> jsonrpc|WARN|ssl:127.0.0.1:47052: receive error: Protocol error
> reconnect|WARN|ssl:127.0.0.1:47052: connection dropped (Protocol error)
>
> would occur whenever the socket is closed. This just adds an
> SSLStream.close() that calls shutdown() and ignores SSL errors, the
> same way that lib/stream-ssl.c does in ssl_close().
>
> Signed-off-by: Terry Wilson 
> ---
>  python/ovs/stream.py | 8 
>  1 file changed, 8 insertions(+)
>
> diff --git a/python/ovs/stream.py b/python/ovs/stream.py
> index c15be4b..a98057e 100644
> --- a/python/ovs/stream.py
> +++ b/python/ovs/stream.py
> @@ -825,6 +825,14 @@ class SSLStream(Stream):
>  except SSL.SysCallError as e:
>  return -ovs.socket_util.get_exception_errno(e)
>
> +def close(self):
> +if self.socket:
> +try:
> +self.socket.shutdown()
> +except SSL.Error:
> +pass
> +return super(SSLStream, self).close()
> +
>
>  if SSL:
>  # Register SSL only if the OpenSSL module is available
> --
> 1.8.3.1
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH branch2.11] ovsdb-server: drop all connections on read/write status change

[Daniel Alvarez Sanchez]

On Fri, Jul 12, 2019 at 4:17 PM Daniel Alvarez  wrote:
>
> Prior to this patch, only db change aware connections were dropped
> on a read/write status change. However, current schema in OVN does
> not allow clients to monitor whether a particular DB changes this
> status. In order to accomplish this, we'd need to change the schema
> and adapting ovsdb-server and existing clients.
>
> Before tackling that, this patch is changing ovsdb-server to drop
> *all* the existing connections upon a read/write status change. This
> will force clients to reconnect and honor the change.
>
> Reported-at: 
> https://mail.openvswitch.org/pipermail/ovs-discuss/2019-July/048981.html
> Signed-off-by: Daniel Alvarez 
> Signed-off-by: Ben Pfaff 
> ---
>  ovsdb/jsonrpc-server.c |  2 +-
>  tests/ovn.at   | 21 +
>  2 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/ovsdb/jsonrpc-server.c b/ovsdb/jsonrpc-server.c
> index 7c7a277f0..739e0e72e 100644
> --- a/ovsdb/jsonrpc-server.c
> +++ b/ovsdb/jsonrpc-server.c
> @@ -365,7 +365,7 @@ ovsdb_jsonrpc_server_set_read_only(struct 
> ovsdb_jsonrpc_server *svr,
>  {
>  if (svr->read_only != read_only) {
>  svr->read_only = read_only;
> -ovsdb_jsonrpc_server_reconnect(svr, false,
> +ovsdb_jsonrpc_server_reconnect(svr, true,
> xstrdup(read_only
> ? "making server read-only"
> : "making server 
> read/write"));
> diff --git a/tests/ovn.at b/tests/ovn.at
> index 4b14720f3..291330b3b 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -12090,3 +12090,24 @@ ovn-nbctl list logical_switch_port
>  ovn-nbctl list logical_router_port
>
>  AT_CLEANUP
> +
> +# Run ovn-nbctl in daemon mode, change to a backup database and verify that
> +# an insert operation is not allowed.
> +AT_SETUP([ovn -- can't write to a backup database server instance])
> +ovn_start
> +on_exit 'kill $(cat ovn-nbctl.pid)'
> +export OVN_NB_DAEMON=$(ovn-nbctl --pidfile --detach)
> +
> +AT_CHECK([ovn-nbctl ls-add sw0])
> +as ovn-nb
> +AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep active 
> | wc -l], [0], [1
> +])
> +ovs-appctl -t ovsdb-server ovsdb-server/set-active-ovsdb-server 
> tcp:192.0.2.2:6641
> +ovs-appctl -t ovsdb-server ovsdb-server/connect-active-ovsdb-server
> +AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep -c 
> backup], [0], [1
> +])
> +AT_CHECK([ovn-nbctl ls-add sw1], [1], [ignore],
> +[ovn-nbctl: transaction error: {"details":"insert operation not allowed when 
> database server is in read only mode","error":"not allowed"}
> +])
> +
> +AT_CLEANUP
> --
Sorry, disregard this patch. The test is failing, I'll try to take a
look ASAP (next week).
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH branch2.11] ovsdb-server: drop all connections on read/write status change

[Daniel Alvarez]

Prior to this patch, only db change aware connections were dropped
on a read/write status change. However, current schema in OVN does
not allow clients to monitor whether a particular DB changes this
status. In order to accomplish this, we'd need to change the schema
and adapting ovsdb-server and existing clients.

Before tackling that, this patch is changing ovsdb-server to drop
*all* the existing connections upon a read/write status change. This
will force clients to reconnect and honor the change.

Reported-at: 
https://mail.openvswitch.org/pipermail/ovs-discuss/2019-July/048981.html
Signed-off-by: Daniel Alvarez 
Signed-off-by: Ben Pfaff 
---
 ovsdb/jsonrpc-server.c |  2 +-
 tests/ovn.at   | 21 +
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/ovsdb/jsonrpc-server.c b/ovsdb/jsonrpc-server.c
index 7c7a277f0..739e0e72e 100644
--- a/ovsdb/jsonrpc-server.c
+++ b/ovsdb/jsonrpc-server.c
@@ -365,7 +365,7 @@ ovsdb_jsonrpc_server_set_read_only(struct 
ovsdb_jsonrpc_server *svr,
 {
 if (svr->read_only != read_only) {
 svr->read_only = read_only;
-ovsdb_jsonrpc_server_reconnect(svr, false,
+ovsdb_jsonrpc_server_reconnect(svr, true,
xstrdup(read_only
? "making server read-only"
: "making server read/write"));
diff --git a/tests/ovn.at b/tests/ovn.at
index 4b14720f3..291330b3b 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -12090,3 +12090,24 @@ ovn-nbctl list logical_switch_port
 ovn-nbctl list logical_router_port

 AT_CLEANUP
+
+# Run ovn-nbctl in daemon mode, change to a backup database and verify that
+# an insert operation is not allowed.
+AT_SETUP([ovn -- can't write to a backup database server instance])
+ovn_start
+on_exit 'kill $(cat ovn-nbctl.pid)'
+export OVN_NB_DAEMON=$(ovn-nbctl --pidfile --detach)
+
+AT_CHECK([ovn-nbctl ls-add sw0])
+as ovn-nb
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep active | 
wc -l], [0], [1
+])
+ovs-appctl -t ovsdb-server ovsdb-server/set-active-ovsdb-server 
tcp:192.0.2.2:6641
+ovs-appctl -t ovsdb-server ovsdb-server/connect-active-ovsdb-server
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep -c 
backup], [0], [1
+])
+AT_CHECK([ovn-nbctl ls-add sw1], [1], [ignore],
+[ovn-nbctl: transaction error: {"details":"insert operation not allowed when 
database server is in read only mode","error":"not allowed"}
+])
+
+AT_CLEANUP
--

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovsdb-server: drop all connections on read/write status change

[Daniel Alvarez Sanchez]

On Wed, Jul 10, 2019 at 10:24 PM Ben Pfaff  wrote:
>
> On Tue, Jul 09, 2019 at 12:16:30PM +0200, Daniel Alvarez wrote:
> > Prior to this patch, only db change aware connections were dropped
> > on a read/write status change. However, current schema in OVN does
> > not allow clients to monitor whether a particular DB changes this
> > status. In order to accomplish this, we'd need to change the schema
> > and adapting ovsdb-server and existing clients.
> >
> > Before tackling that, this patch is changing ovsdb-server to drop
> > *all* the existing connections upon a read/write status change. This
> > will force clients to reconnect and honor the change.
> >
> > Reported-at: 
> > https://mail.openvswitch.org/pipermail/ovs-discuss/2019-July/048981.html
> > Signed-off-by: Daniel Alvarez 
>
> Thanks, applied to master.
Thanks a lot Ben. Do you think it would make sense to backport it as
far as we could get to the stable branches?
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] ovsdb-server: drop all connections on read/write status change

[Daniel Alvarez]

Prior to this patch, only db change aware connections were dropped
on a read/write status change. However, current schema in OVN does
not allow clients to monitor whether a particular DB changes this
status. In order to accomplish this, we'd need to change the schema
and adapting ovsdb-server and existing clients.

Before tackling that, this patch is changing ovsdb-server to drop
*all* the existing connections upon a read/write status change. This
will force clients to reconnect and honor the change.

Reported-at: 
https://mail.openvswitch.org/pipermail/ovs-discuss/2019-July/048981.html
Signed-off-by: Daniel Alvarez 
---
 ovsdb/jsonrpc-server.c |  2 +-
 tests/ovn.at   | 21 +
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/ovsdb/jsonrpc-server.c b/ovsdb/jsonrpc-server.c
index 4dda63a9d..ddbbc2e94 100644
--- a/ovsdb/jsonrpc-server.c
+++ b/ovsdb/jsonrpc-server.c
@@ -365,7 +365,7 @@ ovsdb_jsonrpc_server_set_read_only(struct 
ovsdb_jsonrpc_server *svr,
 {
 if (svr->read_only != read_only) {
 svr->read_only = read_only;
-ovsdb_jsonrpc_server_reconnect(svr, false,
+ovsdb_jsonrpc_server_reconnect(svr, true,
xstrdup(read_only
? "making server read-only"
: "making server read/write"));
diff --git a/tests/ovn.at b/tests/ovn.at
index daace1128..4da7059b3 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -14313,3 +14313,24 @@ OVN_CHECK_PACKETS([hv2/vif22-tx.pcap], 
[vif22.expected])
 OVN_CLEANUP([hv1],[hv2])
 
 AT_CLEANUP
+
+# Run ovn-nbctl in daemon mode, change to a backup database and verify that
+# an insert operation is not allowed.
+AT_SETUP([ovn -- can't write to a backup database server instance])
+ovn_start
+on_exit 'kill $(cat ovn-nbctl.pid)'
+export OVN_NB_DAEMON=$(ovn-nbctl --pidfile --detach)
+
+AT_CHECK([ovn-nbctl ls-add sw0])
+as ovn-nb
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep active | 
wc -l], [0], [1
+])
+ovs-appctl -t ovsdb-server ovsdb-server/set-active-ovsdb-server 
tcp:192.0.2.2:6641
+ovs-appctl -t ovsdb-server ovsdb-server/connect-active-ovsdb-server
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/sync-status | grep -c 
backup], [0], [1
+])
+AT_CHECK([ovn-nbctl ls-add sw1], [1], [ignore],
+[ovn-nbctl: transaction error: {"details":"insert operation not allowed when 
database server is in read only mode","error":"not allowed"}
+])
+
+AT_CLEANUP
-- 
2.21.0 (Apple Git-120)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] OVN resource agent - make promotion synchronous

[Daniel Alvarez Sanchez]

Thanks a lot Michele.
Just mentioning that this has been tested in an OpenStack environment
successfully. A timeout is not needed for the while loop since
pacemaker will enforce its own.

On Tue, Jul 9, 2019 at 9:20 AM Michele Baldessari  wrote:
>
> Currently inside the ovsdb_server_promote() function we call 'promote_ovnnb'
> and 'promote_ovnsb' and then just record the new master state in the
> CIB.
>
> This creates a race because those two promote commands are asynchronous
> so when we exit the ovsdb_server_promote() function the underlying DBs
> are not guaranteed to be in master state. That means that clients might
> connect to an instance that is in read-only mode.
>
> We add a simple sleep loop where we wait for the underlying DB state to
> confirm the master state. We do not need to add a timeout loop because
> in case of an issue the resource timeout set within pacemaker will kick
> in and the resource agent script will be killed by pacemaker.
>
> Tested this within an openstack environment using ovn with roughly ~20
> reboots and was unable to trigger the issue (before the patch we would
> trigger the issue after a couple of reboots tops).
>
> Signed-off-by: Michele Baldessari 
> ---
>  ovn/utilities/ovndb-servers.ocf | 12 +++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/ovn/utilities/ovndb-servers.ocf b/ovn/utilities/ovndb-servers.ocf
> index 10313304cb7c..cd47426689ef 100755
> --- a/ovn/utilities/ovndb-servers.ocf
> +++ b/ovn/utilities/ovndb-servers.ocf
> @@ -516,6 +516,8 @@ ovsdb_server_stop() {
>  }
>
>  ovsdb_server_promote() {
> +local state
> +
>  ovsdb_server_check_status ignore_northd
>  rc=$?
>  case $rc in
> @@ -540,7 +542,15 @@ ovsdb_server_promote() {
>  ${OVN_CTL} --ovn-manage-ovsdb=no start_northd
>  fi
>
> -ocf_log debug "ovndb_servers: Promoting $host_name as the master"
> +ocf_log debug "ovndb_servers: Waiting for promotion $host_name as master 
> to complete"
> +ovsdb_server_check_status
> +state=$?
> +while [ "$state" != "$OCF_RUNNING_MASTER" ]; do
> +  sleep 1
> +  ovsdb_server_check_status
> +  state=$?
> +done
> +ocf_log debug "ovndb_servers: Promotion of $host_name as the master 
> completed"
>  # Record ourselves so that the agent has a better chance of doing
>  # the right thing at startup
>  ${CRM_ATTR_REPL_INFO} -v "$host_name"
> --
> 2.21.0

Acked-By: Daniel Alvarez 
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] OVN: fix DVR Floating IP support

[Daniel Alvarez Sanchez]

Even I successfully tested this before getting merged, I just hit some
scenario where traffic goes to the gateway node. If we have logical
port lp1 with a dnat_and_snat NAT rule to fip1 and from lp1 you ping
fip1, the traffic is leaving the hypervisor to the gateway node and
comes back to lp1.

I'd expect this traffic to be recirculated within the OVN bridge and
not being pushed out through the localnet port. It may be a corner
case but I believe it should be taken into account. What do you folks
think?

Cheers,
Daniel

On Tue, Apr 16, 2019 at 7:41 PM Ben Pfaff  wrote:
>
> On Sat, Apr 06, 2019 at 05:42:52PM +0200, Lorenzo Bianconi wrote:
> > When DVR is enabled FIP traffic need to be forwarded directly using
> > external connection to the underlay network and not be distributed
> > through geneve tunnels.
> > Fix this adding new logical flows to take care of distributed DNAT/SNAT
> >
> > Acked-by: Mark Michelson 
> > Signed-off-by: Lorenzo Bianconi 
>
> Thanks, applied to master.
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v3] OVN: Add support for Transport Zones

[Daniel Alvarez Sanchez]

Tested-by: Daniel Alvarez 

On Thu, Mar 28, 2019 at 11:37 AM  wrote:
>
> From: Lucas Alvares Gomes 
>
> This patch is adding support for Transport Zones. Transport zones (a.k.a
> TZs) is way to enable users of OVN to separate Chassis into different
> logical groups that will form tunnels only between members of the same
> group(s).
>
> Each Chassis can belong to one or more Transport Zones. If not set,
> the Chassis will be considered part of a default group; this feature
> is backward compatible and did not require any changes to the database
> schemas.
>
> Configuring Transport Zones is done by creating a key called
> "ovn-transport-zones" in the external_ids of the Open_vSwitch table of the
> local OVS instance. The value is a string with the name of the Transport
> Zone that this instance is part of. Multiple TZs may be specified with
> a comma-separated list. For example:
>
> $ sudo ovs-vsctl set open . external-ids:ovn-transport-zones=tz1
>
> or
>
> $ sudo ovs-vsctl set open . external-ids:ovn-transport-zones=tz1,tz2,tz3
>
> This configuration will be also exposed in the Chassis table of the OVN
> Southbound Database so that external systems can see what TZ(s) each
> Chassis are part of and make decisions based those values.
>
> The use for Transport Zones includes but are not limited to:
>
> * Edge computing: As a way to preventing edge sites from trying to create
>   tunnels with every node on every other edge site while still allowing
>   these sites to create tunnels with the central node.
>
> * Extra security layer: Where users wants to create "trust zones"
>   and prevent computes in a more secure zone to communicate with a less
>   secure zone.
>
> Reported-by: Daniel Alvarez Sanchez 
> Reported-at: 
> https://mail.openvswitch.org/pipermail/ovs-discuss/2019-February/048255.html
> Signed-off-by: Lucas Alvares Gomes 
> ---
> v2 -> v3
> * Enhanced the test to include two more Chassis and assert the case
>   where Chassis with no TZs set will have tunnels formed between them.
> * Rebased the patch on top of the latest master branch.
>
> v1 -> v2
>  * Rename the function check_chassis_tzones to chassis_tzones_overlap.
>  * Fix a memory leak in chassis_tzones_overlap.
>  * Pass the transport_zones to encaps_run() as a "const char *"
>instead of "struct sbrec_chassis". With this we can also avoid not
>running the function in case the Chassis entry is not yet created.
>
>  NEWS|   3 +
>  ovn/controller/chassis.c|   8 +-
>  ovn/controller/encaps.c |  53 +-
>  ovn/controller/encaps.h |   3 +-
>  ovn/controller/ovn-controller.8.xml |   9 ++
>  ovn/controller/ovn-controller.c |  14 ++-
>  tests/ovn.at| 151 
>  7 files changed, 236 insertions(+), 5 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 1e4744dbd..4adf49f57 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -24,6 +24,9 @@ Post-v2.11.0
> protocol extension.
> - OVN:
>   * Select IPAM mac_prefix in a random manner if not provided by the user
> + * Support for Transport Zones, a way to separate chassis into
> +   logical groups which results in tunnels only been formed between
> +   members of the same transport zone(s).
> - New QoS type "linux-netem" on Linux.
>
>  v2.11.0 - 19 Feb 2019
> diff --git a/ovn/controller/chassis.c b/ovn/controller/chassis.c
> index 3ea908d18..34c260410 100644
> --- a/ovn/controller/chassis.c
> +++ b/ovn/controller/chassis.c
> @@ -139,6 +139,8 @@ chassis_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
>  const char *datapath_type =
>  br_int && br_int->datapath_type ? br_int->datapath_type : "";
>  const char *cms_options = get_cms_options(>external_ids);
> +const char *transport_zones = smap_get_def(>external_ids,
> +   "ovn-transport-zones", "");
>
>  struct ds iface_types = DS_EMPTY_INITIALIZER;
>  ds_put_cstr(_types, "");
> @@ -167,18 +169,22 @@ chassis_run(struct ovsdb_idl_txn *ovnsb_idl_txn,
>  = smap_get_def(_rec->external_ids, "iface-types", "");
>  const char *chassis_cms_options
>  = get_cms_options(_rec->external_ids);
> +const char *chassis_transport_zones = smap_get_def(
> +_rec->external_ids, "ovn-transport-zones", "");
>
>  /* If any of the external-ids should change, update them. */
>  if (strcmp(bridge_mappings, chassis_bridge_mappings) ||
>

[ovs-dev] [PATCH] ovn: change load balancer references to weak in NB schema

[Daniel Alvarez]

When a load balancer is added to multiple logical switches
and routers it has be to be removed from all of them before
being able to delete due to the current 'strong' reference
in the NB schema.

By changing it to 'weak', users can simply remove the load
balancer without having to remove all the references manually.
In particular, this will make things easier for networking-ovn,
the OpenStack integration project as it'll save some
calculations upon load balancer deletion.

The update path has been successfully from the previous version
of the schema.

Signed-off-by: Daniel Alvarez 
---
 ovn/ovn-nb.ovsschema |  8 
 tests/ovn-nbctl.at   | 10 +-
 2 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/ovn/ovn-nb.ovsschema b/ovn/ovn-nb.ovsschema
index f3683df14..10a59649a 100644
--- a/ovn/ovn-nb.ovsschema
+++ b/ovn/ovn-nb.ovsschema
@@ -1,7 +1,7 @@
 {
 "name": "OVN_Northbound",
-"version": "5.14.0",
-"cksum": "3600467067 20513",
+"version": "5.14.1",
+"cksum": "3758097843 20509",
 "tables": {
 "NB_Global": {
 "columns": {
@@ -46,7 +46,7 @@
   "max": "unlimited"}},
 "load_balancer": {"type": {"key": {"type": "uuid",
   "refTable": "Load_Balancer",
-  "refType": "strong"},
+  "refType": "weak"},
"min": 0,
"max": "unlimited"}},
 "dns_records": {"type": {"key": {"type": "uuid",
@@ -250,7 +250,7 @@
  "max": "unlimited"}},
 "load_balancer": {"type": {"key": {"type": "uuid",
   "refTable": "Load_Balancer",
-  "refType": "strong"},
+  "refType": "weak"},
"min": 0,
"max": "unlimited"}},
 "options": {
diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at
index f55277cee..7a5903c3a 100644
--- a/tests/ovn-nbctl.at
+++ b/tests/ovn-nbctl.at
@@ -752,7 +752,15 @@ AT_CHECK([ovn-nbctl lr-lb-add lr0 lb0])
 AT_CHECK([ovn-nbctl lr-lb-add lr0 lb1])
 AT_CHECK([ovn-nbctl lr-lb-add lr0 lb3])
 AT_CHECK([ovn-nbctl lr-lb-del lr0])
-AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [])])
+AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [])
+
+dnl Remove load balancers after adding them to a logical router/switch.
+AT_CHECK([ovn-nbctl lr-lb-add lr0 lb0])
+AT_CHECK([ovn-nbctl ls-lb-add ls0 lb1])
+AT_CHECK([ovn-nbctl lb-del lb0])
+AT_CHECK([ovn-nbctl lb-del lb1])
+AT_CHECK([ovn-nbctl lr-lb-list lr0 | uuidfilt], [0], [])
+AT_CHECK([ovn-nbctl ls-lb-list ls0 | uuidfilt], [0], [])])
 
 dnl -
 
-- 
2.17.2 (Apple Git-113)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ofp-packet: Fix NXT_RESUME with geneve tunnel metadata

[Daniel Alvarez Sanchez]

We have hit this issue as well on 2.9, would it be possible to backport it?

On Mon, Oct 8, 2018 at 7:17 PM Ben Pfaff  wrote:
>
> On Fri, Oct 05, 2018 at 09:19:54AM -0700, Yi-Hung Wei wrote:
> > The patch address vswitchd crash when it receives NXT_RESUME with geneve
> > tunnel metadata.  The crash is due to segmentation fault with the
> > following stack trace, and it is observed only in kernel datapath.
> > A test is added to prevent regression.
> >
> > Thread 1 "ovs-vswitchd" received signal SIGSEGV, Segmentation fault.
> > 0  0x7fcffd0c5412 in tun_metadata_to_geneve__ 
> > (flow=flow@entry=0x7ffcb7106680, b=b@entry=0x7ffcb70eb5a8, 
> > crit_opt=crit_opt@entry=0x7ffcb70eb287)
> >at lib/tun-metadata.c:676
> > 1  0x7fcffd0c6858 in tun_metadata_to_geneve_nlattr_flow 
> > (b=0x7ffcb70eb5a8, flow=0x7ffcb7106638) at lib/tun-metadata.c:706
> > 2  tun_metadata_to_geneve_nlattr (tun=tun@entry=0x7ffcb7106638, 
> > flow=flow@entry=0x7ffcb7106638, key=key@entry=0x0, b=b@entry=0x7ffcb70eb5a8)
> >at lib/tun-metadata.c:810
> > 3  0x7fcffd048464 in tun_key_to_attr (a=a@entry=0x7ffcb70eb5a8, 
> > tun_key=tun_key@entry=0x7ffcb7106638, 
> > tun_flow_key=tun_flow_key@entry=0x7ffcb7106638,
> >key_buf=key_buf@entry=0x0, tnl_type=, tnl_type@entry=0x0) 
> > at lib/odp-util.c:2886
> > 4  0x7fcffd0551cf in odp_key_from_dp_packet 
> > (buf=buf@entry=0x7ffcb70eb5a8, packet=0x7ffcb7106590) at lib/odp-util.c:5909
> > 5  0x7fcffd0d7870 in dpif_netlink_encode_execute (buf=0x7ffcb70eb5a8, 
> > d_exec=0x7ffcb7106428, dp_ifindex=) at 
> > lib/dpif-netlink.c:1873
> > 6  dpif_netlink_operate__ (dpif=dpif@entry=0xe65e00, 
> > ops=ops@entry=0x7ffcb7106418, n_ops=n_ops@entry=1) at 
> > lib/dpif-netlink.c:1959
> > 7  0x7fcffd0d842e in dpif_netlink_operate_chunks (n_ops=1, 
> > ops=0x7ffcb7106418, dpif=) at lib/dpif-netlink.c:2258
> > 8  dpif_netlink_operate (dpif_=0xe65e00, ops=, 
> > n_ops=) at lib/dpif-netlink.c:2294
> > 9  0x7fcffd014680 in dpif_operate (dpif=, ops= > out>, ops@entry=0x7ffcb7106418, n_ops=n_ops@entry=1) at lib/dpif.c:1359
> > 10 0x7fcffd014c58 in dpif_execute (dpif=, 
> > execute=execute@entry=0x7ffcb71064e0) at lib/dpif.c:1324
> > 11 0x7fcffd40d3e6 in nxt_resume (ofproto_=0xe6af50, pin=0x7ffcb7107150) 
> > at ofproto/ofproto-dpif.c:4885
> > 12 0x7fcffd3f88c3 in handle_nxt_resume (ofconn=ofconn@entry=0xf8c8f0, 
> > oh=oh@entry=0xf7ebd0) at ofproto/ofproto.c:3612
> > 13 0x7fcffd404a3b in handle_openflow__ (msg=0xeac460, ofconn=0xf8c8f0) 
> > at ofproto/ofproto.c:8137
> > 14 handle_openflow (ofconn=0xf8c8f0, ofp_msg=0xeac460) at 
> > ofproto/ofproto.c:8258
> > 15 0x7fcffd3f4653 in ofconn_run (handle_openflow=0x7fcffd4046f0 
> > , ofconn=0xf8c8f0) at ofproto/connmgr.c:1432
> > 16 connmgr_run (mgr=0xe422f0, 
> > handle_openflow=handle_openflow@entry=0x7fcffd4046f0 ) at 
> > ofproto/connmgr.c:363
> > 17 0x7fcffd3fdc76 in ofproto_run (p=0xe6af50) at ofproto/ofproto.c:1821
> > 18 0x0040ca94 in bridge_run__ () at vswitchd/bridge.c:2939
> > 19 0x00411d44 in bridge_run () at vswitchd/bridge.c:2997
> > 20 0x004094fd in main (argc=12, argv=0x7ffcb71085b8) at 
> > vswitchd/ovs-vswitchd.c:119
> >
> > VMWare-BZ: #2210216
> > Signed-off-by: Yi-Hung Wei 
> > ---
> > Please backport it to 2.10.
>
> Applied to master and branch-2.10, thanks!
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] Un-revert Work around Python/C JSON unicode differences

[Daniel Alvarez Sanchez]

Thanks for this!

On Tue, Jan 15, 2019 at 8:48 PM Ben Pfaff  wrote:
>
> On Mon, Jan 14, 2019 at 08:15:36AM -0600, Terry Wilson wrote:
> > This fix was reverted because it depended on a small bit of code
> > in a patch that was reverted that changed some python/ovs testing
> > and build. The fix is still necessary.
> >
> > The OVS C-based JSON parser operates on bytes, so the parser_feed
> > function returns the number of bytes that are processed. The pure
> > Python JSON parser currently operates on unicode, so it expects
> > that Parser.feed() returns a number of characters. This difference
> > leads to parsing errors when unicode characters are passed to the
> > C JSON parser from Python.
> >
> > Signed-off-by: Terry Wilson 
>
> Thanks, applied to master.  Let me know if this should be backported.
It would be great to have it in 2.10 and 2.9 if possible :)
Thanks!
Daniel

> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovn: Fix indentation in TODO.

[Daniel Alvarez]

Thanks a lot folks!
How about getting it into 2.10 branch? Would that be ok?

> On 12 Dec 2018, at 21:08, Ben Pfaff  wrote:
> 
>> On Thu, Sep 20, 2018 at 04:44:11PM -0700, Ben Pfaff wrote:
>> Some items listed under ovsdb-server should have been top-level items.
>> 
>> Signed-off-by: Ben Pfaff 
> 
> I got tired of waiting for a review for an internal documentation
> update, so I applied this to master.
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2] ovn-controller: Inject GARPs to logical switch pipeline to update neighbors

[Daniel Alvarez]

Prior to this patch, GARPs announcing NAT addresses or new VIFs
were sent out to localnet ofport through an output action.
This can lead to problems since local datapaths won't get those
GARPs and ovn-controller won't update MAC_Binding entries (as
upstream switch will not send back the GARP to this port hence
other logical routers won't update their neighbours).

This patch is changing the behavior so that GARPs get injected
to OVN pipeline of the external switch. This way, they'll get
broadcasted to local pipelines and also sent out to the external
network through the localnet port.

Reported-at: 
https://mail.openvswitch.org/pipermail/ovs-discuss/2018-October/047604.html
Signed-off-by: Daniel Alvarez 
---

v1->v2
Fix VLAN tests to account for the GARPs received by the local pipeline.
Remove localnet_ofports parameter as it's not used anymore.

 ovn/controller/pinctrl.c |  80 +
 tests/ovn.at | 124 +--
 2 files changed, 135 insertions(+), 69 deletions(-)

diff --git a/ovn/controller/pinctrl.c b/ovn/controller/pinctrl.c
index 56539a891..3cd2ad718 100644
--- a/ovn/controller/pinctrl.c
+++ b/ovn/controller/pinctrl.c
@@ -2019,8 +2019,8 @@ struct garp_data {
 ovs_be32 ipv4;   /* Ipv4 address of port. */
 long long int announce_time; /* Next announcement in ms. */
 int backoff; /* Backoff for the next announcement. */
-ofp_port_t ofport;   /* ofport used to output this GARP. */
-int tag; /* VLAN tag of this GARP packet, or -1. */
+uint32_t dp_key; /* Datapath used to output this GARP. */
+uint32_t port_key;   /* Port to inject the GARP into. */
 };

 /* Contains GARPs to be sent. */
@@ -2043,37 +2043,24 @@ destroy_send_garps(void)
 }

 static void
-add_garp(const char *name, ofp_port_t ofport, int tag,
- const struct eth_addr ea, ovs_be32 ip)
+add_garp(const char *name, const struct eth_addr ea, ovs_be32 ip,
+ uint32_t dp_key, uint32_t port_key)
 {
 struct garp_data *garp = xmalloc(sizeof *garp);
 garp->ea = ea;
 garp->ipv4 = ip;
 garp->announce_time = time_msec() + 1000;
 garp->backoff = 1;
-garp->ofport = ofport;
-garp->tag = tag;
+garp->dp_key = dp_key;
+garp->port_key = port_key;
 shash_add(_garp_data, name, garp);
 }

 /* Add or update a vif for which GARPs need to be announced. */
 static void
 send_garp_update(const struct sbrec_port_binding *binding_rec,
- struct simap *localnet_ofports,
- const struct hmap *local_datapaths,
  struct shash *nat_addresses)
 {
-/* Find the localnet ofport to send this GARP. */
-struct local_datapath *ld
-= get_local_datapath(local_datapaths,
- binding_rec->datapath->tunnel_key);
-if (!ld || !ld->localnet_port) {
-return;
-}
-ofp_port_t ofport = u16_to_ofp(simap_get(localnet_ofports,
- ld->localnet_port->logical_port));
-int tag = ld->localnet_port->n_tag ? *ld->localnet_port->tag : -1;
-
 volatile struct garp_data *garp = NULL;
 /* Update GARP for NAT IP if it exists.  Consider port bindings with type
  * "l3gateway" for logical switch ports attached to gateway routers, and
@@ -2090,11 +2077,13 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 laddrs->ipv4_addrs[i].addr_s);
 garp = shash_find_data(_garp_data, name);
 if (garp) {
-garp->ofport = ofport;
-garp->tag = tag;
+garp->dp_key = binding_rec->datapath->tunnel_key;
+garp->port_key = binding_rec->tunnel_key;
 } else {
-add_garp(name, ofport, tag, laddrs->ea,
- laddrs->ipv4_addrs[i].addr);
+add_garp(name, laddrs->ea,
+ laddrs->ipv4_addrs[i].addr,
+ binding_rec->datapath->tunnel_key,
+ binding_rec->tunnel_key);
 }
 free(name);
 }
@@ -2107,7 +2096,8 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 /* Update GARP for vif if it exists. */
 garp = shash_find_data(_garp_data, binding_rec->logical_port);
 if (garp) {
-garp->ofport = ofport;
+garp->dp_key = binding_rec->datapath->tunnel_key;
+garp->port_key = binding_rec->tunnel_key;
 return;
 }

@@ -2120,8 +2110,9 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 continue;
 }

-add_garp(binding_rec->logical_port, ofport, tag,
- laddr

Re: [ovs-dev] [PATCH] ovn-controller: Inject GARPs to logical switch pipeline to update neighbors

[Daniel Alvarez Sanchez]

This patch is making test "ovn -- vlan traffic for external network
with distributed router gateway port" fail at [0].
The reason seems to be that now we're sending GARPs so n_packets is
not 0 anymore for the flow that the test is checking:

./ovn.at:8816: as hv1 ovs-ofctl dump-flows br-int table=65 | grep
"priority=100,reg15=0x1,metadata=0x2" \
| grep actions=clone | grep -v n_packets=0 | wc -l

cookie=0x0, duration=6.454s, table=65, n_packets=2, n_bytes=84,
idle_age=3, priority=100,reg15=0x1,metadata=0x2
actions=clone(ct_clear,load:0->NXM_NX_REG11[],load:0->NXM_NX_REG12[],load:0->NXM_NX_REG13[],load:0x2->NXM_NX_REG11[],load:0x7->NXM_NX_REG12[],load:0x1->OXM_OF_METADATA[],load:0x1->NXM_NX_REG14[],load:0->NXM_NX_REG10[],load:0->NXM_NX_REG15[],load:0->NXM_NX_REG0[],load:0->NXM_NX_REG1[],load:0->NXM_NX_REG2[],load:0->NXM_NX_REG3[],load:0->NXM_NX_REG4[],load:0->NXM_NX_REG5[],load:0->NXM_NX_REG6[],load:0->NXM_NX_REG7[],load:0->NXM_NX_REG8[],load:0->NXM_NX_REG9[],load:0->NXM_OF_IN_PORT[],resubmit(,8))

As you can see n_packets is 2 at this point. I'm waiting for inputs
here from authors of the patch [1] (Numan, Anil and Guru) or other
folks. This is the only test failing and if we would expect 2 packets
instead of comment out that particular line, the rest of the test
passes.

Thanks a lot!
Daniel

[0] https://github.com/openvswitch/ovs/blob/master/tests/ovn.at#L8816
[1] 
https://github.com/openvswitch/ovs/commit/85706c34d53d4810f54bec1de662392a3c06a996
On Mon, Dec 3, 2018 at 6:54 PM Daniel Alvarez  wrote:
>
> Prior to this patch, GARPs announcing NAT addresses or new VIFs
> were sent out to localnet ofport through an output action.
> This can lead to problems since local datapaths won't get those
> GARPs and ovn-controller won't update MAC_Binding entries (as
> upstream switch will not send back the GARP to this port hence
> other logical routers won't update their neighbours).
>
> This patch is changing the behavior so that GARPs get injected
> to OVN pipeline of the external switch. This way, they'll get
> broadcasted to local pipelines and also sent out to the external
> network through the localnet port.
>
> Reported-at: 
> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-October/047604.html
> Signed-off-by: Daniel Alvarez 
> ---
>  ovn/controller/pinctrl.c |  62 ++--
>  tests/ovn.at | 100 +++
>  2 files changed, 125 insertions(+), 37 deletions(-)
>
> diff --git a/ovn/controller/pinctrl.c b/ovn/controller/pinctrl.c
> index 56539a891..3e8af41cc 100644
> --- a/ovn/controller/pinctrl.c
> +++ b/ovn/controller/pinctrl.c
> @@ -2019,8 +2019,8 @@ struct garp_data {
>  ovs_be32 ipv4;   /* Ipv4 address of port. */
>  long long int announce_time; /* Next announcement in ms. */
>  int backoff; /* Backoff for the next announcement. */
> -ofp_port_t ofport;   /* ofport used to output this GARP. */
> -int tag; /* VLAN tag of this GARP packet, or -1. */
> +uint32_t dp_key; /* Datapath used to output this GARP. */
> +uint32_t port_key;   /* Port to inject the GARP into. */
>  };
>
>  /* Contains GARPs to be sent. */
> @@ -2043,37 +2043,24 @@ destroy_send_garps(void)
>  }
>
>  static void
> -add_garp(const char *name, ofp_port_t ofport, int tag,
> - const struct eth_addr ea, ovs_be32 ip)
> +add_garp(const char *name, const struct eth_addr ea, ovs_be32 ip,
> + uint32_t dp_key, uint32_t port_key)
>  {
>  struct garp_data *garp = xmalloc(sizeof *garp);
>  garp->ea = ea;
>  garp->ipv4 = ip;
>  garp->announce_time = time_msec() + 1000;
>  garp->backoff = 1;
> -garp->ofport = ofport;
> -garp->tag = tag;
> +garp->dp_key = dp_key;
> +garp->port_key = port_key;
>  shash_add(_garp_data, name, garp);
>  }
>
>  /* Add or update a vif for which GARPs need to be announced. */
>  static void
>  send_garp_update(const struct sbrec_port_binding *binding_rec,
> - struct simap *localnet_ofports,
> - const struct hmap *local_datapaths,
>   struct shash *nat_addresses)
>  {
> -/* Find the localnet ofport to send this GARP. */
> -struct local_datapath *ld
> -= get_local_datapath(local_datapaths,
> - binding_rec->datapath->tunnel_key);
> -if (!ld || !ld->localnet_port) {
> -return;
> -}
> -ofp_port_t ofport = u16_to_ofp(simap_get(localnet_ofports,
> - 
> ld->localnet_port->logical_port));
> -int tag = ld->localnet

[ovs-dev] [PATCH] ovn-controller: Inject GARPs to logical switch pipeline to update neighbors

[Daniel Alvarez]

Prior to this patch, GARPs announcing NAT addresses or new VIFs
were sent out to localnet ofport through an output action.
This can lead to problems since local datapaths won't get those
GARPs and ovn-controller won't update MAC_Binding entries (as
upstream switch will not send back the GARP to this port hence
other logical routers won't update their neighbours).

This patch is changing the behavior so that GARPs get injected
to OVN pipeline of the external switch. This way, they'll get
broadcasted to local pipelines and also sent out to the external
network through the localnet port.

Reported-at: 
https://mail.openvswitch.org/pipermail/ovs-discuss/2018-October/047604.html
Signed-off-by: Daniel Alvarez 
---
 ovn/controller/pinctrl.c |  62 ++--
 tests/ovn.at | 100 +++
 2 files changed, 125 insertions(+), 37 deletions(-)

diff --git a/ovn/controller/pinctrl.c b/ovn/controller/pinctrl.c
index 56539a891..3e8af41cc 100644
--- a/ovn/controller/pinctrl.c
+++ b/ovn/controller/pinctrl.c
@@ -2019,8 +2019,8 @@ struct garp_data {
 ovs_be32 ipv4;   /* Ipv4 address of port. */
 long long int announce_time; /* Next announcement in ms. */
 int backoff; /* Backoff for the next announcement. */
-ofp_port_t ofport;   /* ofport used to output this GARP. */
-int tag; /* VLAN tag of this GARP packet, or -1. */
+uint32_t dp_key; /* Datapath used to output this GARP. */
+uint32_t port_key;   /* Port to inject the GARP into. */
 };

 /* Contains GARPs to be sent. */
@@ -2043,37 +2043,24 @@ destroy_send_garps(void)
 }

 static void
-add_garp(const char *name, ofp_port_t ofport, int tag,
- const struct eth_addr ea, ovs_be32 ip)
+add_garp(const char *name, const struct eth_addr ea, ovs_be32 ip,
+ uint32_t dp_key, uint32_t port_key)
 {
 struct garp_data *garp = xmalloc(sizeof *garp);
 garp->ea = ea;
 garp->ipv4 = ip;
 garp->announce_time = time_msec() + 1000;
 garp->backoff = 1;
-garp->ofport = ofport;
-garp->tag = tag;
+garp->dp_key = dp_key;
+garp->port_key = port_key;
 shash_add(_garp_data, name, garp);
 }

 /* Add or update a vif for which GARPs need to be announced. */
 static void
 send_garp_update(const struct sbrec_port_binding *binding_rec,
- struct simap *localnet_ofports,
- const struct hmap *local_datapaths,
  struct shash *nat_addresses)
 {
-/* Find the localnet ofport to send this GARP. */
-struct local_datapath *ld
-= get_local_datapath(local_datapaths,
- binding_rec->datapath->tunnel_key);
-if (!ld || !ld->localnet_port) {
-return;
-}
-ofp_port_t ofport = u16_to_ofp(simap_get(localnet_ofports,
- ld->localnet_port->logical_port));
-int tag = ld->localnet_port->n_tag ? *ld->localnet_port->tag : -1;
-
 volatile struct garp_data *garp = NULL;
 /* Update GARP for NAT IP if it exists.  Consider port bindings with type
  * "l3gateway" for logical switch ports attached to gateway routers, and
@@ -2090,11 +2077,13 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 laddrs->ipv4_addrs[i].addr_s);
 garp = shash_find_data(_garp_data, name);
 if (garp) {
-garp->ofport = ofport;
-garp->tag = tag;
+garp->dp_key = binding_rec->datapath->tunnel_key;
+garp->port_key = binding_rec->tunnel_key;
 } else {
-add_garp(name, ofport, tag, laddrs->ea,
- laddrs->ipv4_addrs[i].addr);
+add_garp(name, laddrs->ea,
+ laddrs->ipv4_addrs[i].addr,
+ binding_rec->datapath->tunnel_key,
+ binding_rec->tunnel_key);
 }
 free(name);
 }
@@ -2107,7 +2096,8 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 /* Update GARP for vif if it exists. */
 garp = shash_find_data(_garp_data, binding_rec->logical_port);
 if (garp) {
-garp->ofport = ofport;
+garp->dp_key = binding_rec->datapath->tunnel_key;
+garp->port_key = binding_rec->tunnel_key;
 return;
 }

@@ -2120,8 +2110,9 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 continue;
 }

-add_garp(binding_rec->logical_port, ofport, tag,
- laddrs.ea, laddrs.ipv4_addrs[0].addr);
+add_garp(binding_rec->logical_port,
+ laddrs.ea, laddrs.ipv4_addrs[0].addr,
+ 

[ovs-dev] [RFC] ovn-controller: Inject GARPs to logical switch pipeline to update neighbours

[Daniel Alvarez]

Prior to this patch, GARPs announcing NAT addresses or new VIFs
were sent out to localnet ofport through an output action.
This can lead to problems since local datapaths won't get those
GARPs and ovn-controller won't update MAC_Binding entries (as
upstream switch will not send back the GARP to this port hence
other logical routers won't update their neighbours).

This patch is changing the behavior so that GARPs get injected
to OVN pipeline of the external switch. This way, they'll get
broadcasted to local pipelines and also sent out to the external
network through the localnet port.

Reported-at: 
https://mail.openvswitch.org/pipermail/ovs-discuss/2018-October/047604.html
Signed-off-by: Daniel Alvarez 
---
 ovn/controller/pinctrl.c | 61 +---
 1 file changed, 26 insertions(+), 35 deletions(-)

diff --git a/ovn/controller/pinctrl.c b/ovn/controller/pinctrl.c
index 56539a891..b3b887cd0 100644
--- a/ovn/controller/pinctrl.c
+++ b/ovn/controller/pinctrl.c
@@ -2021,6 +2021,8 @@ struct garp_data {
 int backoff; /* Backoff for the next announcement. */
 ofp_port_t ofport;   /* ofport used to output this GARP. */
 int tag; /* VLAN tag of this GARP packet, or -1. */
+uint32_t dp_key; /* Datapath used to output this GARP. */
+uint32_t port_key;   /* Port to inject the GARP into. */
 };

 /* Contains GARPs to be sent. */
@@ -2043,37 +2045,24 @@ destroy_send_garps(void)
 }

 static void
-add_garp(const char *name, ofp_port_t ofport, int tag,
- const struct eth_addr ea, ovs_be32 ip)
+add_garp(const char *name, const struct eth_addr ea, ovs_be32 ip,
+ uint32_t dp_key, uint32_t port_key)
 {
 struct garp_data *garp = xmalloc(sizeof *garp);
 garp->ea = ea;
 garp->ipv4 = ip;
 garp->announce_time = time_msec() + 1000;
 garp->backoff = 1;
-garp->ofport = ofport;
-garp->tag = tag;
+garp->dp_key = dp_key;
+garp->port_key = port_key;
 shash_add(_garp_data, name, garp);
 }

 /* Add or update a vif for which GARPs need to be announced. */
 static void
 send_garp_update(const struct sbrec_port_binding *binding_rec,
- struct simap *localnet_ofports,
- const struct hmap *local_datapaths,
  struct shash *nat_addresses)
 {
-/* Find the localnet ofport to send this GARP. */
-struct local_datapath *ld
-= get_local_datapath(local_datapaths,
- binding_rec->datapath->tunnel_key);
-if (!ld || !ld->localnet_port) {
-return;
-}
-ofp_port_t ofport = u16_to_ofp(simap_get(localnet_ofports,
- ld->localnet_port->logical_port));
-int tag = ld->localnet_port->n_tag ? *ld->localnet_port->tag : -1;
-
 volatile struct garp_data *garp = NULL;
 /* Update GARP for NAT IP if it exists.  Consider port bindings with type
  * "l3gateway" for logical switch ports attached to gateway routers, and
@@ -2090,11 +2079,13 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 laddrs->ipv4_addrs[i].addr_s);
 garp = shash_find_data(_garp_data, name);
 if (garp) {
-garp->ofport = ofport;
-garp->tag = tag;
+garp->dp_key = binding_rec->datapath->tunnel_key;
+garp->port_key = binding_rec->tunnel_key;
 } else {
-add_garp(name, ofport, tag, laddrs->ea,
- laddrs->ipv4_addrs[i].addr);
+add_garp(name, laddrs->ea,
+ laddrs->ipv4_addrs[i].addr,
+ binding_rec->datapath->tunnel_key,
+ binding_rec->tunnel_key);
 }
 free(name);
 }
@@ -2107,7 +2098,8 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 /* Update GARP for vif if it exists. */
 garp = shash_find_data(_garp_data, binding_rec->logical_port);
 if (garp) {
-garp->ofport = ofport;
+garp->dp_key = binding_rec->datapath->tunnel_key;
+garp->port_key = binding_rec->tunnel_key;
 return;
 }

@@ -2120,8 +2112,9 @@ send_garp_update(const struct sbrec_port_binding 
*binding_rec,
 continue;
 }

-add_garp(binding_rec->logical_port, ofport, tag,
- laddrs.ea, laddrs.ipv4_addrs[0].addr);
+add_garp(binding_rec->logical_port,
+ laddrs.ea, laddrs.ipv4_addrs[0].addr,
+ binding_rec->datapath->tunnel_key, binding_rec->tunnel_key);

 destroy_lport_addresses();
 break;
@@ -2150,16 +2143,15 @@ send_garp(struct garp_data

Re: [ovs-dev] [PATCH v3] netdev: Retry getting interfaces on inconsistent dumps from kernel

[Daniel Alvarez Sanchez]

I forgot about this, sorry. Just sent the patch with the comment.
Thanks a lot,
Daniel
On Sat, Aug 18, 2018 at 5:41 PM Ben Pfaff  wrote:
>
> On August 18, 2018 8:18:51 AM PDT, Daniel Alvarez Sanchez 
>  wrote:
>>
>> Thanks a lot Ben.
>> It was fixed in glibc 2.28. Shall I send a patch to add the comment in the 
>> code? I think it's a good idea so that we can remove the workaround 
>> eventually. Not sure about sending a patch for just a comment though :)
>>
>> On Wed, Aug 15, 2018 at 10:40 PM Ben Pfaff  wrote:
>>>
>>> On Mon, Aug 13, 2018 at 05:39:07PM -0700, Ben Pfaff wrote:
>>> > On Mon, Aug 13, 2018 at 02:07:45PM +0200, Daniel Alvarez wrote:
>>> > > This patch in glibc [0] is fixing a bug where we may be getting
>>> > > inconsistent dumps from the kernel when listing interfaces due to
>>> > > a race condition.
>>> > >
>>> > > This could happen if we try to retrieve them while interfaces are
>>> > > being added/removed from the system at the same time.
>>> > > For systems running against old glibc versions, this patch is retrying
>>> > > the operation up to 3 times and then proceeding by logging a
>>> > > warning.
>>> > >
>>> > > Note that 3 times should be enough to not delay the operation much
>>> > > and since it's unlikely that we hit the race condition 3 times in
>>> > > a row. Still, if this happened, this patch is not changing the
>>> > > current behavior.
>>> > >
>>> > > [0] 
>>> > > https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c1f86a33ca32e26a9d6e29fc961e5ecb5e2e5eb4
>>> > >
>>> > > Signed-off-by: Daniel Alvarez 
>>> > > Co-authored-by: Jiri Benc 
>>> >
>>> > Thanks for the patch.
>>> >
>>> > As a co-author, Jiri also needs to sign off.
>>> >
>>> > Acked-by: Ben Pfaff 
>>>
>>> I see that Jiri signed off earlier and it just didn't get propagated.
>>>
>>> Thanks, I applied this to master and branch-2.10.
>
>
> The comment would be useful, so if you have time for it please send a patch.
>
> Thanks,
>
> Ben.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] netdev: Add comment to allow removing a workaround in the future

[Daniel Alvarez]

This patch [0] in glibc fixes an issue which is right now workarounded
in OVS by [1]. I'm adding a comment to indicate that from glibc 2.28
and beyond, the workaround is not needed so that we can eventually
remove it.

[0] 
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c1f86a33ca32e26a9d6e29fc961e5ecb5e2e5eb4
[1] 
https://github.com/openvswitch/ovs/commit/3434d306866d825084d2d186d1f8dd98662ff650

Signed-off-by: Daniel Alvarez 
---
 lib/netdev.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/netdev.c b/lib/netdev.c
index 84874408a..45b50f26c 100644
--- a/lib/netdev.c
+++ b/lib/netdev.c
@@ -2063,7 +2063,8 @@ retry:
  * address addition which may cause one of the returned
  * ifa_name values to be NULL. In such case, we know that we've
  * got an inconsistent dump. Retry but beware of an endless
- * loop. */
+ * loop. From glibc 2.28 and beyond, this workaround is not
+ * needed and should be eventually removed. */
 freeifaddrs(if_addr_list);
 goto retry;
 } else {
--
2.17.2

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovn-ctl: Fix the wrong pidfile argument passed to ovsdb-servers

[Daniel Alvarez Sanchez]

Thanks Numan!
On Tue, Oct 9, 2018 at 9:17 AM  wrote:

> From: Numan Siddique 
>
> When OVN db servers are started usinb ovn-ctl, if the pid files
> (/var/run/openvswitch/ovnnb_db.pid for example) are already
> present, then ovn-ctl passes "--pidfile=123" if the pid file has
> '123' stored in it. Later on when OVN pacemaker RA script calls
> status_ovnnb/status_ovnsb() functions, these returns "not running".
>
> The shell function 'pidfile_is_running()' stores the contents of
> the pid file as  "pid=`cat "$pidfile"`". If the caller also
> uses the same variable "pid" to store the file name, it gets
> overriden.
>
> This patch fixes this issue by renaming the local variable "pid"
> in the "start_ovsdb__()" shell function to "db_file_name".
>
> Signed-off-by: Numan Siddique 
> ---
>  ovn/utilities/ovn-ctl | 8 
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/ovn/utilities/ovn-ctl b/ovn/utilities/ovn-ctl
> index 3ff0df68e..950467c4e 100755
> --- a/ovn/utilities/ovn-ctl
> +++ b/ovn/utilities/ovn-ctl
> @@ -95,7 +95,7 @@ promote_ovnsb() {
>
>  start_ovsdb__() {
>  local DB=$1 db=$2 schema_name=$3 table_name=$4
> -local pid
> +local db_pid_file
>  local cluster_local_addr
>  local cluster_local_port
>  local cluster_local_proto
> @@ -116,7 +116,7 @@ start_ovsdb__() {
>  local addr
>  local active_conf_file
>  local use_remote_in_db
> -eval pid=\$DB_${DB}_PID
> +eval db_pid_file=\$DB_${DB}_PID
>  eval cluster_local_addr=\$DB_${DB}_CLUSTER_LOCAL_ADDR
>  eval cluster_local_port=\$DB_${DB}_CLUSTER_LOCAL_PORT
>  eval cluster_local_proto=\$DB_${DB}_CLUSTER_LOCAL_PROTO
> @@ -139,7 +139,7 @@ start_ovsdb__() {
>  eval use_remote_in_db=\$DB_${DB}_USE_REMOTE_IN_DB
>
>  # Check and eventually start ovsdb-server for DB
> -if pidfile_is_running $pid; then
> +if pidfile_is_running $db_pid_file; then
>  return
>  fi
>
> @@ -169,7 +169,7 @@ $cluster_remote_port
>
>  set ovsdb-server
>  set "$@" $log --log-file=$logfile
> -set "$@" --remote=punix:$sock --pidfile=$pid
> +set "$@" --remote=punix:$sock --pidfile=$db_pid_file
>  set "$@" --unixctl=ovn${db}_db.ctl
>
>  [ "$OVS_USER" != "" ] && set "$@" --user "$OVS_USER"
> --
> 2.17.1
>
> Acked-by: Daniel Alvarez 

> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v3] netdev: Retry getting interfaces on inconsistent dumps from kernel

[Daniel Alvarez Sanchez]

Thanks a lot Ben.
It was fixed in glibc 2.28. Shall I send a patch to add the comment in the
code? I think it's a good idea so that we can remove the workaround
eventually. Not sure about sending a patch for just a comment though :)

On Wed, Aug 15, 2018 at 10:40 PM Ben Pfaff  wrote:

> On Mon, Aug 13, 2018 at 05:39:07PM -0700, Ben Pfaff wrote:
> > On Mon, Aug 13, 2018 at 02:07:45PM +0200, Daniel Alvarez wrote:
> > > This patch in glibc [0] is fixing a bug where we may be getting
> > > inconsistent dumps from the kernel when listing interfaces due to
> > > a race condition.
> > >
> > > This could happen if we try to retrieve them while interfaces are
> > > being added/removed from the system at the same time.
> > > For systems running against old glibc versions, this patch is retrying
> > > the operation up to 3 times and then proceeding by logging a
> > > warning.
> > >
> > > Note that 3 times should be enough to not delay the operation much
> > > and since it's unlikely that we hit the race condition 3 times in
> > > a row. Still, if this happened, this patch is not changing the
> > > current behavior.
> > >
> > > [0]
> https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c1f86a33ca32e26a9d6e29fc961e5ecb5e2e5eb4
> > >
> > > Signed-off-by: Daniel Alvarez 
> > > Co-authored-by: Jiri Benc 
> >
> > Thanks for the patch.
> >
> > As a co-author, Jiri also needs to sign off.
> >
> > Acked-by: Ben Pfaff 
>
> I see that Jiri signed off earlier and it just didn't get propagated.
>
> Thanks, I applied this to master and branch-2.10.
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v3] netdev: Retry getting interfaces on inconsistent dumps from kernel

[Daniel Alvarez]

This patch in glibc [0] is fixing a bug where we may be getting
inconsistent dumps from the kernel when listing interfaces due to
a race condition.

This could happen if we try to retrieve them while interfaces are
being added/removed from the system at the same time.
For systems running against old glibc versions, this patch is retrying
the operation up to 3 times and then proceeding by logging a
warning.

Note that 3 times should be enough to not delay the operation much
and since it's unlikely that we hit the race condition 3 times in
a row. Still, if this happened, this patch is not changing the
current behavior.

[0] 
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c1f86a33ca32e26a9d6e29fc961e5ecb5e2e5eb4

Signed-off-by: Daniel Alvarez 
Co-authored-by: Jiri Benc 
---
 lib/netdev.c | 17 +
 1 file changed, 17 insertions(+)

diff --git a/lib/netdev.c b/lib/netdev.c
index 82ffeb901..690d28409 100644
--- a/lib/netdev.c
+++ b/lib/netdev.c
@@ -2039,19 +2039,36 @@ netdev_get_addrs(const char dev[], struct in6_addr 
**paddr,
 struct in6_addr *addr_array, *mask_array;
 const struct ifaddrs *ifa;
 int cnt = 0, i = 0;
+int retries = 3;
 
 ovs_mutex_lock(_addr_list_lock);
 if (!if_addr_list) {
 int err;
 
+retry:
 err = getifaddrs(_addr_list);
 if (err) {
 ovs_mutex_unlock(_addr_list_lock);
 return -err;
 }
+retries--;
 }
 
 for (ifa = if_addr_list; ifa; ifa = ifa->ifa_next) {
+if (!ifa->ifa_name) {
+if (retries) {
+/* Older versions of glibc have a bug on race condition with
+ * address addition which may cause one of the returned
+ * ifa_name values to be NULL. In such case, we know that we've
+ * got an inconsistent dump. Retry but beware of an endless
+ * loop. */
+freeifaddrs(if_addr_list);
+goto retry;
+} else {
+VLOG_WARN("Proceeding with an inconsistent dump of "
+  "interfaces from the kernel. Some may be missing");
+}
+}
 if (ifa->ifa_addr && ifa->ifa_name && ifa->ifa_netmask) {
 int family;
 
-- 
2.15.2 (Apple Git-101.1)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] OVN/OVS split: OVN mailing list?

[Daniel Alvarez Sanchez]

+1 for the split of the ML

On Mon, Aug 13, 2018 at 11:17 AM Lucas Alvares Gomes 
wrote:

> Hi,
>
> > Before starting in-depth technical discussions on this list or the
> > ovs-dev list, I'm curious if people would be interested in splitting off
> > a separate OVN list for this and future OVN-related discussions? I can
> > see merits of keeping discussions on this list and of starting a new
> > one, so I'm interested in what others think about the matter.
> >
>
> As someone that relies on filters to get the OVN related
> changes/discussions from the current ML I can definitely see the
> benefits of creating a separated list for OVN. Also, since the code
> split is on it's way and OVN will become a project own its own this
> sounds like a natural first step to be taken towards that goal.
>
> Cheers,
> Lucas
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2] netdev: Retry getting interfaces on inconsistent dumps from kernel

[Daniel Alvarez]

This patch in glibc [0] is fixing a bug where we may be getting
inconsistent dumps from the kernel when listing interfaces due to
a race condition.

This could happen if we try to retrieve them while interfaces are
being added/removed from the system at the same time.
For systems running against old glibc versions, this patch is retrying
the operation up to 3 times and then proceeding by logging a
warning.

Note that 3 times should be enough to not delay the operation much
and since it's unlikely that we hit the race condition 3 times in
a row. Still, if this happened, this patch is not changing the
current behavior.

[0] 
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c1f86a33ca32e26a9d6e29fc961e5ecb5e2e5eb4

Signed-off-by: Daniel Alvarez 
Co-authored-by: Jiri Benc 
---
 lib/netdev.c | 16 
 1 file changed, 16 insertions(+)

Signed-off-by: Daniel Alvarez 
---
 lib/netdev.c | 17 +
 1 file changed, 17 insertions(+)

diff --git a/lib/netdev.c b/lib/netdev.c
index 82ffeb901..690d28409 100644
--- a/lib/netdev.c
+++ b/lib/netdev.c
@@ -2039,19 +2039,36 @@ netdev_get_addrs(const char dev[], struct in6_addr 
**paddr,
 struct in6_addr *addr_array, *mask_array;
 const struct ifaddrs *ifa;
 int cnt = 0, i = 0;
+int retries = 3;
 
 ovs_mutex_lock(_addr_list_lock);
 if (!if_addr_list) {
 int err;
 
+retry:
 err = getifaddrs(_addr_list);
 if (err) {
 ovs_mutex_unlock(_addr_list_lock);
 return -err;
 }
+retries--;
 }
 
 for (ifa = if_addr_list; ifa; ifa = ifa->ifa_next) {
+if (!ifa->ifa_name) {
+if (retries) {
+/* Older versions of glibc have a bug on race condition with
+ * address addition which may cause one of the returned
+ * ifa_name values to be NULL. In such case, we know that we've
+ * got an inconsistent dump. Retry but beware of an endless
+ * loop. */
+freeifaddrs(if_addr_list);
+goto retry;
+} else {
+VLOG_WARN("Proceeding with an inconsistent dump of "
+  "interfaces from the kernel. Some may be missing");
+}
+}
 if (ifa->ifa_addr && ifa->ifa_name && ifa->ifa_netmask) {
 int family;
 
-- 
2.15.2 (Apple Git-101.1)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] netdev: Retry getting interfaces on inconsistent dumps from kernel

[Daniel Alvarez]

This patch in glibc [0] is fixing a bug where we may be getting
inconsistent dumps from the kernel when listing interfaces due to
a race condition.

This could happen if we try to retrieve them while interfaces are
being added/removed from the system at the same time.
For systems running against old glibc versions, this patch is retrying
the operation up to 3 times and then proceeding by logging a
warning.

Note that 3 times should be enough to not delay the operation much
and since it's unlikely that we hit the race condition 3 times in
a row. Still, if this happened, this patch is not changing the
current behavior.

[0] 
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=c1f86a33ca32e26a9d6e29fc961e5ecb5e2e5eb4

Signed-off-by: Daniel Alvarez 
Signed-off-by: Jiri Benc 
---
 lib/netdev.c | 16 
 1 file changed, 16 insertions(+)

diff --git a/lib/netdev.c b/lib/netdev.c
index 82ffeb901..448d6e661 100644
--- a/lib/netdev.c
+++ b/lib/netdev.c
@@ -2039,19 +2039,35 @@ netdev_get_addrs(const char dev[], struct in6_addr 
**paddr,
 struct in6_addr *addr_array, *mask_array;
 const struct ifaddrs *ifa;
 int cnt = 0, i = 0;
+int retries = 3;
 
 ovs_mutex_lock(_addr_list_lock);
 if (!if_addr_list) {
 int err;
 
+retry:
 err = getifaddrs(_addr_list);
 if (err) {
 ovs_mutex_unlock(_addr_list_lock);
 return -err;
 }
+retries--;
 }
 
 for (ifa = if_addr_list; ifa; ifa = ifa->ifa_next) {
+if (!ifa->ifa_name) {
+if (retries) {
+/* Older versions of glibc have a bug on race condition with
+ * address addition which may cause one of the returned 
ifa_name
+ * values to be NULL. In such case, we know that we've got an
+ * inconsistent dump. Retry but beware of an endless loop. */
+freeifaddrs(if_addr_list);
+goto retry;
+} else {
+VLOG_WARN("Proceeding with an inconsistent dump of "
+  "interfaces from the kernel. Some may be missing");
+}
+}
 if (ifa->ifa_addr && ifa->ifa_name && ifa->ifa_netmask) {
 int family;
 
-- 
2.15.1 (Apple Git-101)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v3] ovn.at: Add stateful test for ACL on port groups.

[Daniel Alvarez Sanchez]

Thanks a lot Han!


On Tue, Jun 26, 2018 at 9:41 AM Jakub Sitnicki  wrote:

> On Mon, 25 Jun 2018 10:03:02 -0700
> Han Zhou  wrote:
>
> > A bug was reported on the feature of applying ACLs on port groups [1].
> > This bug was not detected by the original test case, because it didn't
> > test the return traffic and so didn't ensure the stateful feature is
> > working. The fix [2] causes the original test case fail, because
> > once the conntrack is enabled, the test packets are dropped because
> > the checksum in those packets are invalid and so marked with "invalid"
> > state by conntrack. To avoid the test case failure, the fix [2] changed
> > it to test stateless acl only, which leaves the scenario untested,
> > although it is fixed. This patch adds back the stateful ACL in the
> > test, and replaced the dummy/receive with inject-pkt to send the test
> > packets, so that checksums can be properly filled in, and it also
> > adds tests for the return traffic, which ensures the stateful is
> > working.
> >
> > [1]
> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-June/046927.html
> >
> > [2] https://patchwork.ozlabs.org/patch/931913/
> >
> > Signed-off-by: Han Zhou 
> > ---
> > Note: this patch depends on Daniel's patch [2] which is not merged yet.
> > v1->v2:
> > - Addressed Jacub's comments - simplified packet expr and removed
> >   debug information.
> > - Renamed test_ip to test_icmp.
> > v2->v3:
> > - Updated comments.
> >
> >  tests/ovn.at | 69
> ++--
> >  1 file changed, 48 insertions(+), 21 deletions(-)
>
> Acked-by: Jakub Sitnicki 
>
Acked-by: Daniel Alvarez 

> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2 3/3] OVN: add protocol unreachable support to OVN router ports

[Daniel Alvarez Sanchez]

Yes, let's hope we can get it in soon... expecting an v3 from Darrell
apparently.
Thanks!

On Fri, Jun 29, 2018 at 2:15 PM Lucas Alvares Gomes 
wrote:

>  Hi,
>
> > this should be the same issue reported by Darrell ('ovn: Fix gateway
> > load balancing')
> > Regards,
> >
>
> Yeah that's right, I have applied Darrell's patch [0] and re-run the
> same test that myself and Daniel were debugging [1] and I can confirm
> that it works now.
>
> Thanks for the pointers!
>
> [0] http://patchwork.ozlabs.org/patch/935938/
> [1]
> https://github.com/openstack/neutron-tempest-plugin/blob/8bc66e3205b834e17e9a9e6b72b6203a7a02cada/neutron_tempest_plugin/scenario/test_floatingip.py#L180-L200
>
> Cheers,
> Lucas
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2 3/3] OVN: add protocol unreachable support to OVN router ports

[Daniel Alvarez Sanchez]

Hi all,

We are hitting issues with this patch on OpenStack CI in this particular
test [0].
The scenario is one VM trying to ping the router interface; replies are
sent by
the router interface but never reached the instance back as the last NAT
action
doesn't happen. Stopping northd and deleting manually the Logical Flows
inserted
by this patch, fixed the issue.

VM with IP 10.0.0.9 trying to ping 172.24.4.1. Router interface for this
network is
172.24.4.12 and router interface for the LS of the VM is 10.0.0.1.
Deleting the following lflows solved the issue:

_uuid   : 82dd40a7-1ff9-4676-b450-7b5dfdb2fb3a
actions : "icmp4 {eth.dst <-> eth.src; ip4.dst <-> ip4.src;
ip.ttl = 255; icmp4.type = 3; icmp4.code = 2; next; };"
external_ids: {source="ovn-northd.c:5185",
stage-name=lr_in_ip_input}
logical_datapath: 3b6be958-290c-4479-9864-007dd89bb056
match   : "ip4 && ip4.dst == 10.0.0.1 && !ip.later_frag"
pipeline: ingress
priority: 70
table_id: 1
hash: 0

_uuid   : c276a38d-f49c-4e8b-a005-25e92a17ba03
actions : "icmp4 {eth.dst <-> eth.src; ip4.dst <-> ip4.src;
ip.ttl = 255; icmp4.type = 3; icmp4.code = 2; next; };"
external_ids: {source="ovn-northd.c:5185",
stage-name=lr_in_ip_input}
logical_datapath: 4908c283-fffd-42bc-80ef-86ed63e50b53
match   : "ip4 && ip4.dst == 10.0.0.1 && !ip.later_frag"
pipeline: ingress
priority: 70
table_id: 1
hash: 0

_uuid   : 07df1365-c4fd-4301-8e0f-96bf07a39f21
actions : "icmp4 {eth.dst <-> eth.src; ip4.dst <-> ip4.src;
ip.ttl = 255; icmp4.type = 3; icmp4.code = 2; next; };"
external_ids: {source="ovn-northd.c:5185",
stage-name=lr_in_ip_input}
logical_datapath: 3b6be958-290c-4479-9864-007dd89bb056
match   : "ip4 && ip4.dst == 172.24.4.12 && !ip.later_frag"
pipeline: ingress
priority: 70
table_id: 1
hash: 0


[0]
https://github.com/openstack/neutron-tempest-plugin/blob/8bc66e3205b834e17e9a9e6b72b6203a7a02cada/neutron_tempest_plugin/scenario/test_floatingip.py#L180


On Mon, Jun 18, 2018 at 1:58 PM Lorenzo Bianconi <
lorenzo.bianc...@redhat.com> wrote:

> Add priority-70 flows to generate ICMP protocol unreachable messages
> in reply to packets directed to the router's IP address on IP protocols
> other than UDP, TCP, and ICMP
>
> Signed-off-by: Lorenzo Bianconi 
> ---
>  ovn/northd/ovn-northd.8.xml |  4 
>  ovn/northd/ovn-northd.c | 14 ++
>  tests/ovn.at|  1 +
>  3 files changed, 15 insertions(+), 4 deletions(-)
>
> diff --git a/ovn/northd/ovn-northd.8.xml b/ovn/northd/ovn-northd.8.xml
> index 18a481b3d..cfd35115e 100644
> --- a/ovn/northd/ovn-northd.8.xml
> +++ b/ovn/northd/ovn-northd.8.xml
> @@ -1342,10 +1342,6 @@ nd_na {
>  
>These flows should not match IP fragments with nonzero offset.
>  
> -
> -
> -  Details TBD.  Not yet implemented.
> -
>
>
>
> diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
> index 27d7aab06..b83f5 100644
> --- a/ovn/northd/ovn-northd.c
> +++ b/ovn/northd/ovn-northd.c
> @@ -5175,6 +5175,20 @@ build_lrouter_flows(struct hmap *datapaths, struct
> hmap *ports,
>  "next; };";
>  ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 80,
>ds_cstr(), action);
> +
> +ds_clear();
> +ds_put_format(,
> +  "ip4 && ip4.dst == %s && !ip.later_frag",
> +  op->lrp_networks.ipv4_addrs[i].addr_s);
> +action = "icmp4 {"
> +"eth.dst <-> eth.src; "
> +"ip4.dst <-> ip4.src; "
> +"ip.ttl = 255; "
> +"icmp4.type = 3; "
> +"icmp4.code = 2; "
> +"next; };";
> +ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 70,
> +  ds_cstr(), action);
>  }
>
>  ds_clear();
> diff --git a/tests/ovn.at b/tests/ovn.at
> index 4648a303c..6553d17c6 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -10444,6 +10444,7 @@ OVN_POPULATE_ARP
>  ovn-nbctl --wait=hv sync
>
>  test_ip_packet 1 1 0001 ff01 $(ip_to_hex 192 168 1 1)
> $(ip_to_hex 192 168 1 254) 11  7dae fcfc 0303
> +test_ip_packet 1 1 0001 ff01 $(ip_to_hex 192 168 1 1)
> $(ip_to_hex 192 168 1 254) 84  7dae fcfd 0302
>  OVN_CHECK_PACKETS([hv1/vif1-tx.pcap], [vif1.expected])
>
>  test_tcp_syn_packet 2 2 0002 ff02 $(ip_to_hex 192 168 2
> 1) $(ip_to_hex 192 168 2 254)  8b40 3039  7bae 4486
> --
> 2.17.1
>
> ___
> dev mailing list
> d...@openvswitch.org
> 

Re: [ovs-dev] [PATCH] ovn-northd: Apply pre ACLs when using Port Groups

[Daniel Alvarez Sanchez]

Thanks a lot Han for the review. Just sent the v2 with the test fixed.
I'll leave the hash index for a follow up as I'm short in time but if
you want to edit my patch feel free to do it or send another one.

Thanks again for the Port Groups implementation! :)
Cheers,
Daniel

On Wed, Jun 20, 2018 at 4:15 AM, Han Zhou  wrote:

>
>
> On Tue, Jun 19, 2018 at 5:49 PM, Ben Pfaff  wrote:
> >
> > On Tue, Jun 19, 2018 at 05:27:20PM -0700, Han Zhou wrote:
> > > All looks good to me except that the test case "ovn -- ACLs on Port
> Groups"
> > > is broken with this change. I think it is because conntrack is not
> > > supported in the dummy datapath and so the stateful ACL would not work
> in
> > > the test suites, and it was passing just because of this bug. So, to
> fix
> > > the test case, you need below change:
> >
> > I would have guessed that conntrack works OK in the dummy datapath
> > because dpif-netdev supports conntrack.
>
> Ah, I admit that I am ignorant on this. I need to study more on it to
> understand why this test case doesn't work. Is there any
> tool/documentation/example on how to debug the dummy datapath conntrack,
> such as dumping the conntrack table entries?
>
> Thanks,
> Han
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2] ovn-northd: Apply pre ACLs when using Port Groups

[Daniel Alvarez]

When using Port Groups, the pre ACLs were not applied so the
conntrack action was not performed. This patch takes Port Groups
into account when processing the pre ACLs.

As a follow up, we could enhance this patch by creating an index
from lswitch to port groups.

Signed-off-by: Daniel Alvarez 
---
 ovn/northd/ovn-northd.c | 100 +++-
 tests/ovn.at|   2 +-
 2 files changed, 57 insertions(+), 45 deletions(-)

diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
index 72fe4e795..818ac59fa 100644
--- a/ovn/northd/ovn-northd.c
+++ b/ovn/northd/ovn-northd.c
@@ -2835,8 +2835,47 @@ build_dhcpv6_action(struct ovn_port *op, struct in6_addr 
*offer_ip,
 return true;
 }
 
+struct ovn_port_group_ls {
+struct hmap_node key_node;  /* Index on 'key'. */
+struct uuid key;/* nb_ls->header_.uuid. */
+const struct nbrec_logical_switch *nb_ls;
+};
+
+struct ovn_port_group {
+struct hmap_node key_node;  /* Index on 'key'. */
+struct uuid key;/* nb_pg->header_.uuid. */
+const struct nbrec_port_group *nb_pg;
+struct hmap nb_lswitches;   /* NB lswitches related to the port group */
+size_t n_acls;  /* Number of ACLs applied to the port group */
+struct nbrec_acl **acls;/* ACLs applied to the port group */
+};
+
+static void
+ovn_port_group_ls_add(struct ovn_port_group *pg,
+  const struct nbrec_logical_switch *nb_ls)
+{
+struct ovn_port_group_ls *pg_ls = xzalloc(sizeof *pg_ls);
+pg_ls->key = nb_ls->header_.uuid;
+pg_ls->nb_ls = nb_ls;
+hmap_insert(>nb_lswitches, _ls->key_node, uuid_hash(_ls->key));
+}
+
+static struct ovn_port_group_ls *
+ovn_port_group_ls_find(struct ovn_port_group *pg, const struct uuid *ls_uuid)
+{
+struct ovn_port_group_ls *pg_ls;
+
+HMAP_FOR_EACH_WITH_HASH (pg_ls, key_node, uuid_hash(ls_uuid),
+ >nb_lswitches) {
+if (uuid_equals(ls_uuid, _ls->key)) {
+return pg_ls;
+}
+}
+return NULL;
+}
+
 static bool
-has_stateful_acl(struct ovn_datapath *od)
+has_stateful_acl(struct ovn_datapath *od, struct hmap *port_groups)
 {
 for (size_t i = 0; i < od->nbs->n_acls; i++) {
 struct nbrec_acl *acl = od->nbs->acls[i];
@@ -2845,13 +2884,25 @@ has_stateful_acl(struct ovn_datapath *od)
 }
 }
 
+struct ovn_port_group *pg;
+HMAP_FOR_EACH (pg, key_node, port_groups) {
+if (ovn_port_group_ls_find(pg, >nbs->header_.uuid)) {
+for (size_t i = 0; i < pg->n_acls; i++) {
+struct nbrec_acl *acl = pg->acls[i];
+if (!strcmp(acl->action, "allow-related")) {
+return true;
+}
+}
+}
+}
 return false;
 }
 
 static void
-build_pre_acls(struct ovn_datapath *od, struct hmap *lflows)
+build_pre_acls(struct ovn_datapath *od, struct hmap *lflows,
+   struct hmap *port_groups)
 {
-bool has_stateful = has_stateful_acl(od);
+bool has_stateful = has_stateful_acl(od, port_groups);
 
 /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are
  * allowed by default. */
@@ -3309,21 +3360,6 @@ consider_acl(struct hmap *lflows, struct ovn_datapath 
*od,
 free(stage_hint);
 }
 
-struct ovn_port_group_ls {
-struct hmap_node key_node;  /* Index on 'key'. */
-struct uuid key;/* nb_ls->header_.uuid. */
-const struct nbrec_logical_switch *nb_ls;
-};
-
-struct ovn_port_group {
-struct hmap_node key_node;  /* Index on 'key'. */
-struct uuid key;/* nb_pg->header_.uuid. */
-const struct nbrec_port_group *nb_pg;
-struct hmap nb_lswitches;   /* NB lswitches related to the port group */
-size_t n_acls;  /* Number of ACLs applied to the port group */
-struct nbrec_acl **acls;/* ACLs applied to the port group */
-};
-
 static struct ovn_port_group *
 ovn_port_group_create(struct hmap *pgs,
   const struct nbrec_port_group *nb_pg)
@@ -3338,30 +3374,6 @@ ovn_port_group_create(struct hmap *pgs,
 return pg;
 }
 
-static void
-ovn_port_group_ls_add(struct ovn_port_group *pg,
-  const struct nbrec_logical_switch *nb_ls)
-{
-struct ovn_port_group_ls *pg_ls = xzalloc(sizeof *pg_ls);
-pg_ls->key = nb_ls->header_.uuid;
-pg_ls->nb_ls = nb_ls;
-hmap_insert(>nb_lswitches, _ls->key_node, uuid_hash(_ls->key));
-}
-
-static struct ovn_port_group_ls *
-ovn_port_group_ls_find(struct ovn_port_group *pg, const struct uuid *ls_uuid)
-{
-struct ovn_port_group_ls *pg_ls;
-
-HMAP_FOR_EACH_WITH_HASH (pg_ls, key_node, uuid_hash(ls_uuid),
- >nb_lswitches) {
-if (uuid_equals(ls_uuid, _ls->key)) {
-return pg_ls;
-}
-}
-return NULL;
-}
-
 static void
 ovn_port_grou

[ovs-dev] [PATCH] ovn-northd: Apply pre ACLs when using Port Groups

[Daniel Alvarez]

When using Port Groups, the pre ACLs were not applied so the
conntrack action was not performed. This patch takes Port Groups
into account when processing the pre ACLs.

Signed-off-by: Daniel Alvarez 
---
 ovn/northd/ovn-northd.c | 100 +++-
 1 file changed, 56 insertions(+), 44 deletions(-)

diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
index 72fe4e795..818ac59fa 100644
--- a/ovn/northd/ovn-northd.c
+++ b/ovn/northd/ovn-northd.c
@@ -2835,8 +2835,47 @@ build_dhcpv6_action(struct ovn_port *op, struct in6_addr 
*offer_ip,
 return true;
 }
 
+struct ovn_port_group_ls {
+struct hmap_node key_node;  /* Index on 'key'. */
+struct uuid key;/* nb_ls->header_.uuid. */
+const struct nbrec_logical_switch *nb_ls;
+};
+
+struct ovn_port_group {
+struct hmap_node key_node;  /* Index on 'key'. */
+struct uuid key;/* nb_pg->header_.uuid. */
+const struct nbrec_port_group *nb_pg;
+struct hmap nb_lswitches;   /* NB lswitches related to the port group */
+size_t n_acls;  /* Number of ACLs applied to the port group */
+struct nbrec_acl **acls;/* ACLs applied to the port group */
+};
+
+static void
+ovn_port_group_ls_add(struct ovn_port_group *pg,
+  const struct nbrec_logical_switch *nb_ls)
+{
+struct ovn_port_group_ls *pg_ls = xzalloc(sizeof *pg_ls);
+pg_ls->key = nb_ls->header_.uuid;
+pg_ls->nb_ls = nb_ls;
+hmap_insert(>nb_lswitches, _ls->key_node, uuid_hash(_ls->key));
+}
+
+static struct ovn_port_group_ls *
+ovn_port_group_ls_find(struct ovn_port_group *pg, const struct uuid *ls_uuid)
+{
+struct ovn_port_group_ls *pg_ls;
+
+HMAP_FOR_EACH_WITH_HASH (pg_ls, key_node, uuid_hash(ls_uuid),
+ >nb_lswitches) {
+if (uuid_equals(ls_uuid, _ls->key)) {
+return pg_ls;
+}
+}
+return NULL;
+}
+
 static bool
-has_stateful_acl(struct ovn_datapath *od)
+has_stateful_acl(struct ovn_datapath *od, struct hmap *port_groups)
 {
 for (size_t i = 0; i < od->nbs->n_acls; i++) {
 struct nbrec_acl *acl = od->nbs->acls[i];
@@ -2845,13 +2884,25 @@ has_stateful_acl(struct ovn_datapath *od)
 }
 }
 
+struct ovn_port_group *pg;
+HMAP_FOR_EACH (pg, key_node, port_groups) {
+if (ovn_port_group_ls_find(pg, >nbs->header_.uuid)) {
+for (size_t i = 0; i < pg->n_acls; i++) {
+struct nbrec_acl *acl = pg->acls[i];
+if (!strcmp(acl->action, "allow-related")) {
+return true;
+}
+}
+}
+}
 return false;
 }
 
 static void
-build_pre_acls(struct ovn_datapath *od, struct hmap *lflows)
+build_pre_acls(struct ovn_datapath *od, struct hmap *lflows,
+   struct hmap *port_groups)
 {
-bool has_stateful = has_stateful_acl(od);
+bool has_stateful = has_stateful_acl(od, port_groups);
 
 /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are
  * allowed by default. */
@@ -3309,21 +3360,6 @@ consider_acl(struct hmap *lflows, struct ovn_datapath 
*od,
 free(stage_hint);
 }
 
-struct ovn_port_group_ls {
-struct hmap_node key_node;  /* Index on 'key'. */
-struct uuid key;/* nb_ls->header_.uuid. */
-const struct nbrec_logical_switch *nb_ls;
-};
-
-struct ovn_port_group {
-struct hmap_node key_node;  /* Index on 'key'. */
-struct uuid key;/* nb_pg->header_.uuid. */
-const struct nbrec_port_group *nb_pg;
-struct hmap nb_lswitches;   /* NB lswitches related to the port group */
-size_t n_acls;  /* Number of ACLs applied to the port group */
-struct nbrec_acl **acls;/* ACLs applied to the port group */
-};
-
 static struct ovn_port_group *
 ovn_port_group_create(struct hmap *pgs,
   const struct nbrec_port_group *nb_pg)
@@ -3338,30 +3374,6 @@ ovn_port_group_create(struct hmap *pgs,
 return pg;
 }
 
-static void
-ovn_port_group_ls_add(struct ovn_port_group *pg,
-  const struct nbrec_logical_switch *nb_ls)
-{
-struct ovn_port_group_ls *pg_ls = xzalloc(sizeof *pg_ls);
-pg_ls->key = nb_ls->header_.uuid;
-pg_ls->nb_ls = nb_ls;
-hmap_insert(>nb_lswitches, _ls->key_node, uuid_hash(_ls->key));
-}
-
-static struct ovn_port_group_ls *
-ovn_port_group_ls_find(struct ovn_port_group *pg, const struct uuid *ls_uuid)
-{
-struct ovn_port_group_ls *pg_ls;
-
-HMAP_FOR_EACH_WITH_HASH (pg_ls, key_node, uuid_hash(ls_uuid),
- >nb_lswitches) {
-if (uuid_equals(ls_uuid, _ls->key)) {
-return pg_ls;
-}
-}
-return NULL;
-}
-
 static void
 ovn_port_group_destroy(struct hmap *pgs, struct ovn_port_group *pg)
 {
@@ -3416,7 +3428,7 @@ static void
 build_acls(struct ovn_datapath *od, str

Re: [ovs-dev] [PATCH] OVN python IDL: avoid useless JSON conversion to enhance performance

[Daniel Alvarez Sanchez]

Hi,
Bringing this up again.
Can we please get this backported into 2.9 branch? The enhancement is very
noticeable.
Thanks a lot!
Daniel

On Tue, Mar 6, 2018 at 5:09 PM, Lucas Alvares Gomes 
wrote:

> Hi,
>
> Excellent investigative work here Daniel, thanks for that!
>
> On Wed, Feb 28, 2018 at 9:37 AM, Miguel Angel Ajo Pelayo
>  wrote:
> > And if we can get backports down the lane, that'd be great :)
> >
>
> +1 for backporting it. The changes seems straight forward to do so.
>
> > On Wed, Feb 28, 2018 at 9:37 AM Miguel Angel Ajo Pelayo <
> majop...@redhat.com>
> > wrote:
> >
> >> Acked-by: Miguel Angel Ajo 
> >>
> >> On Wed, Feb 28, 2018 at 9:13 AM Daniel Alvarez Sanchez <
> >> dalva...@redhat.com> wrote:
> >>
> >>> Thanks Terry and Han for the reviews!
> >>> I removed the 'OVN' keyword from the title and submitted a v2:
> >>> [PATCH v2] python: avoid useless JSON conversion to enhance performance
> >>>
> >>> Thanks again.
> >>> Daniel
> >>>
> >>> On Wed, Feb 28, 2018 at 12:41 AM, Han Zhou  wrote:
> >>>
> >>> > Great catch!
> >>> > This patch is generic and not only for OVN, so I suggest to remove
> the
> >>> > "OVN" keyword in commit title.
> >>> >
> >>> > Acked-by: Han Zhou 
> >>> >
> >>> > On Tue, Feb 27, 2018 at 12:44 PM, Terry Wilson 
> >>> wrote:
> >>> >
> >>> >> On Tue, Feb 27, 2018 at 9:25 AM, Daniel Alvarez <
> dalva...@redhat.com>
> >>> >> wrote:
> >>> >> > This patch removes a useless conversion to/from JSON in the
> >>> >> > processing of any 'modify' operations inside the process_update2
> >>> >> > method in Python IDL implementation.
> >>> >> >
> >>> >> > Previous code will make resources creation take longer as the
> number
> >>> >> > of elements in the row grows because of that JSON conversion. This
> >>> >> > patch eliminates it and now the time remains consant regardless
> >>> >> > of the database contents improving performance and scaling.
> >>> >> >
> >>> >> > Reported-by: Daniel Alvarez Sanchez 
> >>> >> > Reported-at: https://mail.openvswitch.org/p
> >>> >> ipermail/ovs-discuss/2018-February/046263.html
> >>> >> > Signed-off-by: Daniel Alvarez 
> >>> >> > ---
> >>> >> >  python/ovs/db/idl.py | 12 +---
> >>> >> >  1 file changed, 5 insertions(+), 7 deletions(-)
> >>> >> >
> >>> >> > diff --git a/python/ovs/db/idl.py b/python/ovs/db/idl.py
> >>> >> > index 60548bcf5..5a4d129c0 100644
> >>> >> > --- a/python/ovs/db/idl.py
> >>> >> > +++ b/python/ovs/db/idl.py
> >>> >> > @@ -518,10 +518,8 @@ class Idl(object):
> >>> >> >  if not row:
> >>> >> >  raise error.Error('Modify non-existing row')
> >>> >> >
> >>> >> > -old_row_diff_json = self.__apply_diff(table, row,
> >>> >> > -
> >>> row_update['modify'])
> >>> >> > -self.notify(ROW_UPDATE, row,
> >>> >> > -Row.from_json(self, table, uuid,
> >>> >> old_row_diff_json))
> >>> >> > +old_row = self.__apply_diff(table, row,
> >>> >> row_update['modify'])
> >>> >> > +self.notify(ROW_UPDATE, row, Row(self, table, uuid,
> >>> >> old_row))
> >>> >> >  changed = True
> >>> >> >  else:
> >>> >> >  raise error.Error(' unknown operation',
> >>> >> > @@ -584,7 +582,7 @@ class Idl(object):
> >>> >> >  row_update[column.name] =
> >>> >> self.__column_name(column)
> >>> >> >
> >>> >> >  def __apply_diff(self, table, row, row_diff):
> >>> >> > -old_row_diff_json = {}
> >>> >> > +old_row = {}
> >>> >> >  for column_name, datum_diff_json in
> six.iteritems(row_diff):
> >>> >> >

Re: [ovs-dev] [PATCH] datapath: RHEL 7.5 ndo_change_mtu backward compatibility

[Daniel Alvarez Sanchez]

Thanks Lucas, this makes sense. There is something that this patch is
fixing and I'm not sure why.
Maybe someone can shed some light:

Using datapath from OVS master, and a setup where we have a physical
interface connected to
an OVS bridge (br-ex) connected to another OVS bridge (br-int) through a
patch port, there's a lot
of retransmissions of TCP packets when connecting from the host to a VM
connected to br-int.
The retransmissions seem to be due to a wrong checksum from the VM to the
host and only after
a few attempts, the checksum is correct and the host sends the ACK back.
The packets I am
sending using netcat are very small so there shouldn't be a problem with
the MTU. However, could
it be a side effect of this patch that the checksum gets now correctly
received at the host?

As a side not: if instead from connecting to the VM from the host I do it
from a namespace where
I have an OVS internal port connected to br-ex, then I don't see the
checksum problems.

Acked-by: Daniel Alvarez <dalva...@redhat.com>
Tested-by: Daniel Alvarez <dalva...@redhat.com>

On Thu, May 17, 2018 at 1:27 PM, <lucasago...@gmail.com> wrote:

> From: Lucas Alvares Gomes <lucasago...@gmail.com>
>
> The commit [0] partially fixed the problem but in RHEL 7.5 neither
> .{min, max}_mtu or 'ndo_change_mtu' were being set/implemented for
> vport-internal_dev.c.
>
> As pointed out by commit [0], the ndo_change_mtu function pointer has been
> moved from 'struct net_device_ops' to 'struct net_device_ops_extended'
> on RHEL 7.5.
>
> So this patch fixes the backport issue by setting the
> .extended.ndo_change_mtu when necessary.
>
> [0] 39ca338374abe367e28a2247bac9159695f19710
> ---
>  datapath/vport-internal_dev.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/datapath/vport-internal_dev.c b/datapath/vport-internal_dev.c
> index 3cb8d06b2..16f4aaeee 100644
> --- a/datapath/vport-internal_dev.c
> +++ b/datapath/vport-internal_dev.c
> @@ -88,7 +88,7 @@ static const struct ethtool_ops internal_dev_ethtool_ops
> = {
> .get_link   = ethtool_op_get_link,
>  };
>
> -#if!defined(HAVE_NET_DEVICE_WITH_MAX_MTU) &&
> !defined(HAVE_RHEL7_MAX_MTU)
> +#ifndef HAVE_NET_DEVICE_WITH_MAX_MTU
>  static int internal_dev_change_mtu(struct net_device *dev, int new_mtu)
>  {
> if (new_mtu < ETH_MIN_MTU) {
> @@ -155,6 +155,8 @@ static const struct net_device_ops
> internal_dev_netdev_ops = {
> .ndo_set_mac_address = eth_mac_addr,
>  #if!defined(HAVE_NET_DEVICE_WITH_MAX_MTU) &&
> !defined(HAVE_RHEL7_MAX_MTU)
> .ndo_change_mtu = internal_dev_change_mtu,
> +#elif  !defined(HAVE_NET_DEVICE_WITH_MAX_MTU) &&
> defined(HAVE_RHEL7_MAX_MTU)
> +   .extended.ndo_change_mtu = internal_dev_change_mtu,
>  #endif
> .ndo_get_stats64 = (void *)internal_get_stats,
>  };
> --
> 2.17.0
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] Fix datapath compilation on RHEL >= 7.5

[Daniel Alvarez]

Yes, let’s go with that one then.
Thanks a lot!
Daniel 

> On 10 May 2018, at 23:15, Gregory Rose <gvrose8...@gmail.com> wrote:
> 
>> On 5/10/2018 7:11 AM, Daniel Alvarez wrote:
>> On RHEL 7.5 we get compilation errors due to field ndo_change_mtu
>> missing. This patch checks the RHEL version and redefines it to
>> ndo_change_mtu_rh74.
>> 
>> Reported-by: Lucas Alvares <lucasago...@gmail.com>
>> Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
>> ---
>>  datapath/datapath.h | 5 +
>>  1 file changed, 5 insertions(+)
>> 
>> diff --git a/datapath/datapath.h b/datapath/datapath.h
>> index 93c9ed505..d418bf381 100644
>> --- a/datapath/datapath.h
>> +++ b/datapath/datapath.h
>> @@ -25,6 +25,7 @@
>>  #include 
>>  #include 
>>  #include 
>> +#include 
>>  #include 
>>  #include 
>>  @@ -36,6 +37,10 @@
>>  #define DP_MAX_PORTS   USHRT_MAX
>>  #define DP_VPORT_HASH_BUCKETS  1024
>>  +#if RHEL_RELEASE_CODE >= RHEL_RELEASE_VERSION(7,5)
>> +#define ndo_change_mtu ndo_change_mtu_rh74
>> +#endif
>> +
>>  /**
>>   * struct dp_stats_percpu - per-cpu packet processing statistics for a given
>>   * datapath.
> 
> There are dueling patches for this.  I think the one posted by Yi-hung is a 
> bit more complete because it doesn't just check version numbers.
> 
> Thanks,
> 
> - Greg
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovn: Add a new action 'nd_na_router' to handle NS requests for router IPs

[Daniel Alvarez Sanchez]

On Tue, May 8, 2018 at 3:26 PM, Mark Michelson  wrote:

> On 05/08/2018 05:36 AM, Numan Siddique wrote:
>
>>
>>
>> On Tue, May 8, 2018 at 1:20 PM, Miguel Angel Ajo Pelayo <
>> majop...@redhat.com > wrote:
>>
>> Thank you Numan!
>>
>> It took me a bit to find what the "RSO"  flag was, because they are
>> R, S and O, may be we can change that in the commit, or reference
>> the RFC/section  (RFC4861 page 23).
>>
>>
>> Ack. I will update the commit message and send v2.
>>
>>
>>   R  Router flag.  When set, the R-bit indicates that
>>   the sender is a router.  The R-bit is used by
>>   Neighbor Unreachability Detection to detect a
>>   router that changes to a host.
>>
>>S  Solicited flag.  When set, the S-bit indicates
>> that
>>   the advertisement was sent in response to a
>>   Neighbor Solicitation from the Destination
>> address.
>>   The S-bit is used as a reachability confirmation
>>   for Neighbor Unreachability Detection.  It MUST
>> NOT
>>   be set in multicast advertisements or in
>>   unsolicited unicast advertisements.
>>
>>O  Override flag.  When set, the O-bit indicates
>> that
>>   the advertisement should override an existing
>> cache
>>   entry and update the cached link-layer address.
>>   When it is not set the advertisement will not
>>   update a cached link-layer address though it
>> will
>>   update an existing Neighbor Cache entry for
>> which
>>   no link-layer address is known.  It SHOULD NOT
>> be
>>   set in solicited advertisements for anycast
>>   addresses and in solicited proxy advertisements.
>>   It SHOULD be set in other solicited
>> advertisements
>>   and in unsolicited advertisements.
>>
>>
>>
>>
>> And same question Mark did.
>>
>> Thanks for handling this, good work :)
>>
>> On Mon, May 7, 2018 at 9:16 PM Mark Michelson > > wrote:
>>
>> Hi Numan, thanks for the fix.
>>
>> Out of curiosity, why did you add a new action instead of adding
>> a new
>> logical field (something like nd.rso)?
>>
>>
>> The logical flow which uses nd_na looks like
>>
>>table=11(ls_in_arp_rsp  ), priority=50   , match=(nd_ns && ip6.dst
>> == {2002::f816:3eff:fefe:9e2e, ff02::1:fffe:9e2e} && nd.target ==
>> 2002::f816:3eff:fefe:9e2e), action=(nd_na { eth.src = fa:16:3e:fe:9e:2e;
>> ip6.src = 2002::f816:3eff:fefe:9e2e; nd.target = 2002::f816:3eff:fefe:9e2e;
>> nd.tll = fa:16:3e:fe:9e:2e; outport = inport; flags.loopback = 1; output;
>> };)
>>
>>
>> I suppose you are suggesting to have something like - " nd_na { ..,
>> nd.rso = router ..} ". All the inner logical fields are defined in expr
>> symtab table [1]. But we cannot add nd.rso there as there is no
>> corresponding ovs field to modify the ICMPv6 flags of a packet.
>>
>>  From the email discussion here [2] I suppose Zak is working to add this
>> feature. Once we have this support, we can make use of that in OVN. Right
>> now IPv6 feature is blocked in Openstack + OVN  because of this issue so we
>> need this fix.
>>
>
> Yes, this is what I had been thinking. It sounds like if the timing had
> been different, you could have waited for Zak's patch first. But given the
> circumstances, I suppose this will work.
>
+1 to this and then follow up :)

>
>
>> [1] - https://github.com/openvswitch/ovs/blob/master/ovn/lib/
>> logical-fields.c#L205 > h/ovs/blob/master/ovn/lib/logical-fields.c#L205>
>> [2] - https://mail.openvswitch.org/pipermail/ovs-discuss/2018-May/
>> 046662.html
>> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-April/046528.html
>>
>>
>> Thanks
>> Numan
>>
>>
>>
>> On 05/07/2018 02:36 PM, nusid...@redhat.com
>>  wrote:
>>  > From: Numan Siddique > >
>>  >
>>  > Presently when a VM's IPv6 stack sends a Neighbor
>> Solicitation request for its
>>  > router IP, (mostly when the ND cache entry for the router is
>> in STALE state)
>>  > ovn-controller responds with a Neighbor Adv packet (using the
>> action nd_na).
>>  > But it doesn't set 'ND_RSO_ROUTER' in the RSO flags. Because
>> of which, the
>>  > VM deletes the default route. The default route gets added
>> again when the next
>>  > RA is received (but would 

[ovs-dev] [PATCH] Fix datapath compilation on RHEL >= 7.5

[Daniel Alvarez]

On RHEL 7.5 we get compilation errors due to field ndo_change_mtu
missing. This patch checks the RHEL version and redefines it to
ndo_change_mtu_rh74.

Reported-by: Lucas Alvares <lucasago...@gmail.com>
Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---
 datapath/datapath.h | 5 +
 1 file changed, 5 insertions(+)

diff --git a/datapath/datapath.h b/datapath/datapath.h
index 93c9ed505..d418bf381 100644
--- a/datapath/datapath.h
+++ b/datapath/datapath.h
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -36,6 +37,10 @@
 #define DP_MAX_PORTS   USHRT_MAX
 #define DP_VPORT_HASH_BUCKETS  1024
 
+#if RHEL_RELEASE_CODE >= RHEL_RELEASE_VERSION(7,5)
+#define ndo_change_mtu ndo_change_mtu_rh74
+#endif
+
 /**
  * struct dp_stats_percpu - per-cpu packet processing statistics for a given
  * datapath.
-- 
2.15.1 (Apple Git-101)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

Wow great job Han!
I'll take a look ASAP, this is really useful indeed.

Thanks!
Daniel

On Sun, Apr 22, 2018 at 7:17 PM, Han Zhou <zhou...@gmail.com> wrote:

>
>
> On Fri, Mar 2, 2018 at 7:26 AM, Guru Shetty <g...@ovn.org> wrote:
> >
> >
> >
> > On 1 March 2018 at 15:43, Han Zhou <zhou...@gmail.com> wrote:
> >>
> >>
> >>
> >> On Thu, Mar 1, 2018 at 12:26 PM, Guru Shetty <g...@ovn.org> wrote:
> >> >
> >> >
> >> >
> >> > On 1 March 2018 at 12:21, Han Zhou <zhou...@gmail.com> wrote:
> >> >>
> >> >>
> >> >>
> >> >> On Thu, Mar 1, 2018 at 12:13 PM, Guru Shetty <g...@ovn.org> wrote:
> >> >> >
> >> >> >
> >> >> >
> >> >> > On 28 February 2018 at 19:37, Han Zhou <zhou...@gmail.com> wrote:
> >> >> >>
> >> >> >> This patch enables using port group names in ACL match conditions.
> >> >> >> Users can create a port group in northbound DB Port_Group table,
> >> >> >> and then use the name of the port group in ACL match conditions
> >> >> >> for "inport" or "outport". It can help reduce the number of ACLs
> >> >> >> for CMS clients such as OpenStack Neutron, for the use cases
> >> >> >> where a group of logical ports share same ACL rules except the
> >> >> >> "inport"/"outport" part. Without this patch, the clients have to
> >> >> >> create N (N = number of lports) ACLs, and this patch helps achieve
> >> >> >> the same goal with only one ACL. E.g.:
> >> >> >>
> >> >> >> to-lport 1000 "outport == @port_group1 && ip4.src == {IP1, IP2,
> ...}" allow-related
> >> >> >>
> >> >> >> There was a similar attempt by Zong Kai Li in 2016 [1]. This patch
> >> >> >> takes a slightly different approach by using weak refs instead of
> >> >> >> strings, which requires a new table instead of reusing the address
> >> >> >> set table. This way it will also benefit for a follow up patch
> that
> >> >> >> enables generating address sets automatically from port groups to
> >> >> >> avoid a lot a trouble from client perspective [2].
> >> >> >>
> >> >> >> [1] https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/
> 077118.html
> >> >> >> [2] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> February/046260.html
> >> >> >>
> >> >> >> Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
> >> >> >> Reported-at: https://mail.openvswitch.org/
> pipermail/ovs-discuss/2018-February/046166.html
> >> >> >> Signed-off-by: Han Zhou <hzh...@ebay.com>
> >> >> >
> >> >> >
> >> >> > Wouldn't it be more complete and useful if we add the acl to a
> port group too? And then internally, you decide which switches you want to
> add the ACL to.
> >> >> >
> >> >> > For e.g: ovn-nbctl --port-group add-acl port_group1 to-lport 1000
> "outport == @port_group1 && ip4.src == {IP1, IP2, ...}" allow-related
> >> >> >
> >> >> > This way, the client does not have to keep track of all the
> logical switches it needs to apply an ACL to. Thoughts?
> >> >> >
> >> >> Yes, this is a good idea. Since it is only about the ovn-nbctl tool
> improvement, it can be a follow up patch.
> >> >
> >> >
> >> > What if we have something like a acl column in the port_group table
> so that we just have one entry in OVN NB database? Logically, we apply a
> ACL to a security group instead of a  logical switch. And then ovn-northd
> decided which logical switches to apply it to. Would that make difference
> in performance? It does reduce the size of the NB database. Any drawbacks?
> >> >
> >> Ok, I thought you were talking about ovn-nbctl tool only. Now I get
> your point. I think it is a good idea, since it is a common work for
> different clients: figuring out which lswitches are needed for each group
> of ACLs.
> >
> > Right. And sending in multiple ACLs and deleting multiple ACLs instead
> of just one ACL with this approach.
> >
> >
> >>
> >> So it makes

Re: [ovs-dev] [PATCH v2 2/2] ovn: Support address sets generated from port groups

[Daniel Alvarez Sanchez]

Acked-by: Daniel Alvarez <dalva...@redhat.com>

Thanks Han! Everything LGTM and the tests pass okay against current master.

On Thu, Apr 5, 2018 at 2:51 AM, Han Zhou <zhou...@gmail.com> wrote:

> Address sets are automatically generated from corresponding port
> groups, and can be used directly in ACL match conditions.
>
> There are two address sets generated for each port group:
>
> _ip4
> _ip6
>
> For example, if port_group1 is created, we can directly use below
> match condition in ACL:
> "outport == @port_group1 && ip4.src == $port_group1_ip4"
>
> This will simplify OVN client implementation, and avoid some tricky
> problems such as race conditions when maintaining address set
> memberships as discussed in the link below.
>
> Reported-by: Lucas Alvares Gomes <lucasago...@gmail.com>
> Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> February/046174.html
> Reviewed-by: Mark Michelson <mmich...@redhat.com>
> Reviewed-by: Daniel Alvarez <dalva...@redhat.com>
> Signed-off-by: Han Zhou <hzh...@ebay.com>
> ---
>
> Notes:
> v1->v2: rebase
>
>  NEWS|   3 +-
>  ovn/northd/ovn-northd.c |  87 ---
>  ovn/ovn-nb.xml  |  18 
>  ovn/ovn-sb.xml  |  23 -
>  tests/ovn.at| 226 ++
> ++
>  5 files changed, 340 insertions(+), 17 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index ba9f0d8..c20edfc 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -16,7 +16,8 @@ Post-v2.9.0
> - Linux kernel 4.14
>   * Add support for compiling OVS with the latest Linux 4.14 kernel
> - OVN:
> - * Port_Group is supported in ACL match conditions.
> + * Port_Group and generated address sets are supported in ACL match
> +   conditions. See ovn-nb(5) and ovn-sb(5) for details.
>
>  v2.9.0 - 19 Feb 2018
>  
> diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
> index d4addf6..244e0cd 100644
> --- a/ovn/northd/ovn-northd.c
> +++ b/ovn/northd/ovn-northd.c
> @@ -6141,9 +6141,32 @@ build_lflows(struct northd_context *ctx, struct
> hmap *datapaths,
>  hmap_destroy();
>  }
>
> -/* OVN_Northbound and OVN_Southbound have an identical Address_Set table.
> - * We always update OVN_Southbound to match the current data in
> - * OVN_Northbound, so that the address sets used in Logical_Flows in
> +static void
> +sync_address_set(struct northd_context *ctx, const char *name,
> + const char **addrs, size_t n_addrs,
> + struct shash *sb_address_sets)
> +{
> +const struct sbrec_address_set *sb_address_set;
> +sb_address_set = shash_find_and_delete(sb_address_sets,
> +   name);
> +if (!sb_address_set) {
> +sb_address_set = sbrec_address_set_insert(ctx->ovnsb_txn);
> +sbrec_address_set_set_name(sb_address_set, name);
> +}
> +
> +sbrec_address_set_set_addresses(sb_address_set,
> +addrs, n_addrs);
> +}
> +
> +/* OVN_Southbound Address_Set table contains same records as in north
> + * bound, plus the records generated from Port_Group table in north bound.
> + *
> + * There are 2 records generated from each port group, one for IPv4, and
> + * one for IPv6, named in the format: _ip4 and
> + * _ip6 respectively. MAC addresses are ignored.
> + *
> + * We always update OVN_Southbound to match the Address_Set and Port_Group
> + * in OVN_Northbound, so that the address sets used in Logical_Flows in
>   * OVN_Southbound is checked against the proper set.*/
>  static void
>  sync_address_sets(struct northd_context *ctx)
> @@ -6155,19 +6178,55 @@ sync_address_sets(struct northd_context *ctx)
>  shash_add(_address_sets, sb_address_set->name, sb_address_set);
>  }
>
> -const struct nbrec_address_set *nb_address_set;
> -NBREC_ADDRESS_SET_FOR_EACH (nb_address_set, ctx->ovnnb_idl) {
> -sb_address_set = shash_find_and_delete(_address_sets,
> -   nb_address_set->name);
> -if (!sb_address_set) {
> -sb_address_set = sbrec_address_set_insert(ctx->ovnsb_txn);
> -sbrec_address_set_set_name(sb_address_set,
> nb_address_set->name);
> +/* sync port group generated address sets first */
> +const struct nbrec_port_group *nb_port_group;
> +NBREC_PORT_GROUP_FOR_EACH (nb_port_group, ctx->ovnnb_idl) {
> +const char **ipv4_addrs = xcalloc(1, sizeof *ipv4_addrs);
> +size_t n_ipv4_addrs = 0;
> +const char **ipv6_addrs = xcalloc(1, sizeof *ip

Re: [ovs-dev] [PATCH v2 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

Acked-by: Daniel Alvarez <dalva...@redhat.com>

Thanks Han! Everything LGTM and the test pass okay against current master.

On Thu, Apr 5, 2018 at 2:51 AM, Han Zhou <zhou...@gmail.com> wrote:

> This patch enables using port group names in ACL match conditions.
> Users can create a port group in northbound DB Port_Group table,
> and then use the name of the port group in ACL match conditions
> for "inport" or "outport". It can help reduce the number of ACLs
> for CMS clients such as OpenStack Neutron, for the use cases
> where a group of logical ports share same ACL rules except the
> "inport"/"outport" part. Without this patch, the clients have to
> create N (N = number of lports) ACLs, and this patch helps achieve
> the same goal with only one ACL. E.g.:
>
> to-lport 1000 "outport == @port_group1 && ip4.src == {IP1, IP2, ...}"
> allow-related
>
> There was a similar attempt by Zong Kai Li in 2016 [1]. This patch
> takes a slightly different approach by using weak refs instead of
> strings, which requires a new table instead of reusing the address
> set table. This way it will also benefit for a follow up patch that
> enables generating address sets automatically from port groups to
> avoid a lot a trouble from client perspective [2].
>
> An extra benefit of this patch is that it could enable conjunctive
> match effectively. As reported at [3], this patch was tested together
> with the conjunctive match enhancement patch [4], and huge performance
> improvement (more than 10x faster) was seen because of this.
>
> [1] https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/077118.html
> [2] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> February/046260.html
> [3] https://mail.openvswitch.org/pipermail/ovs-dev/2018-March/344873.html
> [4] https://patchwork.ozlabs.org/patch/874433/
>
> Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
> Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> February/046166.html
> Tested-by: Mark Michelson <mmich...@redhat.com>
> Reviewed-by: Mark Michelson <mmich...@redhat.com>
> Reviewed-by: Daniel Alvarez <dalva...@redhat.com>
> Signed-off-by: Han Zhou <hzh...@ebay.com>
> ---
>
> Notes:
> v1->v2:
> - rebased on master
> - corrected typos in logs/comments
>
>  NEWS|   2 +
>  include/ovn/expr.h  |  24 +---
>  include/ovn/lex.h   |   1 +
>  ovn/controller/lflow.c  |  16 +++--
>  ovn/controller/lflow.h  |   1 +
>  ovn/controller/ofctrl.c |   6 +-
>  ovn/controller/ofctrl.h |   3 +-
>  ovn/controller/ovn-controller.c |  31 +++---
>  ovn/lib/actions.c   |   3 +-
>  ovn/lib/expr.c  | 127 --
> --
>  ovn/lib/lex.c   |  20 +++
>  ovn/northd/ovn-northd.c |  47 +++
>  ovn/ovn-nb.ovsschema|  15 -
>  ovn/ovn-nb.xml  |  30 ++
>  ovn/ovn-sb.ovsschema|  10 +++-
>  ovn/ovn-sb.xml  |  20 +++
>  ovn/utilities/ovn-trace.c   |  27 +++--
>  tests/ovn.at|  34 +++
>  tests/test-ovn.c|  43 ++
>  19 files changed, 381 insertions(+), 79 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 0cfcac5..ba9f0d8 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -15,6 +15,8 @@ Post-v2.9.0
>   * OFPT_ROLE_STATUS is now available in OpenFlow 1.3.
> - Linux kernel 4.14
>   * Add support for compiling OVS with the latest Linux 4.14 kernel
> +   - OVN:
> + * Port_Group is supported in ACL match conditions.
>
>  v2.9.0 - 19 Feb 2018
>  
> diff --git a/include/ovn/expr.h b/include/ovn/expr.h
> index 711713e..3995e62 100644
> --- a/include/ovn/expr.h
> +++ b/include/ovn/expr.h
> @@ -382,9 +382,11 @@ expr_from_node(const struct ovs_list *node)
>  void expr_format(const struct expr *, struct ds *);
>  void expr_print(const struct expr *);
>  struct expr *expr_parse(struct lexer *, const struct shash *symtab,
> -const struct shash *addr_sets);
> +const struct shash *addr_sets,
> +const struct shash *port_groups);
>  struct expr *expr_parse_string(const char *, const struct shash *symtab,
> const struct shash *addr_sets,
> +   const struct shash *port_groups,
> char **errorp);
>
>  struct expr *expr_clone(struct expr *);
> @@ -4

Re: [ovs-dev] [PATCH v1] Fix AUTHORS.rst

[Daniel Alvarez Sanchez]

Acked-by: Daniel Alvarez <dalva...@redhat.com>


Just as a note, normally v1 is not used in the subject, that you can avoid
it by using "git send-email -1 --to=d...@openvswitch.org" and it will send
just the latest commit without adding the v1 tag to the email subject.


On Wed, Apr 11, 2018 at 11:30 AM, <lucasago...@gmail.com> wrote:

> From: Lucas Alvares Gomes <lucasago...@gmail.com>
>
> The commit 9afc6f14ee7b2622703d98689acb0044d4a5492e added a new author
> which name was too long that broke the column size. Runinng "make
> docs-checks" was failing because of that.
>
> All this patch does is to enlarge the "Name" column to fit the new
> author's name.
>
> Signed-off-by: Lucas Alvares Gomes <lucasago...@gmail.com>
> ---
>  AUTHORS.rst | 688 ++--
>  1 file changed, 344 insertions(+), 344 deletions(-)
>
> diff --git a/AUTHORS.rst b/AUTHORS.rst
> index c3e34df88..a097370a8 100644
> --- a/AUTHORS.rst
> +++ b/AUTHORS.rst
> @@ -28,351 +28,351 @@ Authors
>  The following people authored or signed off on commits in the Open
>  vSwitch source code or webpage version control repository.
>
> -=== ==
> =
> -NameEmail
> -=== ==
> =
> -Aaron Conoleacon...@redhat.com
> -Aaron Rosen aro...@clemson.edu
> -Alan Pevec  alan.pe...@redhat.com
> -Alexander Duyck alexander.h.du...@redhat.com
> -Alexandru Copot alex.miha...@gmail.com
> -Alexei Starovoitov  a...@plumgrid.com
> -Alexey I. Froloff   ra...@raorn.name
> -Alex Wang   ee07b...@gmail.com
> -Alfredo Finelli a...@computationes.de
> -Alin Balutoiu   abalut...@cloudbasesolutions.com
> -Alin Serdeanaserd...@cloudbasesolutions.com
> -Ambika Aroraambika.ar...@tcs.com
> -Amit Bose   b...@noironetworks.com
> -Amitabha Biswas azbis...@gmail.com
> -Anand Kumar kumaran...@vmware.com
> -Andrew Evansaev...@nicira.com
> -Andrew Beekhof  abeek...@redhat.com
> -Andrew Kampjes  a.kamp...@gmail.com
> -Andrew Lambeth  w...@nicira.com
> -Andy Hill   hil...@gmail.com
> -Andy Southgate  andy.southg...@citrix.com
> -Andy Zhou   az...@ovn.org
> -Ankur Sharmaankursha...@vmware.com
> -Anoob Soman anoob.so...@citrix.com
> -Ansis Attekaaatt...@nicira.com
> -Antonio Fischetti   antonio.fische...@intel.com
> -Anupam Chanda   acha...@nicira.com
> -Ariel Tubaltsev atubalt...@vmware.com
> -Arnoldo Lutzarnoldo.lutz.guev...@hpe.com
> -Arun Sharma arun.sha...@calsoftinc.com
> -Aryan TaheriMonfaredaryan.taherimonfa...@uis.no
> -Ashish Varmaashishvarma@gmail.com
> -Ashwin Swaminathan  ashwi...@arista.com
> -Babu Shanmugam  bscha...@redhat.com
> -Ben Pfaff   b...@ovn.org
> -Ben Warren  b...@skyportsystems.com
> -Benli Yedani...@vmware.com
> -Bert Vermeulen  b...@biot.com
> -Bhanuprakash Bodireddy  bhanuprakash.bodire...@intel.com
> -Billy O'Mahony  billy.o.mah...@intel.com
> -Binbin Xu   xu.binb...@zte.com.cn
> -Brian Krugerbkruger+ovs...@gmail.com
> -Bruce Davie b...@nicira.com
> -Bryan Phillippe b...@toroki.com
> -Carlo Andreotti c.andreo...@m3s.it
> -Casey Barkercrbar...@google.com
> -Chandra Sekhar Vejendla csvej...@us.ibm.com
> -Christoph Jaegerc...@linux.com
> -Chris Wrightchr...@sous-sol.org
> -Chuck Short zul...@ubuntu.com
> -Ciara Loftusciara.lof...@intel.com
> -Clint Byrum cl...@fewbar.com
> -Cong Wang   amw...@redhat.com
> -Conner Herriges conner.herri...@ibm.com
> -Damien Millescamps  damien.millesca...@6wind.com
> -Dan Carpenter   dan.carpen...@oracle.com
> -Dan McGregordan.mcgre...@usask.ca
> -Dan Wendlandt   d...@nicira.com
> -Dan Williamsd...@r

Re: [ovs-dev] [PATCH 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

Thanks Han!

On Thu, Apr 5, 2018 at 2:58 AM, Han Zhou  wrote:

> On Wed, Apr 4, 2018 at 4:58 PM, Ben Pfaff  wrote:
>
> > I see that there was a lot of feedback on this series, which seems to
> > have resulted in some constructive design decisions.  But I don't see a
> > v2 of the series.  Maybe I missed it, or maybe it is not posted yet.  Is
> > a v2 coming up?
> >
> > Thanks,
> >
> > Ben.
> >
>
> Hi Ben,
>
> I just sent v2:
> http://patchwork.ozlabs.org/project/openvswitch/list/?series=37476
>
> So far the major comment was about supporting applying ACLs directly on
> port-groups. We agreed that it will be very useful for both k8s and
> networking-ovn and now I am working on it, but it is a new feature on top
> of current patches, so it should not block reviewing/merging current ones.
>
> The other comments were about typos in warning messages/comments, which are
> corrected in v2, and v2 is also rebased on master. Please review :)
>
> Thanks,
> Han
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] python: Trivial fix flake8 error

[Daniel Alvarez]

make flake8-check fails due to missing whitespaces around
arithmetic operator.

Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---
 utilities/ovs-pipegen.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/utilities/ovs-pipegen.py b/utilities/ovs-pipegen.py
index 8a2a4266e..ee5797221 100755
--- a/utilities/ovs-pipegen.py
+++ b/utilities/ovs-pipegen.py
@@ -64,7 +64,7 @@ def l4(stage, action):
 match += ",ip_dst=%s/%d" % rand_ip_mask()
 
 src_dst = "tp_src" if rand_bool() else "tp_dst"
-match += ",%s=%d" % (src_dst, random.randint(1024, 2**16 - 1))
+match += ",%s=%d" % (src_dst, random.randint(1024, 2 ** 16 - 1))
 return flow_str(stage, match, action)
 
 
-- 
2.15.1 (Apple Git-101)

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

Reviewed-by: Daniel Alvarez <dalva...@redhat.com>

On Mon, Mar 12, 2018 at 8:41 PM, Han Zhou <zhou...@gmail.com> wrote:

> Thanks Daniel and Mark for the review. I will fix these in V2.
>
> On Mon, Mar 12, 2018 at 7:01 AM, Daniel Alvarez Sanchez <
> dalva...@redhat.com> wrote:
>
>> Looks good to me as well!
>> Just a tiny nit besides the correction that Mark pointed out about $/@.
>> Thanks, Han!
>>
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] ovsdb: Fix database compaction check

[Daniel Alvarez]

Thanks Ben!
Sorry for the confusion with the previous version,  I forgot to commit the 
changes and sent the wrong version. 

> On 14 Mar 2018, at 23:21, Ben Pfaff <b...@ovn.org> wrote:
> 
>> On Sat, Mar 10, 2018 at 02:50:14PM +0100, Daniel Alvarez wrote:
>> We want to compact database file if it has been over 24 hours since we
>> last compacted it and there's more than 100 commits regardless of the
>> size of the database. This patch fixes the previous comparisson which
>> checked if 24 hours was elapsed since the next scheduled compaction.
>> 
>> Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
> 
> OK, v2 is much better.  Applied to master, thanks!
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

FWIW, this [0] is making port deletion slow in OpenStack right now, which
basically is figuring out which ACL's belong to this particular Logical
Switch Port.
On a system with tons of ACL's this is expensive but it would be definitely
improved with this patch (as the # of ACL's will drop down) but we could
totally
get rid of it if we would add the ACLs to the Port Group table.

[0]
https://github.com/openstack/networking-ovn/blob/master/networking_ovn/ovsdb/commands.py#L460-#L463


On Thu, Mar 1, 2018 at 9:26 PM, Guru Shetty <g...@ovn.org> wrote:

> On 1 March 2018 at 12:21, Han Zhou <zhou...@gmail.com> wrote:
>
> >
> >
> > On Thu, Mar 1, 2018 at 12:13 PM, Guru Shetty <g...@ovn.org> wrote:
> > >
> > >
> > >
> > > On 28 February 2018 at 19:37, Han Zhou <zhou...@gmail.com> wrote:
> > >>
> > >> This patch enables using port group names in ACL match conditions.
> > >> Users can create a port group in northbound DB Port_Group table,
> > >> and then use the name of the port group in ACL match conditions
> > >> for "inport" or "outport". It can help reduce the number of ACLs
> > >> for CMS clients such as OpenStack Neutron, for the use cases
> > >> where a group of logical ports share same ACL rules except the
> > >> "inport"/"outport" part. Without this patch, the clients have to
> > >> create N (N = number of lports) ACLs, and this patch helps achieve
> > >> the same goal with only one ACL. E.g.:
> > >>
> > >> to-lport 1000 "outport == @port_group1 && ip4.src == {IP1, IP2, ...}"
> > allow-related
> > >>
> > >> There was a similar attempt by Zong Kai Li in 2016 [1]. This patch
> > >> takes a slightly different approach by using weak refs instead of
> > >> strings, which requires a new table instead of reusing the address
> > >> set table. This way it will also benefit for a follow up patch that
> > >> enables generating address sets automatically from port groups to
> > >> avoid a lot a trouble from client perspective [2].
> > >>
> > >> [1] https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/
> > 077118.html
> > >> [2] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> > February/046260.html
> > >>
> > >> Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
> > >> Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> > February/046166.html
> > >> Signed-off-by: Han Zhou <hzh...@ebay.com>
> > >
> > >
> > > Wouldn't it be more complete and useful if we add the acl to a port
> > group too? And then internally, you decide which switches you want to add
> > the ACL to.
> > >
> > > For e.g: ovn-nbctl --port-group add-acl port_group1 to-lport 1000
> > "outport == @port_group1 && ip4.src == {IP1, IP2, ...}" allow-related
> > >
> > > This way, the client does not have to keep track of all the logical
> > switches it needs to apply an ACL to. Thoughts?
> > >
> > Yes, this is a good idea. Since it is only about the ovn-nbctl tool
> > improvement, it can be a follow up patch.
> >
>
> What if we have something like a acl column in the port_group table so that
> we just have one entry in OVN NB database? Logically, we apply a ACL to a
> security group instead of a  logical switch. And then ovn-northd decided
> which logical switches to apply it to. Would that make difference in
> performance? It does reduce the size of the NB database. Any drawbacks?
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

Looks good to me as well!
Just a tiny nit besides the correction that Mark pointed out about $/@.
Thanks, Han!

On Thu, Mar 1, 2018 at 4:37 AM, Han Zhou <zhou...@gmail.com> wrote:

> This patch enables using port group names in ACL match conditions.
> Users can create a port group in northbound DB Port_Group table,
> and then use the name of the port group in ACL match conditions
> for "inport" or "outport". It can help reduce the number of ACLs
> for CMS clients such as OpenStack Neutron, for the use cases
> where a group of logical ports share same ACL rules except the
> "inport"/"outport" part. Without this patch, the clients have to
> create N (N = number of lports) ACLs, and this patch helps achieve
> the same goal with only one ACL. E.g.:
>
> to-lport 1000 "outport == @port_group1 && ip4.src == {IP1, IP2, ...}"
> allow-related
>
> There was a similar attempt by Zong Kai Li in 2016 [1]. This patch
> takes a slightly different approach by using weak refs instead of
> strings, which requires a new table instead of reusing the address
> set table. This way it will also benefit for a follow up patch that
> enables generating address sets automatically from port groups to
> avoid a lot a trouble from client perspective [2].
>
> [1] https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/077118.html
> [2] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-Febr
> uary/046260.html
>
> Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
> Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-Febr
> uary/046166.html
> Signed-off-by: Han Zhou <hzh...@ebay.com>
> ---
>  NEWS|   2 +
>  include/ovn/expr.h  |  24 +---
>  include/ovn/lex.h   |   1 +
>  ovn/controller/lflow.c  |  16 +++--
>  ovn/controller/lflow.h  |   1 +
>  ovn/controller/ofctrl.c |   6 +-
>  ovn/controller/ofctrl.h |   3 +-
>  ovn/controller/ovn-controller.c |  31 +++---
>  ovn/lib/actions.c   |   3 +-
>  ovn/lib/expr.c  | 127 --
> --
>  ovn/lib/lex.c   |  20 +++
>  ovn/northd/ovn-northd.c |  47 +++
>  ovn/ovn-nb.ovsschema|  15 -
>  ovn/ovn-nb.xml  |  30 ++
>  ovn/ovn-sb.ovsschema|  10 +++-
>  ovn/ovn-sb.xml  |  20 +++
>  ovn/utilities/ovn-trace.c   |  27 +++--
>  tests/ovn.at|  34 +++
>  tests/test-ovn.c|  43 ++
>  19 files changed, 381 insertions(+), 79 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 4230189..da2c3ec 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -14,6 +14,8 @@ Post-v2.9.0
>   * OFPT_ROLE_STATUS is now available in OpenFlow 1.3.
> - Linux kernel 4.14
>   * Add support for compiling OVS with the latest Linux 4.14 kernel
> +   - OVN:
> + * Port_Group is supported in ACL match conditions.
>
>  v2.9.0 - 19 Feb 2018
>  
> diff --git a/include/ovn/expr.h b/include/ovn/expr.h
> index 711713e..3995e62 100644
> --- a/include/ovn/expr.h
> +++ b/include/ovn/expr.h
> @@ -382,9 +382,11 @@ expr_from_node(const struct ovs_list *node)
>  void expr_format(const struct expr *, struct ds *);
>  void expr_print(const struct expr *);
>  struct expr *expr_parse(struct lexer *, const struct shash *symtab,
> -const struct shash *addr_sets);
> +const struct shash *addr_sets,
> +const struct shash *port_groups);
>  struct expr *expr_parse_string(const char *, const struct shash *symtab,
> const struct shash *addr_sets,
> +   const struct shash *port_groups,
> char **errorp);
>
>  struct expr *expr_clone(struct expr *);
> @@ -404,6 +406,7 @@ bool expr_is_normalized(const struct expr *);
>
>  char *expr_parse_microflow(const char *, const struct shash *symtab,
> const struct shash *addr_sets,
> +   const struct shash *port_groups,
> bool (*lookup_port)(const void *aux,
> const char *port_name,
> unsigned int *portp),
> @@ -486,19 +489,22 @@ void expr_constant_set_format(const struct
> expr_constant_set *, struct ds *);
>  void expr_constant_set_destroy(struct expr_constant_set *cs);
>
>
> -/* Address sets.
> +/* Constant sets.
>   *
> - * Instead of referring to a set of va

Re: [ovs-dev] [PATCH 2/2] ovn: Support address sets generated from port groups

[Daniel Alvarez Sanchez]

ulate the hypervisors' ARP tables so that we don't lose any
> +# packets for ARP resolution (native tunneling doesn't queue packets
> +# for ARP resolution).
> +OVN_POPULATE_ARP
> +
> +# Allow some time for ovn-northd and ovn-controller to catch up.
> +# XXX This should be more systematic.
> +sleep 1
> +
> +# test_ip INPORT SRC_MAC DST_MAC SRC_IP DST_IP OUTPORT...
> +#
> +# This shell function causes a packet to be received on INPORT.  The
> packet's
> +# content has Ethernet destination DST and source SRC (each exactly 12 hex
> +# digits) and Ethernet type ETHTYPE (4 hex digits).  The OUTPORTs (zero or
> +# more) list the VIFs on which the packet should be received.  INPORT and
> the
> +# OUTPORTs are specified as logical switch port numbers, e.g. 123 for
> vif123.
> +for i in 1 2 3; do
> +for j in 1 2 3; do
> +for k in 1 2 3; do
> +: > $i$j$k.expected
> +done
> +done
> +done
> +test_ip() {
> +# This packet has bad checksums but logical L3 routing doesn't check.
> +local inport=$1 src_mac=$2 dst_mac=$3 src_ip=$4 dst_ip=$5
> +local packet=${dst_mac}${src_mac}0800451c4011${
> src_ip}${dst_ip}00350008
> +shift; shift; shift; shift; shift
> +hv=hv`vif_to_hv $inport`
> +as $hv ovs-appctl netdev-dummy/receive vif$inport $packet
> +#as $hv ovs-appctl ofproto/trace br-int in_port=$inport $packet
> +in_ls=`vif_to_ls $inport`
> +in_lrp=`vif_to_lrp $inport`
> +for outport; do
> +out_ls=`vif_to_ls $outport`
> +if test $in_ls = $out_ls; then
> +# Ports on the same logical switch receive exactly the same
> packet.
> +echo $packet
> +else
> +# Routing decrements TTL and updates source and dest MAC
> +# (and checksum).
> +out_lrp=`vif_to_lrp $outport`
> +echo f${outport}ff$
> {out_lrp}0800451c"3f1101"00${src_ip}${dst_ip}00350008
> +fi >> $outport.expected
> +done
> +}
>
Neat! this and test_arp are really useful functions to tests :)

> +
> +as hv1 ovs-vsctl --columns=name,ofport list interface
> +as hv1 ovn-sbctl list port_binding
> +as hv1 ovn-sbctl list datapath_binding
> +as hv1 ovn-sbctl list port_group
> +as hv1 ovn-sbctl list address_set
> +as hv1 ovn-sbctl dump-flows
> +as hv1 ovs-ofctl dump-flows br-int
>
nit: We can remove this block.

> +
> +# Send IP packets between all pairs of source and destination ports,
> +# packets matches ACL (pg2 to pg1) should be dropped
> +ip_to_hex() {
> +printf "%02x%02x%02x%02x" "$@"
> +}
> +for is in 1 2 3; do
> +  for js in 1 2 3; do
> +for ks in 1 2 3; do
> +  bcast=
> +  s=$is$js$ks
> +  smac=f$s
> +  sip=`ip_to_hex 192 168 $is$js $ks`
> +  for id in 1 2 3; do
> +  for jd in 1 2 3; do
> +  for kd in 1 2 3; do
> +d=$id$jd$kd
> +dip=`ip_to_hex 192 168 $id$jd $kd`
> +if test $is = $id; then dmac=f$d; else
> dmac=ff$is$js; fi
> +if test $d != $s; then unicast=$d; else unicast=; fi
> +
> +# packets matches ACL should be dropped
> +if test $id != 3 && test $kd == 1; then
> +if test $is != 1 && test $ks == 2; then
> +unicast=
> +fi
> +fi
> +test_ip $s $smac $dmac $sip $dip $unicast #1
> +  done
> +  done
> +done
> +  done
> +  done
> +done
> +
> +# Allow some time for packet forwarding.
> +# XXX This can be improved.
> +sleep 1
> +
> +# Now check the packets actually received against the ones expected.
> +for i in 1 2 3; do
> +for j in 1 2 3; do
> +for k in 1 2 3; do
> +OVN_CHECK_PACKETS([hv`vif_to_hv $i$j$k`/vif$i$j$k-tx.pcap],
> +  [$i$j$k.expected])
> +done
> +done
> +done
> +
> +# Gracefully terminate daemons
> +OVN_CLEANUP([hv1], [hv2], [hv3])
> +AT_CLEANUP
> --
> 2.1.0
>

Acked-by: Daniel Alvarez <dalva...@redhat.com>

>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovsdb: compact databases more strictly

[Daniel Alvarez Sanchez]

Done: https://patchwork.ozlabs.org/patch/884143/
Thanks a lot again for taking a look.
I'd appreciate the backport into 2.9 as well if you all agree.

Regards,
Daniel

On Sat, Mar 10, 2018 at 1:35 PM, Daniel Alvarez Sanchez <dalva...@redhat.com
> wrote:

> Thanks a lot Ben and Mark.
> Yes, I'll be sending it right away. Thanks a lot guys.
>
> On Fri, Mar 9, 2018 at 8:13 PM, Ben Pfaff <b...@ovn.org> wrote:
>
>> On Fri, Mar 09, 2018 at 08:05:22AM -0600, Mark Michelson wrote:
>> > Hi Daniel,
>> >
>> > Mostly this looks correct. I had one small finding and have noted it
>> in-line
>> > down below.
>> >
>> > On 03/08/2018 04:20 PM, Daniel Alvarez wrote:
>> > >Before this patch, the databases were automatically compacted when a
>> > >transaction is logged when:
>> > >
>> > >* It's been > 10 minutes after last compaction AND
>> > >* At least 100 commits have occurred AND
>> > >* Database has grown at least 4x since last compaction (and it's > 10M)
>> > >
>> > >This patch changes the conditions as follows:
>> > >
>> > >* It's been > 10 minutes after last compaction AND
>> > >* At least 100 commits have occurred AND either
>> > >- It's been > 24 hours after the last compaction OR
>> > >- Database has grown at least 2x since last compaction (and it's >
>> 10M)
>> > >
>> > >Reported-by: Daniel Alvarez <dalva...@redhat.com>
>> > >Reported-at: https://mail.openvswitch.org/p
>> ipermail/ovs-discuss/2018-March/046309.html
>> > >Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
>> > >---
>> > >  ovsdb/file.c| 17 +
>> > >  ovsdb/log.c |  2 +-
>> > >  ovsdb/ovsdb-server.1.in |  6 --
>> > >  3 files changed, 18 insertions(+), 7 deletions(-)
>> > >
>> > >diff --git a/ovsdb/file.c b/ovsdb/file.c
>> > >index 4b7ad52ab..02e0e8b76 100644
>> > >--- a/ovsdb/file.c
>> > >+++ b/ovsdb/file.c
>> > >@@ -46,6 +46,10 @@ VLOG_DEFINE_THIS_MODULE(ovsdb_file);
>> > >   * compacting fails. */
>> > >  #define COMPACT_RETRY_MSEC  (60 * 1000)  /* 1 minute. */
>> > >+/* Maximum number of milliseconds before compacting the database
>> regardless
>> > >+ * of its size. */
>> > >+#define COMPACT_MAX_MSEC(24 * 60 * 60 * 1000) /* 24 hours. */
>> > >+
>> > >  /* A transaction being converted to JSON for writing to a file. */
>> > >  struct ovsdb_file_txn {
>> > >  struct json *json;  /* JSON for the whole transaction. */
>> > >@@ -586,6 +590,7 @@ ovsdb_file_commit(struct ovsdb_replica *replica,
>> > >  struct ovsdb_file *file = ovsdb_file_cast(replica);
>> > >  struct ovsdb_file_txn ftxn;
>> > >  struct ovsdb_error *error;
>> > >+long long int next_compact_elapsed;
>> > >  ovsdb_file_txn_init();
>> > >  ovsdb_txn_for_each_change(txn, ovsdb_file_change_cb, );
>> > >@@ -604,11 +609,15 @@ ovsdb_file_commit(struct ovsdb_replica *replica,
>> > >  /* If it has been at least COMPACT_MIN_MSEC ms since the last
>> time we
>> > >   * compacted (or at least COMPACT_RETRY_MSEC ms since the last
>> time we
>> > >   * tried), and if there are at least 100 transactions in the
>> database, and
>> > >- * if the database is at least 10 MB, and the database is at
>> least 4x the
>> > >- * size of the previous snapshot, then compact the database. */
>> > >-if (time_msec() >= file->next_compact
>> > >+ * if the database is at least 10 MB, and the database is at
>> least 2x the
>> > >+ * size of the previous snapshot, then compact the database.
>> However, if
>> > >+ * it has been over COMPACT_MAX_MSEC ms, the database size is not
>> taken
>> > >+ * into account. */
>> > >+next_compact_elapsed = time_msec() - file->next_compact;
>> > >+if (next_compact_elapsed >= 0
>> > >  && file->n_transactions >= 100
>> > >-&& ovsdb_log_grew_lots(file->log)) {
>> > >+&& (next_compact_elapsed >= COMPACT_MAX_MSEC
>> >
>> > I think this line is not right. If the idea is to compact if it has been
>> > more than 24 hours since the last time we did, then we should be doing
>> >
>> > time_msec() - file->last_compact >= COMPACT_MAX_MSEC
>> >
>> > instead of
>> >
>> > time_msec() - file->next_compact >= COMPACT_MAX_MSEC
>>
>> Oops, I missed this before I committed.  Daniel, would you mind sending
>> a fix?
>>
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2] ovsdb: Fix database compaction check

[Daniel Alvarez]

We want to compact database file if it has been over 24 hours since we
last compacted it and there's more than 100 commits regardless of the
size of the database. This patch fixes the previous comparisson which
checked if 24 hours was elapsed since the next scheduled compaction.

Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---
 ovsdb/file.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/ovsdb/file.c b/ovsdb/file.c
index 02e0e8b76..40192e30d 100644
--- a/ovsdb/file.c
+++ b/ovsdb/file.c
@@ -590,7 +590,7 @@ ovsdb_file_commit(struct ovsdb_replica *replica,
 struct ovsdb_file *file = ovsdb_file_cast(replica);
 struct ovsdb_file_txn ftxn;
 struct ovsdb_error *error;
-long long int next_compact_elapsed;
+long long int current_time;
 
 ovsdb_file_txn_init();
 ovsdb_txn_for_each_change(txn, ovsdb_file_change_cb, );
@@ -611,12 +611,12 @@ ovsdb_file_commit(struct ovsdb_replica *replica,
  * tried), and if there are at least 100 transactions in the database, and
  * if the database is at least 10 MB, and the database is at least 2x the
  * size of the previous snapshot, then compact the database. However, if
- * it has been over COMPACT_MAX_MSEC ms, the database size is not taken
- * into account. */
-next_compact_elapsed = time_msec() - file->next_compact;
-if (next_compact_elapsed >= 0
+ * it has been over COMPACT_MAX_MSEC ms since the last compaction, the
+ * database size is not taken into account. */
+current_time = time_msec();
+if (current_time >= file->next_compact
 && file->n_transactions >= 100
-&& (next_compact_elapsed >= COMPACT_MAX_MSEC
+&& (current_time - file->last_compact >= COMPACT_MAX_MSEC
 || ovsdb_log_grew_lots(file->log))) {
 error = ovsdb_file_compact(file);
 if (error) {
-- 
2.13.5

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] ovsdb: Fix database compaction check

[Daniel Alvarez]

We want to compact database file if it has been over 24 hours since we
last compacted it and there's more than 100 commits regardless of the
size of the database. This patch fixes the previous comparisson which
checked if 24 hours was elapsed since the next scheduled compaction.

Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---
 ovsdb/file.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/ovsdb/file.c b/ovsdb/file.c
index 02e0e8b76..333cd00a6 100644
--- a/ovsdb/file.c
+++ b/ovsdb/file.c
@@ -611,12 +611,12 @@ ovsdb_file_commit(struct ovsdb_replica *replica,
  * tried), and if there are at least 100 transactions in the database, and
  * if the database is at least 10 MB, and the database is at least 2x the
  * size of the previous snapshot, then compact the database. However, if
- * it has been over COMPACT_MAX_MSEC ms, the database size is not taken
- * into account. */
-next_compact_elapsed = time_msec() - file->next_compact;
-if (next_compact_elapsed >= 0
+ * it has been over COMPACT_MAX_MSEC ms since the last compaction, the
+ * database size is not taken into account. */
+current_time = time_msec();
+if (current_time >= file->next_compact
 && file->n_transactions >= 100
-&& (next_compact_elapsed >= COMPACT_MAX_MSEC
+&& (curent_time - file->last_compact >= COMPACT_MAX_MSEC
 || ovsdb_log_grew_lots(file->log))) {
 error = ovsdb_file_compact(file);
 if (error) {
-- 
2.13.5

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovsdb: compact databases more strictly

[Daniel Alvarez Sanchez]

Thanks a lot Ben and Mark.
Yes, I'll be sending it right away. Thanks a lot guys.

On Fri, Mar 9, 2018 at 8:13 PM, Ben Pfaff <b...@ovn.org> wrote:

> On Fri, Mar 09, 2018 at 08:05:22AM -0600, Mark Michelson wrote:
> > Hi Daniel,
> >
> > Mostly this looks correct. I had one small finding and have noted it
> in-line
> > down below.
> >
> > On 03/08/2018 04:20 PM, Daniel Alvarez wrote:
> > >Before this patch, the databases were automatically compacted when a
> > >transaction is logged when:
> > >
> > >* It's been > 10 minutes after last compaction AND
> > >* At least 100 commits have occurred AND
> > >* Database has grown at least 4x since last compaction (and it's > 10M)
> > >
> > >This patch changes the conditions as follows:
> > >
> > >* It's been > 10 minutes after last compaction AND
> > >* At least 100 commits have occurred AND either
> > >- It's been > 24 hours after the last compaction OR
> > >- Database has grown at least 2x since last compaction (and it's >
> 10M)
> > >
> > >Reported-by: Daniel Alvarez <dalva...@redhat.com>
> > >Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> March/046309.html
> > >Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
> > >---
> > >  ovsdb/file.c| 17 +
> > >  ovsdb/log.c |  2 +-
> > >  ovsdb/ovsdb-server.1.in |  6 --
> > >  3 files changed, 18 insertions(+), 7 deletions(-)
> > >
> > >diff --git a/ovsdb/file.c b/ovsdb/file.c
> > >index 4b7ad52ab..02e0e8b76 100644
> > >--- a/ovsdb/file.c
> > >+++ b/ovsdb/file.c
> > >@@ -46,6 +46,10 @@ VLOG_DEFINE_THIS_MODULE(ovsdb_file);
> > >   * compacting fails. */
> > >  #define COMPACT_RETRY_MSEC  (60 * 1000)  /* 1 minute. */
> > >+/* Maximum number of milliseconds before compacting the database
> regardless
> > >+ * of its size. */
> > >+#define COMPACT_MAX_MSEC(24 * 60 * 60 * 1000) /* 24 hours. */
> > >+
> > >  /* A transaction being converted to JSON for writing to a file. */
> > >  struct ovsdb_file_txn {
> > >  struct json *json;  /* JSON for the whole transaction. */
> > >@@ -586,6 +590,7 @@ ovsdb_file_commit(struct ovsdb_replica *replica,
> > >  struct ovsdb_file *file = ovsdb_file_cast(replica);
> > >  struct ovsdb_file_txn ftxn;
> > >  struct ovsdb_error *error;
> > >+long long int next_compact_elapsed;
> > >  ovsdb_file_txn_init();
> > >  ovsdb_txn_for_each_change(txn, ovsdb_file_change_cb, );
> > >@@ -604,11 +609,15 @@ ovsdb_file_commit(struct ovsdb_replica *replica,
> > >  /* If it has been at least COMPACT_MIN_MSEC ms since the last
> time we
> > >   * compacted (or at least COMPACT_RETRY_MSEC ms since the last
> time we
> > >   * tried), and if there are at least 100 transactions in the
> database, and
> > >- * if the database is at least 10 MB, and the database is at least
> 4x the
> > >- * size of the previous snapshot, then compact the database. */
> > >-if (time_msec() >= file->next_compact
> > >+ * if the database is at least 10 MB, and the database is at least
> 2x the
> > >+ * size of the previous snapshot, then compact the database.
> However, if
> > >+ * it has been over COMPACT_MAX_MSEC ms, the database size is not
> taken
> > >+ * into account. */
> > >+next_compact_elapsed = time_msec() - file->next_compact;
> > >+if (next_compact_elapsed >= 0
> > >  && file->n_transactions >= 100
> > >-&& ovsdb_log_grew_lots(file->log)) {
> > >+&& (next_compact_elapsed >= COMPACT_MAX_MSEC
> >
> > I think this line is not right. If the idea is to compact if it has been
> > more than 24 hours since the last time we did, then we should be doing
> >
> > time_msec() - file->last_compact >= COMPACT_MAX_MSEC
> >
> > instead of
> >
> > time_msec() - file->next_compact >= COMPACT_MAX_MSEC
>
> Oops, I missed this before I committed.  Daniel, would you mind sending
> a fix?
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovsdb: Correctly log time since last compaction.

[Daniel Alvarez Sanchez]

Ops! I think we sent it in the same minute. Thanks a lot Ben! :)

On Wed, Mar 7, 2018 at 7:12 PM, Ben Pfaff <b...@ovn.org> wrote:

> On Wed, Mar 07, 2018 at 10:01:59AM -0800, Ben Pfaff wrote:
> > file->last_compact is initialized from time_msec() so the difference has
> > to be computed relative to that clock, not against wall clock time.
> >
> > This only affected the log message, not the decision about when to
> > compact.
> >
> > Signed-off-by: Ben Pfaff <b...@ovn.org>
> > Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
> > Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> March/046316.html
> > Signed-off-by: Ben Pfaff <b...@ovn.org>
>
> Oops, I guess I noticed this from your message after you send the same
> fix yourself.  I applied yours, and everyone can ignore this one.
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] ovsdb: Fix time in log traces when compacting database

[Daniel Alvarez]

Current code is mixing wall and monotonic clocks and the traces are not
useful since the timestamps are not accurate. This patch fixes it by
using the same time reference for the log as used in the code.

Without this patch, the traces look like this:
compacting database online (1519124364.908 seconds old, 951 transactions)

Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---
 ovsdb/file.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ovsdb/file.c b/ovsdb/file.c
index 90c2b9d20..4b7ad52ab 100644
--- a/ovsdb/file.c
+++ b/ovsdb/file.c
@@ -655,7 +655,7 @@ ovsdb_file_compact(struct ovsdb_file *file)
 
 comment = xasprintf("compacting database online "
 "(%.3f seconds old, %u transactions)",
-(time_wall_msec() - file->last_compact) / 1000.0,
+(time_msec() - file->last_compact) / 1000.0,
 file->n_transactions);
 VLOG_INFO("%s: %s", file->file_name, comment);
 
-- 
2.13.5

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

On Fri, Mar 2, 2018 at 3:22 PM, Mark Michelson <mmich...@redhat.com> wrote:

> On 03/02/2018 05:57 AM, Daniel Alvarez Sanchez wrote:
>
>>
>>
>> On Fri, Mar 2, 2018 at 12:49 AM, Mark Michelson <mmich...@redhat.com
>> <mailto:mmich...@redhat.com>> wrote:
>>
>> Hi Han,
>>
>> Current criticisms notwithstanding, I decided to test your patch
>> with a scenario that I thought it would help with.
>>
>> The base scenario is that I create an address set with 1000
>> addresses in it. I then create logical switch ports and create a new
>> ACL for each port:
>>
>> ovn-nbctl acl-add ls0 to-lport 1000 "outport == \"lsp${COUNT}\" &&
>> ip4.dst == \$set1" allow
>>
>> The idea is that I'm creating a bunch of ACLs with identical L3
>> restrictions but with different ports. I ran this using the 'time'
>> utility, and here's the results:
>>
>> For 100 ports: 2m17.979s
>> For 200 ports: 9m29.771s
>>
>> The big reason for this is that the ACLs generate a lot of flows.
>> For 100 ports, it generates 102501 flows in table 44 (ACL egress
>> table). For 200 ports, it generates 205001 flows in that table.
>> Dobuling the number of ports doubles the number of flows. And
>> doubling the number of flows quadruples the amount of time it takes
>> to run.
>>
>> I applied your patch and modified the scenario a bit. Instead of
>> adding an ACL for each new port, I instead created one ACL. It looks
>> like:
>>
>> ovn-nbctl acl-add ls0 to-lport 1000 "outport == @pg1 && ip4.dst ==
>> \$set1" allow
>>
>> For each added port, I add the port to group pg1. Re-running the
>> scenario, I got pretty much the same results. Honestly, I expected
>> this.
>>
>> Now here's the exciting part. I then applied Numan Siddique's
>> conjunctive match patch on top of yours. I then re-ran the test and
>> got these results:
>>
>> For 100 ports: 6.453s
>> For 200 ports: 16.008s
>>
>> Yeah, that's much better! I actually thought I had messed something
>> up at first, but after double-checking the generated flows, all was
>> correct. With 100 ports, table 44 has 1127 flows. Rather than
>> requiring 100 ports * 1000 addresses, it creates a conjunctive match
>> of 100 ports + 1000 addresses. For 200 ports, table 44 has 1227
>> flows. It added exactly 100 flows over the 100 port scenario.
>>
>>
>> This sounds great Mark! How are you doing those tests? Just invoking
>> ovn-nbctl 2 times per port (1 for creating the port + 1 for inserting the
>> ACL)?  so, every time you create a port, you'll be opening 2 new
>> connections to the OVSDB and getting a full dump of the database contents.
>> That said, maybe with 200 ports it doesn't make a huge difference but the
>> most realistic way IMHO would be to open a single IDL connection and then
>> try to create the ports on that one. Of course, after Han's patch, the NB
>> database size is much smaller so doing everything on a single IDL
>> connection won't be that noticeable but I think it would be in your
>> previous scenario (inserting ACL's for each new port). I wonder how the
>> times would look like as I expect them to be far less than those 9.5
>> minutes you get for 200 ports.
>> In my tests using Python IDL it made a huge difference!
>>
>
> Yes, I'm just using a bash script in the ovs sandbox that uses ovn-nbctl
> to do all database operations. For the unpatched scenario, I call ovn-nbctl
> twice for each port: once to create the port and once to create the new
> ACL. For the patched scenario, I also call ovn-nbctl twice for each port:
> once to create the port and once to add the port to the port group.
>
> As far as realism is concerned, you're definitely correct from an
> OpenStack perspective. However, there are other platforms (ovn-kubernetes
> for one) that are using ovn-nbctl for all their database interactions. I
> think the bigger takeaway here are the deltas than the actual times.
>

Thanks for the explanation Marks. Gotcha! I was not aware that
ovn-kubernetes is using ovn-nbctl for their database interactions although
IMHO they may want to switch eventually to a more "native" way of doing it
since it has the aforementioned advantage. This is just my views though and
there's possibly other reasons why they won't ever do it that I'm
absolutely unaware.
That said, I agree that the important thing here is the deltas and that's
why I ment

Re: [ovs-dev] [PATCH 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

On Fri, Mar 2, 2018 at 12:49 AM, Mark Michelson <mmich...@redhat.com> wrote:

> Hi Han,
>
> Current criticisms notwithstanding, I decided to test your patch with a
> scenario that I thought it would help with.
>
> The base scenario is that I create an address set with 1000 addresses in
> it. I then create logical switch ports and create a new ACL for each port:
>
> ovn-nbctl acl-add ls0 to-lport 1000 "outport == \"lsp${COUNT}\" && ip4.dst
> == \$set1" allow
>
> The idea is that I'm creating a bunch of ACLs with identical L3
> restrictions but with different ports. I ran this using the 'time' utility,
> and here's the results:
>
> For 100 ports: 2m17.979s
> For 200 ports: 9m29.771s
>
> The big reason for this is that the ACLs generate a lot of flows. For 100
> ports, it generates 102501 flows in table 44 (ACL egress table). For 200
> ports, it generates 205001 flows in that table. Dobuling the number of
> ports doubles the number of flows. And doubling the number of flows
> quadruples the amount of time it takes to run.
>
> I applied your patch and modified the scenario a bit. Instead of adding an
> ACL for each new port, I instead created one ACL. It looks like:
>
> ovn-nbctl acl-add ls0 to-lport 1000 "outport == @pg1 && ip4.dst == \$set1"
> allow
>
> For each added port, I add the port to group pg1. Re-running the scenario,
> I got pretty much the same results. Honestly, I expected this.
>
> Now here's the exciting part. I then applied Numan Siddique's conjunctive
> match patch on top of yours. I then re-ran the test and got these results:
>
> For 100 ports: 6.453s
> For 200 ports: 16.008s
>
> Yeah, that's much better! I actually thought I had messed something up at
> first, but after double-checking the generated flows, all was correct. With
> 100 ports, table 44 has 1127 flows. Rather than requiring 100 ports * 1000
> addresses, it creates a conjunctive match of 100 ports + 1000 addresses.
> For 200 ports, table 44 has 1227 flows. It added exactly 100 flows over the
> 100 port scenario.
>

This sounds great Mark! How are you doing those tests? Just invoking
ovn-nbctl 2 times per port (1 for creating the port + 1 for inserting the
ACL)?  so, every time you create a port, you'll be opening 2 new
connections to the OVSDB and getting a full dump of the database contents.
That said, maybe with 200 ports it doesn't make a huge difference but the
most realistic way IMHO would be to open a single IDL connection and then
try to create the ports on that one. Of course, after Han's patch, the NB
database size is much smaller so doing everything on a single IDL
connection won't be that noticeable but I think it would be in your
previous scenario (inserting ACL's for each new port). I wonder how the
times would look like as I expect them to be far less than those 9.5
minutes you get for 200 ports.
In my tests using Python IDL it made a huge difference!


>
> From a performance perspective, this is great! I'll give the actual code a
> review in the upcoming days, hopefully before the weekend.
>
>
> On 02/28/2018 09:37 PM, Han Zhou wrote:
>
>> This patch enables using port group names in ACL match conditions.
>> Users can create a port group in northbound DB Port_Group table,
>> and then use the name of the port group in ACL match conditions
>> for "inport" or "outport". It can help reduce the number of ACLs
>> for CMS clients such as OpenStack Neutron, for the use cases
>> where a group of logical ports share same ACL rules except the
>> "inport"/"outport" part. Without this patch, the clients have to
>> create N (N = number of lports) ACLs, and this patch helps achieve
>> the same goal with only one ACL. E.g.:
>>
>> to-lport 1000 "outport == @port_group1 && ip4.src == {IP1, IP2, ...}"
>> allow-related
>>
>> There was a similar attempt by Zong Kai Li in 2016 [1]. This patch
>> takes a slightly different approach by using weak refs instead of
>> strings, which requires a new table instead of reusing the address
>> set table. This way it will also benefit for a follow up patch that
>> enables generating address sets automatically from port groups to
>> avoid a lot a trouble from client perspective [2].
>>
>> [1] https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/0
>> 77118.html
>> [2] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-Febr
>> uary/046260.html
>>
>> Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
>> Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-Febr
>> uary/046166.html
>> Signed-off-by: Han Zhou <hzh...@ebay.

Re: [ovs-dev] [PATCH 1/2] ovn: Support port groups in ACLs

[Daniel Alvarez Sanchez]

Thanks for the patches Han!
In the other patch of the series I left a script to help others review it.


On Thu, Mar 1, 2018 at 4:37 AM, Han Zhou <zhou...@gmail.com> wrote:

> This patch enables using port group names in ACL match conditions.
> Users can create a port group in northbound DB Port_Group table,
> and then use the name of the port group in ACL match conditions
> for "inport" or "outport". It can help reduce the number of ACLs
> for CMS clients such as OpenStack Neutron, for the use cases
> where a group of logical ports share same ACL rules except the
> "inport"/"outport" part. Without this patch, the clients have to
> create N (N = number of lports) ACLs, and this patch helps achieve
> the same goal with only one ACL. E.g.:
>
> to-lport 1000 "outport == @port_group1 && ip4.src == {IP1, IP2, ...}"
> allow-related
>
> There was a similar attempt by Zong Kai Li in 2016 [1]. This patch
> takes a slightly different approach by using weak refs instead of
> strings, which requires a new table instead of reusing the address
> set table. This way it will also benefit for a follow up patch that
> enables generating address sets automatically from port groups to
> avoid a lot a trouble from client perspective [2].
>
> [1] https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/077118.html
> [2] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> February/046260.html
>
> Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
> Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> February/046166.html
> Signed-off-by: Han Zhou <hzh...@ebay.com>
> ---
>  NEWS|   2 +
>  include/ovn/expr.h  |  24 +---
>  include/ovn/lex.h   |   1 +
>  ovn/controller/lflow.c  |  16 +++--
>  ovn/controller/lflow.h  |   1 +
>  ovn/controller/ofctrl.c |   6 +-
>  ovn/controller/ofctrl.h |   3 +-
>  ovn/controller/ovn-controller.c |  31 +++---
>  ovn/lib/actions.c   |   3 +-
>  ovn/lib/expr.c  | 127 --
> --
>  ovn/lib/lex.c   |  20 +++
>  ovn/northd/ovn-northd.c |  47 +++
>  ovn/ovn-nb.ovsschema|  15 -
>  ovn/ovn-nb.xml  |  30 ++
>  ovn/ovn-sb.ovsschema|  10 +++-
>  ovn/ovn-sb.xml  |  20 +++
>  ovn/utilities/ovn-trace.c   |  27 +++--
>  tests/ovn.at|  34 +++
>  tests/test-ovn.c|  43 ++
>  19 files changed, 381 insertions(+), 79 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 4230189..da2c3ec 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -14,6 +14,8 @@ Post-v2.9.0
>   * OFPT_ROLE_STATUS is now available in OpenFlow 1.3.
> - Linux kernel 4.14
>   * Add support for compiling OVS with the latest Linux 4.14 kernel
> +   - OVN:
> + * Port_Group is supported in ACL match conditions.
>
>  v2.9.0 - 19 Feb 2018
>  
> diff --git a/include/ovn/expr.h b/include/ovn/expr.h
> index 711713e..3995e62 100644
> --- a/include/ovn/expr.h
> +++ b/include/ovn/expr.h
> @@ -382,9 +382,11 @@ expr_from_node(const struct ovs_list *node)
>  void expr_format(const struct expr *, struct ds *);
>  void expr_print(const struct expr *);
>  struct expr *expr_parse(struct lexer *, const struct shash *symtab,
> -const struct shash *addr_sets);
> +const struct shash *addr_sets,
> +const struct shash *port_groups);
>  struct expr *expr_parse_string(const char *, const struct shash *symtab,
> const struct shash *addr_sets,
> +   const struct shash *port_groups,
> char **errorp);
>
>  struct expr *expr_clone(struct expr *);
> @@ -404,6 +406,7 @@ bool expr_is_normalized(const struct expr *);
>
>  char *expr_parse_microflow(const char *, const struct shash *symtab,
> const struct shash *addr_sets,
> +   const struct shash *port_groups,
> bool (*lookup_port)(const void *aux,
> const char *port_name,
> unsigned int *portp),
> @@ -486,19 +489,22 @@ void expr_constant_set_format(const struct
> expr_constant_set *, struct ds *);
>  void expr_constant_set_destroy(struct expr_constant_set *cs);
>
>
> -/* Address sets.
> +/* Constant sets.
>   *
> - * Instead of referring to a set of value as:

Re: [ovs-dev] [PATCH 2/2] ovn: Support address sets generated from port groups

[Daniel Alvarez Sanchez]

ort`
> +for outport; do
> +out_ls=`vif_to_ls $outport`
> +if test $in_ls = $out_ls; then
> +# Ports on the same logical switch receive exactly the same
> packet.
> +echo $packet
> +else
> +# Routing decrements TTL and updates source and dest MAC
> +# (and checksum).
> +out_lrp=`vif_to_lrp $outport`
> +echo f${outport}ff$
> {out_lrp}0800451c"3f1101"00${src_ip}${dst_ip}00350008
> +fi >> $outport.expected
> +done
> +}
> +
> +as hv1 ovs-vsctl --columns=name,ofport list interface
> +as hv1 ovn-sbctl list port_binding
> +as hv1 ovn-sbctl list datapath_binding
> +as hv1 ovn-sbctl list port_group
> +as hv1 ovn-sbctl list address_set
> +as hv1 ovn-sbctl dump-flows
> +as hv1 ovs-ofctl dump-flows br-int
> +
> +# Send IP packets between all pairs of source and destination ports,
> +# packets matches ACL (pg2 to pg1) should be dropped
> +ip_to_hex() {
> +printf "%02x%02x%02x%02x" "$@"
> +}
> +for is in 1 2 3; do
> +  for js in 1 2 3; do
> +for ks in 1 2 3; do
> +  bcast=
> +  s=$is$js$ks
> +  smac=f$s
> +  sip=`ip_to_hex 192 168 $is$js $ks`
> +  for id in 1 2 3; do
> +  for jd in 1 2 3; do
> +  for kd in 1 2 3; do
> +d=$id$jd$kd
> +dip=`ip_to_hex 192 168 $id$jd $kd`
> +if test $is = $id; then dmac=f$d; else
> dmac=ff$is$js; fi
> +if test $d != $s; then unicast=$d; else unicast=; fi
> +
> +# packets matches ACL should be dropped
> +if test $id != 3 && test $kd == 1; then
> +if test $is != 1 && test $ks == 2; then
> +unicast=
> +fi
> +fi
> +test_ip $s $smac $dmac $sip $dip $unicast #1
> +  done
> +  done
> +done
> +  done
> +  done
> +done
> +
> +# Allow some time for packet forwarding.
> +# XXX This can be improved.
> +sleep 1
> +
> +# Now check the packets actually received against the ones expected.
> +for i in 1 2 3; do
> +for j in 1 2 3; do
> +for k in 1 2 3; do
> +OVN_CHECK_PACKETS([hv`vif_to_hv $i$j$k`/vif$i$j$k-tx.pcap],
> +  [$i$j$k.expected])
> +done
> +done
> +done
> +
> +# Gracefully terminate daemons
> +OVN_CLEANUP([hv1], [hv2], [hv3])
> +AT_CLEANUP
> --
> 2.1.0
>

Tested-By: Daniel Alvarez <dalva...@redhat.com>

>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] python: avoid useless JSON conversion to enhance performance

[Daniel Alvarez]

Thanks!
Would it be also possible to get it in in 2.9 branch as well please?

> On 28 Feb 2018, at 22:09, Ben Pfaff <b...@ovn.org> wrote:
> 
>> On Wed, Feb 28, 2018 at 10:11:09AM +0100, Daniel Alvarez wrote:
>> This patch removes a useless conversion to/from JSON in the
>> processing of any 'modify' operations inside the process_update2
>> method in Python IDL implementation.
>> 
>> Previous code will make resources creation take longer as the number
>> of elements in the row grows because of that JSON conversion. This
>> patch eliminates it and now the time remains consant regardless
>> of the database contents improving performance and scaling.
>> 
>> Reported-by: Daniel Alvarez <dalva...@redhat.com>
>> Reported-at: 
>> https://mail.openvswitch.org/pipermail/ovs-discuss/2018-February/046263.html
>> Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
>> Acked-by: Terry Wilson <twil...@redhat.com>
>> Acked-by: Han Zhou <hzh...@ebay.com>
> 
> Thanks!  I applied this to master.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] OVN python IDL: avoid useless JSON conversion to enhance performance

[Daniel Alvarez Sanchez]

Thanks Terry and Han for the reviews!
I removed the 'OVN' keyword from the title and submitted a v2:
[PATCH v2] python: avoid useless JSON conversion to enhance performance

Thanks again.
Daniel

On Wed, Feb 28, 2018 at 12:41 AM, Han Zhou <zhou...@gmail.com> wrote:

> Great catch!
> This patch is generic and not only for OVN, so I suggest to remove the
> "OVN" keyword in commit title.
>
> Acked-by: Han Zhou <hzh...@ebay.com>
>
> On Tue, Feb 27, 2018 at 12:44 PM, Terry Wilson <twil...@redhat.com> wrote:
>
>> On Tue, Feb 27, 2018 at 9:25 AM, Daniel Alvarez <dalva...@redhat.com>
>> wrote:
>> > This patch removes a useless conversion to/from JSON in the
>> > processing of any 'modify' operations inside the process_update2
>> > method in Python IDL implementation.
>> >
>> > Previous code will make resources creation take longer as the number
>> > of elements in the row grows because of that JSON conversion. This
>> > patch eliminates it and now the time remains consant regardless
>> > of the database contents improving performance and scaling.
>> >
>> > Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
>> > Reported-at: https://mail.openvswitch.org/p
>> ipermail/ovs-discuss/2018-February/046263.html
>> > Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
>> > ---
>> >  python/ovs/db/idl.py | 12 +---
>> >  1 file changed, 5 insertions(+), 7 deletions(-)
>> >
>> > diff --git a/python/ovs/db/idl.py b/python/ovs/db/idl.py
>> > index 60548bcf5..5a4d129c0 100644
>> > --- a/python/ovs/db/idl.py
>> > +++ b/python/ovs/db/idl.py
>> > @@ -518,10 +518,8 @@ class Idl(object):
>> >  if not row:
>> >  raise error.Error('Modify non-existing row')
>> >
>> > -old_row_diff_json = self.__apply_diff(table, row,
>> > -  row_update['modify'])
>> > -self.notify(ROW_UPDATE, row,
>> > -Row.from_json(self, table, uuid,
>> old_row_diff_json))
>> > +old_row = self.__apply_diff(table, row,
>> row_update['modify'])
>> > +self.notify(ROW_UPDATE, row, Row(self, table, uuid,
>> old_row))
>> >  changed = True
>> >  else:
>> >  raise error.Error(' unknown operation',
>> > @@ -584,7 +582,7 @@ class Idl(object):
>> >  row_update[column.name] =
>> self.__column_name(column)
>> >
>> >  def __apply_diff(self, table, row, row_diff):
>> > -old_row_diff_json = {}
>> > +old_row = {}
>> >  for column_name, datum_diff_json in six.iteritems(row_diff):
>> >  column = table.columns.get(column_name)
>> >  if not column:
>> > @@ -601,12 +599,12 @@ class Idl(object):
>> >% (column_name, table.name, e))
>> >  continue
>> >
>> > -old_row_diff_json[column_name] =
>> row._data[column_name].to_json()
>> > +old_row[column_name] = row._data[column_name].copy()
>> >  datum = row._data[column_name].diff(datum_diff)
>> >  if datum != row._data[column_name]:
>> >  row._data[column_name] = datum
>> >
>> > -return old_row_diff_json
>> > +return old_row
>> >
>> >  def __row_update(self, table, row, row_json):
>> >  changed = False
>> > --
>> > 2.13.5
>> >
>> > ___
>> > dev mailing list
>> > d...@openvswitch.org
>> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>
>> +1
>>
>> This passes all of the functional tests in ovsdbapp when applied. Nice
>> find!
>>
>> Terry
>> ___
>> dev mailing list
>> d...@openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2] python: avoid useless JSON conversion to enhance performance

[Daniel Alvarez]

This patch removes a useless conversion to/from JSON in the
processing of any 'modify' operations inside the process_update2
method in Python IDL implementation.

Previous code will make resources creation take longer as the number
of elements in the row grows because of that JSON conversion. This
patch eliminates it and now the time remains consant regardless
of the database contents improving performance and scaling.

Reported-by: Daniel Alvarez <dalva...@redhat.com>
Reported-at: 
https://mail.openvswitch.org/pipermail/ovs-discuss/2018-February/046263.html
Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
Acked-by: Terry Wilson <twil...@redhat.com>
Acked-by: Han Zhou <hzh...@ebay.com>
---
 python/ovs/db/idl.py | 12 +---
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/python/ovs/db/idl.py b/python/ovs/db/idl.py
index 60548bcf5..5a4d129c0 100644
--- a/python/ovs/db/idl.py
+++ b/python/ovs/db/idl.py
@@ -518,10 +518,8 @@ class Idl(object):
 if not row:
 raise error.Error('Modify non-existing row')
 
-old_row_diff_json = self.__apply_diff(table, row,
-  row_update['modify'])
-self.notify(ROW_UPDATE, row,
-Row.from_json(self, table, uuid, old_row_diff_json))
+old_row = self.__apply_diff(table, row, row_update['modify'])
+self.notify(ROW_UPDATE, row, Row(self, table, uuid, old_row))
 changed = True
 else:
 raise error.Error(' unknown operation',
@@ -584,7 +582,7 @@ class Idl(object):
 row_update[column.name] = self.__column_name(column)
 
 def __apply_diff(self, table, row, row_diff):
-old_row_diff_json = {}
+old_row = {}
 for column_name, datum_diff_json in six.iteritems(row_diff):
 column = table.columns.get(column_name)
 if not column:
@@ -601,12 +599,12 @@ class Idl(object):
   % (column_name, table.name, e))
 continue
 
-old_row_diff_json[column_name] = row._data[column_name].to_json()
+old_row[column_name] = row._data[column_name].copy()
 datum = row._data[column_name].diff(datum_diff)
 if datum != row._data[column_name]:
 row._data[column_name] = datum
 
-return old_row_diff_json
+return old_row
 
 def __row_update(self, table, row, row_json):
 changed = False
-- 
2.13.5

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] OVN python IDL: avoid useless JSON conversion to enhance performance

[Daniel Alvarez]

This patch removes a useless conversion to/from JSON in the
processing of any 'modify' operations inside the process_update2
method in Python IDL implementation.

Previous code will make resources creation take longer as the number
of elements in the row grows because of that JSON conversion. This
patch eliminates it and now the time remains consant regardless
of the database contents improving performance and scaling.

Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
Reported-at: 
https://mail.openvswitch.org/pipermail/ovs-discuss/2018-February/046263.html
Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---
 python/ovs/db/idl.py | 12 +---
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/python/ovs/db/idl.py b/python/ovs/db/idl.py
index 60548bcf5..5a4d129c0 100644
--- a/python/ovs/db/idl.py
+++ b/python/ovs/db/idl.py
@@ -518,10 +518,8 @@ class Idl(object):
 if not row:
 raise error.Error('Modify non-existing row')
 
-old_row_diff_json = self.__apply_diff(table, row,
-  row_update['modify'])
-self.notify(ROW_UPDATE, row,
-Row.from_json(self, table, uuid, old_row_diff_json))
+old_row = self.__apply_diff(table, row, row_update['modify'])
+self.notify(ROW_UPDATE, row, Row(self, table, uuid, old_row))
 changed = True
 else:
 raise error.Error(' unknown operation',
@@ -584,7 +582,7 @@ class Idl(object):
 row_update[column.name] = self.__column_name(column)
 
 def __apply_diff(self, table, row, row_diff):
-old_row_diff_json = {}
+old_row = {}
 for column_name, datum_diff_json in six.iteritems(row_diff):
 column = table.columns.get(column_name)
 if not column:
@@ -601,12 +599,12 @@ class Idl(object):
   % (column_name, table.name, e))
 continue
 
-old_row_diff_json[column_name] = row._data[column_name].to_json()
+old_row[column_name] = row._data[column_name].copy()
 datum = row._data[column_name].diff(datum_diff)
 if datum != row._data[column_name]:
 row._data[column_name] = datum
 
-return old_row_diff_json
+return old_row
 
 def __row_update(self, table, row, row_json):
 changed = False
-- 
2.13.5

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] util: Use lookup table to optimize hexit_value().

[Daniel Alvarez Sanchez]

Thanks for the patch, Ben. Looks good to me.

Acked-by: Daniel Alvarez <dalva...@redhat.com>

On Sat, Feb 3, 2018 at 12:16 AM, Ben Pfaff <b...@ovn.org> wrote:

> Daniel Alvarez Sanchez reported a significant overall speedup in ovn-northd
> due to a similar patch.
>
> Reported-by: Daniel Alvarez Sanchez <dalva...@redhat.com>
> Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-
> February/046120.html
> Signed-off-by: Ben Pfaff <b...@ovn.org>
> ---
>  lib/util.c | 39 +--
>  1 file changed, 13 insertions(+), 26 deletions(-)
>
> diff --git a/lib/util.c b/lib/util.c
> index a4d22df0c359..2e7765e41f03 100644
> --- a/lib/util.c
> +++ b/lib/util.c
> @@ -843,32 +843,19 @@ str_to_double(const char *s, double *d)
>  int
>  hexit_value(int c)
>  {
> -switch (c) {
> -case '0': case '1': case '2': case '3': case '4':
> -case '5': case '6': case '7': case '8': case '9':
> -return c - '0';
> -
> -case 'a': case 'A':
> -return 0xa;
> -
> -case 'b': case 'B':
> -return 0xb;
> -
> -case 'c': case 'C':
> -return 0xc;
> -
> -case 'd': case 'D':
> -return 0xd;
> -
> -case 'e': case 'E':
> -return 0xe;
> -
> -case 'f': case 'F':
> -return 0xf;
> -
> -default:
> -return -1;
> -}
> +static const signed char tbl[UCHAR_MAX + 1] = {
> +#define TBL(x)  \
> +(  x >= '0' && x <= '9' ? x - '0'   \
> + : x >= 'a' && x <= 'f' ? x - 'a' + 0xa \
> + : x >= 'A' && x <= 'F' ? x - 'A' + 0xa \
> + : -1)
> +#define TBL0(x)  TBL(x),  TBL((x) + 1),   TBL((x) + 2),   TBL((x) + 3)
> +#define TBL1(x) TBL0(x), TBL0((x) + 4),  TBL0((x) + 8),  TBL0((x) + 12)
> +#define TBL2(x) TBL1(x), TBL1((x) + 16), TBL1((x) + 32), TBL1((x) + 48)
> +TBL2(0), TBL2(64), TBL2(128), TBL2(192)
> +};
> +
> +return tbl[c];
>  }
>
>  /* Returns the integer value of the 'n' hexadecimal digits starting at
> 's', or
> --
> 2.15.1
>
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2] ovn-controller: add new external_id 'ovn-cms-options' to Chassis table

[Daniel Alvarez]

This patch makes ovn-controller sets the external_ids key
'ovn-cms-options' to its own Chassis table entry copying its
contents from the same external_ids key in the local OpenvSwitch
database.

The idea behind this patch is to allow setting general options
from the CMS Plugin to a particular chassis.

A good example of an use case is when we want to schedule a router
on a chassis from OpenStack. In this case, we may want to exclude
some nodes because they are more likely to be restarted for
maintenance operations or they simply won't have external connectivity.
This way, if the CMS/deployment tool would set the external_ids
as:

ovs-vsctl set open . external_ids:ovn-cms-options="enable-chassis-as-gw"

Then ovn-controller will write the options to the Chassis table in
southbound database. This value can be later read by the CMS in order
to decide which Chassis are eligible to schedule a router on.

Similarly, this new key would allow to specify additional options to
be consumed by the CMS.

Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---
 ovn/controller/chassis.c| 13 -
 ovn/controller/ovn-controller.8.xml |  7 +++
 ovn/ovn-sb.xml  |  8 
 3 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/ovn/controller/chassis.c b/ovn/controller/chassis.c
index 521b04c..6b5286a 100644
--- a/ovn/controller/chassis.c
+++ b/ovn/controller/chassis.c
@@ -66,6 +66,12 @@ get_bridge_mappings(const struct smap *ext_ids)
 return smap_get_def(ext_ids, "ovn-bridge-mappings", "");
 }
 
+static const char *
+get_cms_options(const struct smap *ext_ids)
+{
+return smap_get_def(ext_ids, "ovn-cms-options", "");
+}
+
 /* Returns this chassis's Chassis record, if it is available and is currently
  * amenable to a transaction. */
 const struct sbrec_chassis *
@@ -119,6 +125,7 @@ chassis_run(struct controller_ctx *ctx, const char 
*chassis_id,
 const char *bridge_mappings = get_bridge_mappings(>external_ids);
 const char *datapath_type =
 br_int && br_int->datapath_type ? br_int->datapath_type : "";
+const char *cms_options = get_cms_options(>external_ids);
 
 struct ds iface_types = DS_EMPTY_INITIALIZER;
 ds_put_cstr(_types, "");
@@ -144,16 +151,20 @@ chassis_run(struct controller_ctx *ctx, const char 
*chassis_id,
 = smap_get_def(_rec->external_ids, "datapath-type", "");
 const char *chassis_iface_types
 = smap_get_def(_rec->external_ids, "iface-types", "");
+const char *chassis_cms_options
+= get_cms_options(_rec->external_ids);
 
 /* If any of the external-ids should change, update them. */
 if (strcmp(bridge_mappings, chassis_bridge_mappings) ||
 strcmp(datapath_type, chassis_datapath_type) ||
-strcmp(iface_types_str, chassis_iface_types)) {
+strcmp(iface_types_str, chassis_iface_types) ||
+strcmp(cms_options, chassis_cms_options)) {
 struct smap new_ids;
 smap_clone(_ids, _rec->external_ids);
 smap_replace(_ids, "ovn-bridge-mappings", bridge_mappings);
 smap_replace(_ids, "datapath-type", datapath_type);
 smap_replace(_ids, "iface-types", iface_types_str);
+smap_replace(_ids, "ovn-cms-options", cms_options);
 sbrec_chassis_verify_external_ids(chassis_rec);
 sbrec_chassis_set_external_ids(chassis_rec, _ids);
 smap_destroy(_ids);
diff --git a/ovn/controller/ovn-controller.8.xml 
b/ovn/controller/ovn-controller.8.xml
index 2af3db5..0df59ac 100644
--- a/ovn/controller/ovn-controller.8.xml
+++ b/ovn/controller/ovn-controller.8.xml
@@ -150,6 +150,13 @@
 network interface card, enabling encapsulation checksum may incur
 performance loss. In such cases, encapsulation checksums can be 
disabled.
   
+
+  external_ids:ovn-cms-options
+  
+A list of options that will be consumed by the CMS Plugin and which
+specific to this particular chassis. An example would be:
+cms_option1,cms_option2:foo.
+  
 
 
 
diff --git a/ovn/ovn-sb.xml b/ovn/ovn-sb.xml
index abc241e..4a75135 100644
--- a/ovn/ovn-sb.xml
+++ b/ovn/ovn-sb.xml
@@ -241,6 +241,14 @@
   read-only. See ovn-controller(8) for more information.
 
 
+
+  ovn-controller populates this key with the set of options
+  configured in the  column of the Open_vSwitch
+  database's  table.
+  See ovn-controller(8) for more information.
+
+
 
   The overall purpose of these columns is described under Common
   Columns at the beginning of this document.
-- 
1.8.3.1

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] ovn-controller: add new external_id 'ovn-cms-options' to Chassis table

[Daniel Alvarez]

This patch makes ovn-controller sets the external_ids key
'ovn-cms-options' to its own Chassis table entry copying its
contents from the same external_ids key in the local OpenvSwitch
database.

The idea behind this patch is to allow setting general options
from the CMS Plugin to a particular chassis.

A good example of an use case is when we want to schedule a router
on a chassis from OpenStack. In this case, we may want to exclude
some nodes because they are more likely to be restarted for
maintenance operations or they simply won't have external connectivity.
This way, if the CMS/deployment tool would set the external_ids
as:

ovs-vsctl set open . external_ids:ovn-cms-options="enable-chassis-as-gw"

Then ovn-controller will write the options to the Chassis table in
southbound database. This value can be later read by the CMS in order
to decide which Chassis are eligible to schedule a router on.

Similarly, this new key would allow to specify additional options to
be consumed by the CMS.
---
 ovn/controller/chassis.c| 13 -
 ovn/controller/ovn-controller.8.xml |  7 +++
 ovn/ovn-sb.xml  |  8 
 3 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/ovn/controller/chassis.c b/ovn/controller/chassis.c
index 521b04c..6b5286a 100644
--- a/ovn/controller/chassis.c
+++ b/ovn/controller/chassis.c
@@ -66,6 +66,12 @@ get_bridge_mappings(const struct smap *ext_ids)
 return smap_get_def(ext_ids, "ovn-bridge-mappings", "");
 }
 
+static const char *
+get_cms_options(const struct smap *ext_ids)
+{
+return smap_get_def(ext_ids, "ovn-cms-options", "");
+}
+
 /* Returns this chassis's Chassis record, if it is available and is currently
  * amenable to a transaction. */
 const struct sbrec_chassis *
@@ -119,6 +125,7 @@ chassis_run(struct controller_ctx *ctx, const char 
*chassis_id,
 const char *bridge_mappings = get_bridge_mappings(>external_ids);
 const char *datapath_type =
 br_int && br_int->datapath_type ? br_int->datapath_type : "";
+const char *cms_options = get_cms_options(>external_ids);
 
 struct ds iface_types = DS_EMPTY_INITIALIZER;
 ds_put_cstr(_types, "");
@@ -144,16 +151,20 @@ chassis_run(struct controller_ctx *ctx, const char 
*chassis_id,
 = smap_get_def(_rec->external_ids, "datapath-type", "");
 const char *chassis_iface_types
 = smap_get_def(_rec->external_ids, "iface-types", "");
+const char *chassis_cms_options
+= get_cms_options(_rec->external_ids);
 
 /* If any of the external-ids should change, update them. */
 if (strcmp(bridge_mappings, chassis_bridge_mappings) ||
 strcmp(datapath_type, chassis_datapath_type) ||
-strcmp(iface_types_str, chassis_iface_types)) {
+strcmp(iface_types_str, chassis_iface_types) ||
+strcmp(cms_options, chassis_cms_options)) {
 struct smap new_ids;
 smap_clone(_ids, _rec->external_ids);
 smap_replace(_ids, "ovn-bridge-mappings", bridge_mappings);
 smap_replace(_ids, "datapath-type", datapath_type);
 smap_replace(_ids, "iface-types", iface_types_str);
+smap_replace(_ids, "ovn-cms-options", cms_options);
 sbrec_chassis_verify_external_ids(chassis_rec);
 sbrec_chassis_set_external_ids(chassis_rec, _ids);
 smap_destroy(_ids);
diff --git a/ovn/controller/ovn-controller.8.xml 
b/ovn/controller/ovn-controller.8.xml
index 2af3db5..0df59ac 100644
--- a/ovn/controller/ovn-controller.8.xml
+++ b/ovn/controller/ovn-controller.8.xml
@@ -150,6 +150,13 @@
 network interface card, enabling encapsulation checksum may incur
 performance loss. In such cases, encapsulation checksums can be 
disabled.
   
+
+  external_ids:ovn-cms-options
+  
+A list of options that will be consumed by the CMS Plugin and which
+specific to this particular chassis. An example would be:
+cms_option1,cms_option2:foo.
+  
 
 
 
diff --git a/ovn/ovn-sb.xml b/ovn/ovn-sb.xml
index abc241e..4a75135 100644
--- a/ovn/ovn-sb.xml
+++ b/ovn/ovn-sb.xml
@@ -241,6 +241,14 @@
   read-only. See ovn-controller(8) for more information.
 
 
+
+  ovn-controller populates this key with the set of options
+  configured in the  column of the Open_vSwitch
+  database's  table.
+  See ovn-controller(8) for more information.
+
+
 
   The overall purpose of these columns is described under Common
   Columns at the beginning of this document.
-- 
1.8.3.1

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH] ovn-controller: Change duplicate flows trace from INFO to DBG

[Daniel Alvarez]

When ovn-controller detects that a flow is duplicated, it will print
an INFO trace. Even though it's rate limited, this patch is changing
the trace level back to DEBUG to reduce noise.

Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---
In our effort to ensure consistency across objects in Neutron and OVN
databases we find some special cases like security group rules which
match OVN ACLs but not in 1:1 relationship. Until now, two identical
security group rules beloning each to a different security group would
generate a single ACL in NB database. With this behavior, there's no
way to map the ACL in OVN to the corresponding Neutron object.

By implementing [0] we're trying to ensure this mapping so we make use
of the external_ids column of every table for this purpose. It may happen
that we'll have two identical ACLs but each referencing a different
Neutron object in their external_ids field. However, this will make
ovn-controller to drop those duplicated flows and log a (rate controlled)
INFO trace. In order to reduce this noise, this patch changes it to DBG.

[0] 
https://docs.openstack.org/networking-ovn/latest/contributor/design/database_consistency.html

ovn/controller/ofctrl.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ovn/controller/ofctrl.c b/ovn/controller/ofctrl.c
index 90afddbb3..2fa980ebd 100644
--- a/ovn/controller/ofctrl.c
+++ b/ovn/controller/ofctrl.c
@@ -620,9 +620,9 @@ ofctrl_add_flow(struct hmap *desired_flows,

 if (ovn_flow_lookup(desired_flows, f)) {
 static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 5);
-if (!VLOG_DROP_INFO()) {
+if (!VLOG_DROP_DBG()) {
 char *s = ovn_flow_to_string(f);
-VLOG_INFO("dropping duplicate flow: %s", s);
+VLOG_DBG("dropping duplicate flow: %s", s);
 free(s);
 }

--
2.13.5

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] ovn-northd: Avoid duplicate logical flows in SB db

[Daniel Alvarez Sanchez]

gt; > error
> > > log?
> > >
> > > Another point, would this type of check increase the difficulty of
> > probably
> > > future incremental-processing in northd?
> > >
> > > From my point of view, it might be better just keep northd simple, and
> > let
> > > clients handle the correctness, and let ovn-controller to do the final
> > > check. In this case, could Neutron maintain multiple sg-rule ids in
> > > external-ids of the same ACL entry?
> > >
> > > Thanks,
> > > Han
> > >
> > > On Mon, Jan 8, 2018 at 8:52 AM, Miguel Angel Ajo Pelayo <
> > majop...@redhat.com
> > > > wrote:
> > >
> > > > Right!
> > > >
> > > > We didn't hit that issue, but it'd make sense to fix in this patch I
> > guess.
> > > >
> > > > We could modify the hashing function to not include the action (not
> > sure if
> > > > it does now..),
> > > > and also have a separate search function that ignores the action.
> > > >
> > > > On Mon, Jan 8, 2018 at 5:39 PM Ben Pfaff <b...@ovn.org> wrote:
> > > >
> > > > > I suspect that this doesn't go far enough, because it includes
> > actions
> > > > > in the hash, so that it would fail to deduplicate two identical
> ACLs
> > > > > with different actions (e.g. "drop" versus "allow").
> > > > >
> > > > > On Fri, Jan 05, 2018 at 10:43:16AM +, Daniel Alvarez wrote:
> > > > > > When there are two ACLs in a Logical Switch with same direction,
> > > > > > priority, match and action fields, ovn-northd will generate the
> > > > > > exact same logical flow for them into SB database. This will make
> > > > > > ovn-controller log messages (INFO) saying that the duplicate flow
> > > > > > is going to be dropped.
> > > > > >
> > > > > > This patch avoids adding duplicate lflows into SB database so
> that
> > > > > > ovn-controller doesn't have to process them.
> > > > > >
> > > > > > Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
> > > > > > ---
> > > > > >
> > > > > > This patch is needed as part of the consistency work we're doing
> > in the
> > > > > > OpenStack integration [0]. In our effort to ensure consistency
> > across
> > > > > > objects in Neutron and OVN databases we find some special cases
> > like
> > > > > > security group rules which match OVN ACLs but not in 1:1
> > relationship.
> > > > > > Until now, two identical security group rules beloning each to a
> > > > > > different security group would generate a single ACL in NB
> > database.
> > > > > > With this behavior, there's no way to map the ACL in OVN to the
> > > > > > corresponding Neutron object.
> > > > > >
> > > > > > By implementing [0] we're trying to ensure this mapping so we
> make
> > use
> > > > > > of the external_ids column of every table for this purpose. It
> may
> > > > happen
> > > > > > that we'll have two identical ACLs but each referencing a
> different
> > > > > > Neutron object in their external_ids field. However, this will
> make
> > > > > > ovn-northd to generate two duplicate lflows into SB database
> which
> > will
> > > > > > make ovn-controller drop them when installing the actual flows.
> > With
> > > > this
> > > > > > patch we'll avoid duplicate flows to be inserted in SB database
> in
> > such
> > > > > > cases.
> > > > > >
> > > > > > [0]
> > > > > https://docs.openstack.org/networking-ovn/latest/
> > > > contributor/design/database_consistency.html
> > > > > >
> > > > > >  ovn/northd/ovn-northd.c | 11 +++
> > > > > >  tests/ovn-northd.at | 24 
> > > > > >  2 files changed, 35 insertions(+)
> > > > > >
> > > > > > diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
> > > > > > index 7e6b1d9..cc64861 100644
> > > > > > --- a/ovn/northd/ovn-northd.c
> > > > > > +++ b/ovn/northd/ovn-northd.c
> > > > > > 

[ovs-dev] [PATCH] ovn-northd: Avoid duplicate logical flows in SB db

[Daniel Alvarez]

When there are two ACLs in a Logical Switch with same direction,
priority, match and action fields, ovn-northd will generate the
exact same logical flow for them into SB database. This will make
ovn-controller log messages (INFO) saying that the duplicate flow
is going to be dropped.

This patch avoids adding duplicate lflows into SB database so that
ovn-controller doesn't have to process them.

Signed-off-by: Daniel Alvarez <dalva...@redhat.com>
---

This patch is needed as part of the consistency work we're doing in the
OpenStack integration [0]. In our effort to ensure consistency across
objects in Neutron and OVN databases we find some special cases like
security group rules which match OVN ACLs but not in 1:1 relationship.
Until now, two identical security group rules beloning each to a
different security group would generate a single ACL in NB database.
With this behavior, there's no way to map the ACL in OVN to the
corresponding Neutron object.

By implementing [0] we're trying to ensure this mapping so we make use
of the external_ids column of every table for this purpose. It may happen
that we'll have two identical ACLs but each referencing a different
Neutron object in their external_ids field. However, this will make
ovn-northd to generate two duplicate lflows into SB database which will
make ovn-controller drop them when installing the actual flows. With this
patch we'll avoid duplicate flows to be inserted in SB database in such
cases.

[0] 
https://docs.openstack.org/networking-ovn/latest/contributor/design/database_consistency.html

 ovn/northd/ovn-northd.c | 11 +++
 tests/ovn-northd.at | 24 
 2 files changed, 35 insertions(+)

diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c
index 7e6b1d9..cc64861 100644
--- a/ovn/northd/ovn-northd.c
+++ b/ovn/northd/ovn-northd.c
@@ -428,6 +428,13 @@ struct macam_node {
 struct eth_addr mac_addr; /* Allocated MAC address. */
 };
 
+static struct ovn_lflow *ovn_lflow_find(struct hmap *lflows,
+struct ovn_datapath *od,
+enum ovn_stage stage,
+uint16_t priority,
+const char *match,
+const char *actions);
+
 static void
 cleanup_macam(struct hmap *macam)
 {
@@ -2298,6 +2305,10 @@ ovn_lflow_add_at(struct hmap *lflow_map, struct 
ovn_datapath *od,
  const char *stage_hint, const char *where)
 {
 ovs_assert(ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od));
+   
+if (ovn_lflow_find(lflow_map, od, stage, priority, match, actions)) {
+return;
+}
 
 struct ovn_lflow *lflow = xmalloc(sizeof *lflow);
 ovn_lflow_init(lflow, od, stage, priority,
diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
index 954e259..ba96c81 100644
--- a/tests/ovn-northd.at
+++ b/tests/ovn-northd.at
@@ -152,3 +152,27 @@ ovn-nbctl lsp-set-options S1-R1 router-port=R1-S1
 AT_CHECK([test x`ovn-nbctl lsp-get-up S1-R1` = xup])
 
 AT_CLEANUP
+
+AT_SETUP([ovn -- check that duplicate acls don't generate duplicate lflows])
+AT_SKIP_IF([test $HAVE_PYTHON = no])
+ovn_start
+
+ovn-nbctl ls-add S1
+
+# Insert a duplicate ACL into NB database.
+ovn-nbctl -- --id=@acl create acl direction=to-lport priority=1000 \
+match='"tcp.dst == 22"' action=drop -- add logical_switch S1 acl @acl
+ovn-nbctl -- --id=@acl create acl direction=to-lport priority=1000 \
+match='"tcp.dst == 22"' action=drop -- add logical_switch S1 acl @acl
+
+# Check that there are two entries in ACL table in NB database.
+AT_CHECK([ovn-nbctl find ACL match='"tcp.dst == 22"' | \
+grep _uuid | wc -l], [0], [2
+])
+
+# Now make sure that only one logical flow is added to SB database.
+AT_CHECK([ovn-sbctl find Logical_Flow match='"tcp.dst == 22"' | \
+grep _uuid | wc -l], [0], [1
+])
+
+AT_CLEANUP
-- 
1.8.3.1

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev