Hi:
Have sent out V4 @ https://patchwork.ozlabs.org/patch/924734/ . Please take
a look and let me know further.
Regards,
Aliasgar
On Thu, May 31, 2018 at 9:12 AM, aginwala <aginw...@asu.edu> wrote:
>
>
> On Wed, May 30, 2018 at 11:24 PM, Numan Siddique <nusid...@redhat.com>
> wrote:
>
>>
>>
>> On Thu, May 31, 2018 at 4:06 AM, Han Zhou <zhou...@gmail.com> wrote:
>>
>>>
>>>
>>> On Wed, May 30, 2018 at 3:27 PM, aginwala <aginw...@asu.edu> wrote:
>>>
>>>>
>>>>
>>>> On Wed, May 30, 2018 at 2:52 PM, Han Zhou <zhou...@gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, May 29, 2018 at 11:40 PM, Numan Siddique <nusid...@redhat.com>
>>>>> wrote:
>>>>> >
>>>>> > On Wed, May 30, 2018 at 5:31 AM, aginwala <aginw...@asu.edu> wrote:
>>>>> >
>>>>> > > Thanks for the Review.
>>>>> > >
>>>>> > > On Wed, May 23, 2018 at 11:36 AM, Numan Siddique <
>>>>> nusid...@redhat.com>
>>>>> > > wrote:
>>>>> > >
>>>>> > >> On Wed, May 9, 2018 at 12:13 AM, aginwala <amgin...@gmail.com>
>>>>> wrote:
>>>>> > >>
>>>>> > >> > using pacemaker so that controllers can be placed in different
>>>>> fault
>>>>> > >> > domains.
>>>>> > >> >
>>>>> > >> > Signed-off-by: aginwala <aginw...@ebay.com>
>>>>> > >> >
>>>>> > >>
>>>>> > >> I see the below warning when applying the patch
>>>>> > >>
>>>>> > >> .git/rebase-apply/patch:153: new blank line at EOF.
>>>>> > >> +
>>>>> > >> warning: 1 line adds whitespace errors.
>>>>> > >>
>>>>> > > >>> Sure. Will fix.
>>>>> > >
>>>>> > >>
>>>>> > >> > ---
>>>>> > >> > ovn/utilities/ovndb-servers.ocf | 83
>>>>> ++++++++++++++++++++++++++++++
>>>>> > >>
>>>>> > >> > ++---------
>>>>> > >> > 1 file changed, 65 insertions(+), 18 deletions(-)
>>>>> > >> >
>>>>> > >> > diff --git a/ovn/utilities/ovndb-servers.ocf
>>>>> > >> > b/ovn/utilities/ovndb-servers.ocf
>>>>> > >> > index 164b6bc..1b4b6ab 100755
>>>>> > >> > --- a/ovn/utilities/ovndb-servers.ocf
>>>>> > >> > +++ b/ovn/utilities/ovndb-servers.ocf
>>>>> > >> > @@ -9,6 +9,8 @@
>>>>> > >> > : ${SB_MASTER_PROTO_DEFAULT="tcp"}
>>>>> > >> > : ${MANAGE_NORTHD_DEFAULT="no"}
>>>>> > >> > : ${INACTIVE_PROBE_DEFAULT="5000"}
>>>>> > >> > +: ${LISTEN_ON_MASTER_IP_ONLY_DEFAULT="yes"}
>>>>> > >> > +: ${LISTEN_ON_SLAVE_DEFAULT="yes"}
>>>>> > >> >
>>>>> > >> > CRM_MASTER="${HA_SBIN_DIR}/crm_master -l reboot"
>>>>> > >> > CRM_ATTR_REPL_INFO="${HA_SBIN_DIR}/crm_attribute --type
>>>>> crm_config
>>>>> > >> > --name OVN_REPL_INFO -s ovn_ovsdb_master_server"
>>>>> > >> > @@ -21,6 +23,14 @@ SB_MASTER_PROTO=${OCF_RESKEY_s
>>>>> > >> b_master_protocol:-${SB_
>>>>> > >> > MASTER_PROTO_DEFAULT}}
>>>>> > >> > MANAGE_NORTHD=${OCF_RESKEY_manage_northd:-${MANAGE_NORTHD_D
>>>>> EFAULT}}
>>>>> > >> > INACTIVE_PROBE=${OCF_RESKEY_inactive_probe_interval:-${
>>>>> > >> > INACTIVE_PROBE_DEFAULT}}
>>>>> > >> >
>>>>> > >> > +# In order for pacemaker to work with LB, we can keep
>>>>> > >> > LISTEN_ON_MASTER_IP_ONLY
>>>>> > >> > +# to false and pass LB vip IP while creating pcs resource.
>>>>> > >> > +LISTEN_ON_MASTER_IP_ONLY=${OCF_RESKEY_listen_on_master_
>>>>> > >> > ip_only:-${LISTEN_ON_MASTER_IP_ONLY_DEFAULT}}
>>>>> > >> > +
>>>>> > >> > +# In order for pacemaker to work with LB, we can also set
>>>>> > >> LISTEN_ON_SLAVE
>>>>> > >> > +# to false so that slaves do not listen on 0.0.0.0.
>>>>> > >> > +LISTEN_ON_SLAVE=${OCF_RESKEY_listen_on_slave:-${LISTEN_ON_S
>>>>> > >> LAVE_DEFAULT}}
>>>>> > >> >
>>>>> > >>
>>>>> > >>
>>>>> > >> You need to define these parameters in the <parameters> section
>>>>> here -
>>>>> > >> https://github.com/openvswitch/ovs/blob/master/ovn/
>>>>> > >> utilities/ovndb-servers.ocf#L118
>>>>> > >> Otherwise we will not be able to pass these options when starting
>>>>> the
>>>>> > >> resource. Ideally we want to do
>>>>> > >> "pcs resource create ovndb_servers ....
>>>>> listen_on_master_ip_only=no
>>>>> > >> listen_on_slave=no
>>>>> > >>
>>>>> > > >> Thanks for the pointer. I will update that and re-test.
>>>>> > >
>>>>> > >>
>>>>> > >>
>>>>> > >> +
>>>>> > >> > # Invalid IP address is an address that can never exist in the
>>>>> > >> network, as
>>>>> > >> > # mentioned in rfc-5737. The ovsdb servers connects to this IP
>>>>> address
>>>>> > >> > till
>>>>> > >> > # a master is promoted and the IPAddr2 resource is started.
>>>>> > >> > @@ -157,22 +167,24 @@ ovsdb_server_notify() {
>>>>> > >> > ${OVN_CTL} --ovn-manage-ovsdb=no start_northd
>>>>> > >> > fi
>>>>> > >> >
>>>>> > >> > - conn=`ovn-nbctl get NB_global . connections`
>>>>> > >> > - if [ "$conn" == "[]" ]
>>>>> > >> > - then
>>>>> > >> > - ovn-nbctl -- --id=@conn_uuid create Connection \
>>>>> > >> > + # TODO: Need to troubleshoot as to removing target is
>>>>> ok as
>>>>> > >> well.
>>>>> > >>
>>>>> > >> + if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xyes ]; then
>>>>> > >> >
>>>>> > >>
>>>>> > >> If LB VIP is used, you don't want to set the probe interval ?
>>>>> > >
>>>>> > > >>> I thought we can stick with common probe interval for any
>>>>> case. Hence,
>>>>> > > kept it.
>>>>> > >
>>>>> > >>
>>>>> > >> > + conn=`ovn-nbctl get NB_global . connections`
>>>>> > >> > + if [ "$conn" == "[]" ]
>>>>> > >> > + then
>>>>> > >> > + ovn-nbctl -- --id=@conn_uuid create Connection
>>>>> \
>>>>> > >> > target="p${NB_MASTER_PROTO}\:${NB_MASTER_PORT}\:${MASTER_IP}"
>>>>> \
>>>>> > >> > inactivity_probe=$INACTIVE_PROBE -- set NB_Global .
>>>>> > >> > connections=@conn_uuid
>>>>> > >> > - fi
>>>>> > >> > + fi
>>>>> > >> >
>>>>> > >> > - conn=`ovn-sbctl get SB_global . connections`
>>>>> > >> > - if [ "$conn" == "[]" ]
>>>>> > >> > - then
>>>>> > >> > - ovn-sbctl -- --id=@conn_uuid create Connection \
>>>>> > >> > + conn=`ovn-sbctl get SB_global . connections`
>>>>> > >> > + if [ "$conn" == "[]" ]
>>>>> > >> > + then
>>>>> > >> > + ovn-sbctl -- --id=@conn_uuid create Connection
>>>>> \
>>>>> > >> > target="p${SB_MASTER_PROTO}\:${SB_MASTER_PORT}\:${MASTER_IP}"
>>>>> \
>>>>> > >> > inactivity_probe=$INACTIVE_PROBE -- set SB_Global .
>>>>> > >> > connections=@conn_uuid
>>>>> > >> > + fi
>>>>> > >> > fi
>>>>> > >> > -
>>>>> > >> > else
>>>>> > >> > if [ "$MANAGE_NORTHD" = "yes" ]; then
>>>>> > >> > # Stop ovn-northd service. Set
>>>>> --ovn-manage-ovsdb=no so
>>>>> > >> that
>>>>> > >> > @@ -295,15 +307,15 @@ ovsdb_server_start() {
>>>>> > >> >
>>>>> > >> > set ${OVN_CTL}
>>>>> > >> >
>>>>> > >> > - set $@ --db-nb-addr=${MASTER_IP}
>>>>> --db-nb-port=${NB_MASTER_PORT}
>>>>> > >> > - set $@ --db-sb-addr=${MASTER_IP}
>>>>> --db-sb-port=${SB_MASTER_PORT}
>>>>> > >> > + # For LB vip to talk to master pool member on a specific
>>>>> tcp port,
>>>>> > >> we
>>>>> > >> > need
>>>>> > >> > + # to listen on 0.0.0.0.instead of master_ip
>>>>> > >> > + if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then
>>>>> > >> > + set $@ --db-nb-port=${NB_MASTER_PORT}
>>>>> > >> > + set $@ --db-sb-port=${SB_MASTER_PORT}
>>>>> > >> >
>>>>> > >> > - if [ "x${NB_MASTER_PROTO}" = xtcp ]; then
>>>>> > >> > - set $@ --db-nb-create-insecure-remote=yes
>>>>> > >> > - fi
>>>>> > >> > -
>>>>> > >> > - if [ "x${SB_MASTER_PROTO}" = xtcp ]; then
>>>>> > >> > - set $@ --db-sb-create-insecure-remote=yes
>>>>> > >> > + else
>>>>> > >> > + set $@ --db-nb-addr=${MASTER_IP}
>>>>> --db-nb-port=${NB_MASTER_PORT}
>>>>> > >> > + set $@ --db-sb-addr=${MASTER_IP}
>>>>> --db-sb-port=${SB_MASTER_PORT}
>>>>> > >> > fi
>>>>> > >> >
>>>>> > >> > if [ "x${present_master}" = x ]; then
>>>>> > >> > @@ -313,15 +325,44 @@ ovsdb_server_start() {
>>>>> > >> > # Force all copies to come up as slaves by pointing
>>>>> them into
>>>>> > >> > # space and let pacemaker pick one to promote:
>>>>> > >> > #
>>>>> > >> > + if [ "x${NB_MASTER_PROTO}" = xtcp ]; then
>>>>> > >> > + set $@ --db-nb-create-insecure-remote=yes
>>>>> > >> > + fi
>>>>> > >> > +
>>>>> > >> > + if [ "x${SB_MASTER_PROTO}" = xtcp ]; then
>>>>> > >> > + set $@ --db-sb-create-insecure-remote=yes
>>>>> > >> > + fi
>>>>> > >> > set $@ --db-nb-sync-from-addr=${INVALID_IP_ADDRESS}
>>>>> > >> > --db-sb-sync-from-addr=${INVALID_IP_ADDRESS}
>>>>> > >> >
>>>>> > >> > elif [ ${present_master} != ${host_name} ]; then
>>>>> > >> > + if [ "x${LISTEN_ON_SLAVE}" = xno ]; then
>>>>> > >> > + # TODO: for using LB vip, need to test for ssl.
>>>>> > >> > + set $@ --db-nb-create-insecure-remote=no
>>>>> > >> >
>>>>> > >>
>>>>> > >> The default value of "db-nb-create-insecure-remote" is no. So
>>>>> there is no
>>>>> > >> need to set this. We can just have
>>>>> > >> if [ "x${LISTEN_ON_SLAVE} = xyes ] ; then
>>>>> > >> if ...
>>>>> > >> ...
>>>>> > >> fi
>>>>> > >
>>>>> > > >>> Sure let me re-change and test. Will update the new patch
>>>>> accordingly.
>>>>> > >
>>>>> > >> + set $@ --db-sb-create-insecure-remote=no
>>>>> > >> > + else
>>>>> > >> > + if [ "x${NB_MASTER_PROTO}" = xtcp ]; then
>>>>> > >> > + set $@ --db-nb-create-insecure-remote=yes
>>>>> > >> > + fi
>>>>> > >> > +
>>>>> > >> > + if [ "x${SB_MASTER_PROTO}" = xtcp ]; then
>>>>> > >> > + set $@ --db-sb-create-insecure-remote=yes
>>>>> > >> > + fi
>>>>> > >> > + fi
>>>>> > >> > # An existing master is active, connect to it
>>>>> > >> > set $@ --db-nb-sync-from-addr=${MASTER_IP}
>>>>> > >> > --db-sb-sync-from-addr=${MASTER_IP}
>>>>> > >> > set $@ --db-nb-sync-from-port=${NB_MASTER_PORT}
>>>>> > >> > set $@ --db-nb-sync-from-proto=${NB_MASTER_PROTO}
>>>>> > >> > set $@ --db-sb-sync-from-port=${SB_MASTER_PORT}
>>>>> > >> > set $@ --db-sb-sync-from-proto=${SB_MASTER_PROTO}
>>>>> > >> > +
>>>>> > >> > + else
>>>>> > >> > + if [ "x${NB_MASTER_PROTO}" = xtcp ]; then
>>>>> > >> > + set $@ --db-nb-create-insecure-remote=yes
>>>>> > >> > + fi
>>>>> > >> > +
>>>>> > >> > + if [ "x${SB_MASTER_PROTO}" = xtcp ]; then
>>>>> > >> > + set $@ --db-sb-create-insecure-remote=yes
>>>>> > >> > + fi
>>>>> > >> > fi
>>>>> > >> >
>>>>> > >> > $@ start_ovsdb
>>>>> > >> > @@ -416,6 +457,11 @@ ovsdb_server_promote() {
>>>>> > >> > ;;
>>>>> > >> > esac
>>>>> > >> >
>>>>> > >> > + if [ "x${LISTEN_ON_SLAVE}" = xno ]; then
>>>>> > >>
>>>>> > >>
>>>>> > >> The use of LISTEN_ON_SLAVE param in "ovsdb_server_promote" is
>>>>> very
>>>>> > >> confusing. I think it should
>>>>> > >> be possible to have just one param - something like -
>>>>> > >> "master_ip_pacemaker_resource=(yes/no)" (or some thing better ).
>>>>> > >> Thoughts ?
>>>>> > >
>>>>> > > >>> Sure, how about master_ip_lb_resource to avoid confusion? So
>>>>> you
>>>>> > > mean, just use this variable and completely get rid of
>>>>> LISTEN_ON_SLAVE
>>>>> > > and LISTEN_ON_MASTER_IP_ONLY?
>>>>> > >
>>>>> >
>>>>> > 'master_ip_lb_resource' seems fine to me. Thats' right. Just have one
>>>>> > parameter.
>>>>> >
>>>>> >
>>>>> > Will there be a case where the user can select
>>>>> > >> listen_on_master_ip_only=no and
>>>>> > >> listen_on_slave=yes ?
>>>>> > >>
>>>>> > > >>> No.
>>>>> >
>>>>> 'master_ip_lb_resource' seems a little confusing to me. What does it
>>>>> really mean?
>>>>> The desired behavior is, when the flag is "no", do the default:
>>>>>
>>>>> - On both master node and slave node, listen to MASTER_IP only
>>>>>
>>>>> When the flag is "yes", it works with an external LB:
>>>>>
>>>>> 1) On mater node, listen on 0.0.0.0 (because the MASTER_IP doesn't
>>>>> belong to the node)
>>>>> 2) On slave node, don't listen on TCP port (because we want the LB to
>>>>> be able to tell which one is the master according to whether the TCP port
>>>>> is open on the node)
>>>>>
>>>>> I am ok with just one parameter, but I'd suggest to make it more
>>>>> straighforward. I am also wondering, can we unify point 2), i.e. never
>>>>> listen on TCP port for slave node? Numan, do you have any use case for
>>>>> accessing slave node DBs using TCP?
>>>>>
>>>>
>>
>> I don't think there is any use case. I actually thought about this and I
>> think it makes sense to unify. The only concern I had was whenever a slave
>> is promoted, it needs a restart of ovsdb-servers. But I think it's not a
>> big deal. It would require a little more testing from my side to make sure
>> that it doesn't break the existing behaviour.
>> @Aliasgar - Are you OK spinning another patch unifying this ? What do
>> you think ? Also we can rename the parameter back to
>> "listen_on_master_ip_only"
>> if we unify this.
>>
>> >>> Sure I thought of keeping it for backward compatibility as I was not
> sure of the implications of unifying it. If little more testing is ok, I am
> fine with that . Will also rename to listen_on_master_ip_only and send
> out v4.
>
>>
>>
>>
>>>
>>>>> >>> Thanks Han for the review. We agreed on this name just to avoid
>>>> confusion that IP is coming from LB and not pacemaker ipaddr2 resource. I
>>>> can update the documentation to avoid confusion since documentation have no
>>>> details of using LB VIP. Let me know if that needs to be in same or
>>>> different patch.
>>>>
>>>
>>> I am fine if we can better document the purpose of the parameter and the
>>> expected behavior. I think it is always good to be in same patch for the
>>> change and its documentation.
>>>
>> +1
>>
>> Thanks
>> Numan
>>
> >>> Noted. Will add docs in v4 too.
>
>>
>>
>>
>>>
>>> Thanks,
>>> Han
>>>
>>>
>>
>
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
