Re: [controller-dev] [OpenDaylight TSC] Transitioning Controller to being MRI

[Robert Varga]

On 25/03/2020 23:25, Robert Varga wrote:
> Hello everyone,

Hello again,

> The process involves three steps:
> 
> 1) transition all of controller's downstreams to use Magnesium version
> of controller artifacts
> 2) remove controller from autorelease
> 3) remove controller's distcheck jobs
> 4) perform a normal MRI bump during the Aluminium MRI Integration Window
> 
> I have staged the patches for 1 through 3 here:
> https://git.opendaylight.org/gerrit/q/topic:"controller-mri;
> 
> Multipatch-test is running here:
> https://jenkins.opendaylight.org/releng/view/integration/job/integration-multipatch-test-aluminium/1
> 
> Once that job succeeds, I will be asking for supercommitter rights to
> get those patches merged, after which controller's master will become
> disconnected from MSI projects participating in Aluminium SimRel.

This has been discussed at today's TSC call and approved. I have raised
the ticket for the rights to merge up the patches.

I plan to execute this tomorrow morning CET and the disruption should be
pretty much the same as for any MRI bump -- so around 6 hours, perhaps
less now that it involves fewer projects.

If you have objects, please yell now.

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14977): 
https://lists.opendaylight.org/g/controller-dev/message/14977
Mute This Topic: https://lists.opendaylight.org/mt/72568591/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[controller-dev] Transitioning Controller to being MRI

[Robert Varga]

Hello everyone,

controller committers have decided to transition the project to being
release integrated many moons ago. Unfortunately execution has been
postponed by various things happening -- and there are only a few points
during our lifecycle when such a transition is easily doable.

Since stable/magnesium and master for MSI projects are currently
synchronized with regards to MRI project versions and given the
interdependencies of Aluminium MRI window[0], NOW is the time to take
action.

The process involves three steps:

1) transition all of controller's downstreams to use Magnesium version
of controller artifacts
2) remove controller from autorelease
3) remove controller's distcheck jobs
4) perform a normal MRI bump during the Aluminium MRI Integration Window

I have staged the patches for 1 through 3 here:
https://git.opendaylight.org/gerrit/q/topic:"controller-mri;

Multipatch-test is running here:
https://jenkins.opendaylight.org/releng/view/integration/job/integration-multipatch-test-aluminium/1

Once that job succeeds, I will be asking for supercommitter rights to
get those patches merged, after which controller's master will become
disconnected from MSI projects participating in Aluminium SimRel.

This transition may break some things (around patch tests involving
controller). There is no known cook book at to what exactly needs to be
fixed and how, and the last time we did this was a long time ago, it was
mdsal, and the transition was executing in between branch and release of
... Fluorine was it? At any rate, such breakages do not really block
anything and we'll figure them out as we go.

If anybody has objections, holler now, or hold your peace (and watch the
fireworks).

Regards,
Robert

[0] odlparent brings Guava-28, which in turn makes Controller's
sal-{dom,binding}-api untenable, hence those have to go before
odlparent-7 is integrated, which really means we need controller/master
to make changes which cannot pass distcheck, as there are still laggard
downstreams using those APIs.



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14975): 
https://lists.opendaylight.org/g/controller-dev/message/14975
Mute This Topic: https://lists.opendaylight.org/mt/72552866/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] [netvirt-dev] [bgpcep-dev] [OpenDaylight TSC] TSC Vote to release latest build # 326 as Sodium SR2

[Robert Varga]

On 30/01/2020 20:21, Daniel De La Rosa wrote:
> @Robert Varga <mailto:n...@hq.sk>  and rest of the controller team, do
> you think that this
> issue, ttps://jira.opendaylight.org/browse/CONTROLLER-1928
> <https://jira.opendaylight.org/browse/CONTROLLER-1928> can be solved in
> the next days?

We do have https://git.opendaylight.org/gerrit/c/controller/+/87336, but
that does need careful review.

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14954): 
https://lists.opendaylight.org/g/controller-dev/message/14954
Mute This Topic: https://lists.opendaylight.org/mt/70252456/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] [Sodium] Failed to build hello world app into ODL

[Robert Varga]

On 20/01/2020 15:53, Aniello Paolo Malinconico wrote:
> Hi all,
> I want to deploy an application for OpendayLight Controller. So, I am
> following this
> guide: 
> https://docs.opendaylight.org/en/stable-sodium/developer-guide/developing-apps-on-the-opendaylight-controller.html
> My steps are quite simple:
> 
>  1. Setting up the infrastructure (correct settings.xml, install maven,
> install jdk);
>  2. Downloaded example module :
> 
> mvnarchetype:generate-DarchetypeGroupId=org.opendaylight.archetypes-DarchetypeArtifactId=opendaylight-startup-archetype
>  -DarchetypeCatalog=remote-DarchetypeVersion=1.2.1
> 
>  1. Giving these info: 
> 
> Define value for property 'groupId': : org.opendaylight.example
> Define value for property 'artifactId': : example
> Define value for property 'version':  1.0-SNAPSHOT: : 1.0.0-SNAPSHOT
> Define value for property 'package':  org.opendaylight.example: :
> Define value for property 'classPrefix':  
> ${artifactId.substring(0,1).toUpperCase()}${artifactId.substring(1)}
> Define value for property 'copyright': : Copyright (c) 2015 Yoyodyne, 
> Inc
> 
>  3. Run *mvn clean install* and after several minutes, it returnes*an
> error (see attached file compile_error.txt). *So I cant compile the
> project.
> 
> 
> Can anybody help me to troubleshoot it ?

This is an intermediate issue, should work once
https://lists.opendaylight.org/g/release/message/18500 is resolved.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14929): 
https://lists.opendaylight.org/g/controller-dev/message/14929
Mute This Topic: https://lists.opendaylight.org/mt/69932720/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] [OpenDaylight TSC] TWS Today

[Robert Varga]

On 06/01/2020 17:55, Tejas Nevrekar wrote:
> Hi all,
> 
> The agenda for today's TWS was for getting together to complete the
> lighty project proposal.
> 
> Not sure if the lighty team is ok for that today.
> 
> Robert, can you please confirm?

Sorry, there's a public holiday in Slovakia, so everyone's off...

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14926): 
https://lists.opendaylight.org/g/controller-dev/message/14926
Mute This Topic: https://lists.opendaylight.org/mt/69469680/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] [netconf-dev] File descriptors opened by ODL process.

[Robert Varga]

On 17/12/2019 15:16, Vikram Darsi wrote:
>  4. *lsof -p 31139 | wc -l    ->   503*

... the number of file descriptors open by the process.

>  5. *lsof | grep 31139 | wc -l   ->    58232*
> 
> *31139 is the ODL-controller PID*
> 

... the number of file descriptors open by the process multiplied by the
number of threads in the process. Examining the output without `wc -l`
makes that quite obvious.

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14922): 
https://lists.opendaylight.org/g/controller-dev/message/14922
Mute This Topic: https://lists.opendaylight.org/mt/68764307/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] [mdsal-dev] RetiredGenerationException : NeonSR1

[Robert Varga]

On 20/11/2019 14:42, Vikram Darsi wrote:
> Can someone please explain what could have gone bad and how to recover?

[...]

> 
> *Caused by:
> org.opendaylight.controller.cluster.access.concepts.RetiredGenerationException:
> Originating generation 0 was superseded by 8*

This indicates persistence has been completely wiped, see CONTROLLER-1626.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14911): 
https://lists.opendaylight.org/g/controller-dev/message/14911
Mute This Topic: https://lists.opendaylight.org/mt/60897577/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] Support for must constraints in the yang

[Robert Varga]

On 03/10/2019 20:50, Tejas Nevrekar wrote:

Thanks Robert. I have copied the controller-dev here.


[+yangtools-dev]

Still trying to guage a rough estimate and ownership for what it will 
take to implement the constraints in yang. Could you please suggest who 
can else can take this up?


Well, we are really operating as an open community. While the overall 
ownership of a particular projects lies with its committers, individual 
issues are not owned per se until somebody has them assigned -- at which 
point that individual owns it, either until it is resolved or it is 
moved back to unassigned state.


That way all efforts are made on a volunteer basis -- hence I cannot 
really nominate anyone to take this up.


Hence there are really three options, just as with any other FOSS project:
- find a willing volunteer
- do the work yourself
- pay someone to do the work

Regards,
Robert



Regards,
Tejas

On Thu, Oct 3, 2019, 11:38 PM Robert Varga <mailto:n...@hq.sk>> wrote:


On 03/10/2019 19:54, Tejas Nevrekar wrote:
 > Hi Robert,

Hello,

 >
 > Continuing here the discussion we had during the DDF when
reviewing the
 > presentation for the northbound validations using cohorts
 >

(https://wiki.lfnetworking.org/download/attachments/19006711/Data%20Validation%20Framework.pdf?version=1=1569500540267=v2),

 > I came across a few tickets as below:
 >
 > - XPath support - https://jira.opendaylight.org/browse/YANGTOOLS-477
 > - Constraints - https://jira.opendaylight.org/browse/YANGTOOLS-194
 >
 > Do you think this is feasible to get included in Mg?

I am not sure what you mean by 'feasible'.

If you are asking whether it is possible to deliver the functionality,
then yes, the ~5 months available are plenty of time.

If you are asking whether I will commit to delivering this
functionality, then no, I cannot make such a commitment at this time.

Just as the DDF was open, so are community discussions.

Please do use mailing lists, as otherwise your emails are at risk of
falling through, without ever having been seen.

Regards,
Robert

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14894): 
https://lists.opendaylight.org/g/controller-dev/message/14894
Mute This Topic: https://lists.opendaylight.org/mt/34385414/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] Taking controller to MRI

[Robert Varga]

On 20/06/2019 14:31, Stephen Kitt wrote:
> On Sat, Jun 01, 2019 at 11:33:27PM +0200, Robert Varga wrote:
> [...]
>> Unlike the previous MRI split-offs of odlparent/yangtools/mdsal, I do
>> not believe controller is ready to also transition to strict SemVer
>> rules -- we have not consolidated our codebase yet and we will need to
>> sweep our codebase at some point for defunct/unused remnants. I would
>> hate to see that effort being hampered by artificial rules.
> Bumping majors isn’t the end of the world ;-). Until we can actually
> use version ranges, it doesn’t make any practical difference to
> downstreams anyway...
> 
>> That having been said, we should (and will, if I can help it) maintain
>> reasonable compatibility where our downstreams are concerned.
>>
>> I am therefore raising the following vote for controller project
>> committers, please vote with usual +1/0/-1:
>>
>> Do you agree to controller transitioning to being a Managed Release
>> Integrated project?
> +1 from me too!

So after three months this stands at: two votes in favor, zero against,
zero abstains, with no other traffic whatsoever.

With my PTL hat on, I am declaring this a GO, unless somebody brings
strong objections by the time Magnesium opens (9/23/2019, i.e. in one week).

The plan of record is to issue a controller release after the Magnesium
Version Bump Checkpoint (10/21/2019) and propagate that release to
downstreams, just as any MRI minor update.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14889): 
https://lists.opendaylight.org/g/controller-dev/message/14889
Mute This Topic: https://lists.opendaylight.org/mt/33223310/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[controller-dev] Read(Write)Transaction.exists(InstanceIdentifier)

[Robert Varga]

Hello everyone,

I realized that it is not general knowledge that aside from
ReadTransaction.read(InstanceIdentifier), there also is an operation to
check for a particular node's existence.

This operation has been available at the DOM layer since Helium, but
somehow it ended up not being exposed from Binding ... sorry :(

It has been exposed since Oxygen SR4 and is documented here:
https://static.javadoc.io/org.opendaylight.controller/sal-binding-api/1.7.4/org/opendaylight/controller/md/sal/binding/api/ReadTransaction.html#exists-org.opendaylight.controller.md.sal.common.api.data.LogicalDatastoreType-org.opendaylight.yangtools.yang.binding.InstanceIdentifier-

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14888): 
https://lists.opendaylight.org/g/controller-dev/message/14888
Mute This Topic: https://lists.opendaylight.org/mt/34156716/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] [integration-dev] [tsc][release] Configure Smart Proxy between ODL Nexus and Central

[Robert Varga]

On 28/08/2019 06:36, Anil Belur wrote:
> Greetings all,
> 
> Sonatype has updated the issue that was opened Maven Central [1.] on the
> missing staging repositories and made some recommendations. A large
> number of staging repositories along with dup artifacts is making
> Central instable. Therefore they have recommended a workaround to
> configure the proxy between ODL Nexus and Maven Central [2.], that will
> inform and publish artifacts downstream when they are created on ODL Nexus.
> 
> Going forward, we'll need to configure the smart proxy on our the
> environment to sync and publish artifacts onto Central. Also, we'll need
> to ensure this is done on priority so that ODL's staging repositories
> for the upcoming releases don't get purged.

Hello again,

I would like to note that this activity has been successfully completed
(thanks Anil and the Sonatype folks).

There is one unforeseen side-effect of this: all of our artifacts ever
released (well, at least all the way back to Helium release) are now
available on Maven Central -- i.e.
https://search.maven.org/search?q=g:org.opendaylight.controller%20AND%20a:sal-binding-api=gav

The immediate upshot of this is that we have javadocs for all artifacts
also available through javadoc.io, for example:

https://static.javadoc.io/org.opendaylight.controller/sal-binding-api/1.7.4/index.html?overview-summary.html

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14887): 
https://lists.opendaylight.org/g/controller-dev/message/14887
Mute This Topic: https://lists.opendaylight.org/mt/34156626/21656
Group Owner: controller-dev+ow...@lists.opendaylight.org
Unsubscribe: https://lists.opendaylight.org/g/controller-dev/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [controller-dev] Migrating inventory/topology models

[Robert Varga]

[+ discuss, mdsal-dev]

On 05/09/2019 19:24, Anil Vishnoi wrote:
> As for the next steps, I think we need to migrate these models to
> openflowplugin, where they can be maintained, as that world is the only
> place that really uses them.
> 
> As far as upstream OpenDaylight is concern this make sense to me, but we
> need to be careful about the downstream consumer. Downstream user who
> just use core ODL projects (Controller, yangtools, mdsal,aaa) to develop
> their standalone application might be using these models, so this
> movement will break them and to solve this they will have to put
> dependency on openflowpluing, which they might not want. 

That is a valid concern, yes.

The answer really lies in packaging, as if you are using karaf, you will
have openflowplugin-features added to you distribution. Those would not
be installed, but will bloat the distro & confuse the users.

On the other hand, every feature we build is a separate repository, so
while you'd depend on openflowplugin-artifacts, you do not have to
depend on openflowplugin-features -- I think.

Without Karaf, you depend on whatever you like, so yeah, you get tied up
with OFP release cycle, but other than that there should not be a problem.

As far as those models are concerned, platform (mdsal/controller/netc)
position on them can be summed up as:

Migrate to using ietf-network(-topology), which are shipped from MD-SAL
for a year now. There are RFC8345 standard and are not expected to make
in the foreseeable future.
As a further note, while performing that migration, also move away from
using yang-ext.yang routed RPCs by switching to YANG 1.1 actions. Such
models are not supported by legacy RESTCONF (which itself is
deprecated), but are fully supported by RFC8040 RESTCONF (which is the
only actively supported version).

On a related note, this discussion will also be coming up with relation to:

ietf-packet-fields
ietf-access-control-list
ietf-lisp-address-types
ietf-ted
ietf-topology
ietf-topology-isis
ietf-topology-l3-unicast-igp
ietf-topology-ospf

as MD-SAL will be gradually dropping at least those, which have been
superseded by RFCs.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

[controller-dev] Migrating inventory/topology models

[Robert Varga]

Hello everyone,

as it currently stands, then only projects which are using
opendaylight-{inventory,topology}*.yang models are OpenFlow-specific
projects (openflowplugin, genius, sfc (in sfc-genius-utils), netvirt),
plus a soon-to-be-deprecated component in controller/netconf.

These models have been deemed as deprecated a long time (3+ years) ago,
but the effort to migrate off of them has never materialized, which has
left us in a sorry state, where the usage of those models incurs
deprecation warnings (all over the place) and there is no target to
transition to.

We have
https://git.opendaylight.org/gerrit/q/+I1e3d27374ffba0e584f194d468cebcfa9cecfe81
merged on master, which will be followed by all other branches. This
will alleviate the deprecation pain downstream.

As for the next steps, I think we need to migrate these models to
openflowplugin, where they can be maintained, as that world is the only
place that really uses them.

Any objections?

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Same service exposed twice with the same odl filter

[Robert Varga]

On 29/07/2019 13:03, Vikram Darsi wrote:
> Hi Team
> 
>  
> 
> Distribution : ODL Neon SR1
> 
>  
> 
> Installed “odl-netconf-all” feature and ran “service:list”
> 
>  
> 
> [org.opendaylight.mdsal.binding.dom.codec.api.BindingCodecTreeFactory,
> org.opendaylight.mdsal.binding.dom.codec.api.BindingNormalizedNodeSerializer]
> 
> 
> 
> service.id = 224
> 
> osgi.service.blueprint.compname = mappingCodec
> 
> service.bundleid = 125
> 
> service.scope = bundle
> 
> type = default
> 
> Provided by :
> 
> org.opendaylight.controller.sal-binding-broker-impl (125)
> 
>  
> 
> [org.opendaylight.mdsal.binding.dom.codec.api.BindingCodecTreeFactory,
> org.opendaylight.mdsal.binding.dom.codec.api.BindingNormalizedNodeSerializer]
> 
> 
> 
> service.id = 169
> 
> osgi.service.blueprint.compname = mappingCodec
> 
> service.bundleid = 144
> 
> service.scope = bundle
> 
> type = default
> 
> Provided by :
> 
> mdsal-binding-dom-adapter (144)
> 
>  
> 
>  
> 
> Is this expected?

Yes.

> When below statement is executed, we are getting ClassCastException
> 
>  
> 
> BindingNormalizedNodeCodecRegistrycodecReg=
> ((BindingToNormalizedNodeCodec)/codecRegistry/).getCodecRegistry();

Yes, obviously. You are attempting to cast a service reference to an
implementation class -- I suspect that blows up on the reference being a
proxy. Why would you even need that cast?

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] 答复: [release] CDS serialization format in Neon SR2/Sodium

[Robert Varga]

On 19/07/2019 02:22, Yi Yang (杨燚)-云服务集团 wrote:
> Robert, the data in https://pantheon.tech/opendaylight-neon-sr1-sr2/ looks 
> amazing, can you show us those gerrit changes? How can we double check those 
> data you got?

For Neon that's ...

https://git.opendaylight.org/gerrit/#/c/82315/
https://git.opendaylight.org/gerrit/#/c/82316/
https://git.opendaylight.org/gerrit/#/c/82381/
https://git.opendaylight.org/gerrit/#/c/82382/
https://git.opendaylight.org/gerrit/#/c/82383/
https://git.opendaylight.org/gerrit/#/c/82409/
https://git.opendaylight.org/gerrit/#/c/82461/
https://git.opendaylight.org/gerrit/#/c/82462/
https://git.opendaylight.org/gerrit/#/c/82465/
https://git.opendaylight.org/gerrit/#/c/82466/
https://git.opendaylight.org/gerrit/#/c/82467/
https://git.opendaylight.org/gerrit/#/c/82468/
https://git.opendaylight.org/gerrit/#/c/82769/
https://git.opendaylight.org/gerrit/#/c/82770/
https://git.opendaylight.org/gerrit/#/c/82773/

To evaluate these ... the simplest is to place a workload on SR1 and
current stable/neon and compare JVM heap composition, most notably on
followers and/or after a node was rebooted.

If you are using segmented journal, you should also see the median size
of persisted messages drop off in the JMX-exposed statistics it provides.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] CDS serialization format in Neon SR2/Sodium

[Robert Varga]

On 18/07/2019 23:41, Luis Gomez wrote:
> During a cluster upgrade where you upgrade members one-by-one, how can we 
> control the leader does not move to a Neon SR2 member?

You can explicitly move the leader via cluster-admin interface -- that
way the leader movement will not be triggered by nodes being taken out
of service for upgrade.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] CDS serialization format in Neon SR2/Sodium

[Robert Varga]

On 18/07/2019 20:52, Luis Gomez wrote:
> 
> 
>> On Jul 18, 2019, at 10:53 AM, Robert Varga  wrote:
>>
>> On 18/07/2019 19:14, Luis Gomez wrote:
>>> Am I understanding we are introducing a non-compatible change in a Service 
>>> Release?
>>
>> If you define non-compatible as 'you cannot just downgrade software',
>> yes. I do not believe we have an in-place downgrade story -- and daexim
>> remains a valid option.
> 
> What about the upgrade itself, can Neon SR1 upgrade to Neon SR2 seamlessly?

No special steps, local state replica is upgraded on first boot, just as
with any other upgrade in the past.

>>> I hope there is good one because otherwise this is not a good practice.
>>
>> We have done this multiple times already, probably most well documented
>> are He -> He SR1 and He SR1 -> He SR2.
> 
> Those were the old times where ODL was a playground, at this moment we have 
> multiple customers using ODL in production so I do not think we can afford a 
> complicated/risky SR upgrade.

I would disagree about the playground bit, it certainly did not feel
that way in these parts :)

I do not believe the upgrade is any riskier than any other upgrade.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] CDS serialization format in Neon SR2/Sodium

[Robert Varga]

On 18/07/2019 21:24, Luis Gomez wrote:
> Also, are these changes already merged in the stable branch?

Yes, they are.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] CDS serialization format in Neon SR2/Sodium

[Robert Varga]

On 18/07/2019 19:14, Luis Gomez wrote:
> Am I understanding we are introducing a non-compatible change in a Service 
> Release?

If you define non-compatible as 'you cannot just downgrade software',
yes. I do not believe we have an in-place downgrade story -- and daexim
remains a valid option.

> If so what is the reason for this?

Scalability, as detailed below.

> I hope there is good one because otherwise this is not a good practice.

We have done this multiple times already, probably most well documented
are He -> He SR1 and He SR1 -> He SR2.

Regards,
Robert


> 
> BR/Luis
>  
> 
>> On Jul 18, 2019, at 8:01 AM, Robert Varga  wrote:
>>
>> Hello everyone,
>>
>> this is a notice that CDS/sal-distributed-datastore will ship an
>> upgraded data streaming format in Neon SR2 -- something we have not done
>> since Lithium.
>>
>> This change impacts interoperability with previous versions (up to and
>> including Neon SR1):
>>
>> 1) each node will upgrade its local data replica to the new format as
>> soon as it performs recovery with Neon SR2 software. The only way to
>> undo this upgrade is to restore the node, using mechanisms like daexim.
>>
>> 2) mixed-version shards will remain operational for as long as no Neon
>> SR2 becomes the leader. Once this happens, Neon SR1-and-older slaves
>> will not be able to join.
>>
>> 3) Neon SR2 shards can still service old clients, i.e. applications
>> running on Neon SR1-or-older nodes.
>>
>> 4) Neon SR1 shards can service new clients, i.e. applications running on
>> Neon SR2 nodes.
>>
>> 5) Unversioned users, like netconf-topology-singleton request routing
>> and cluster-wide RPCs, are not affected by this change
>>
>> This break in compatibility is redeemed by the new streaming format
>> being both faster and smaller -- as detailed here:
>> https://pantheon.tech/opendaylight-neon-sr1-sr2/
>>
>> Sodium release train also has these changes, with the notable difference
>> that unversioned endpoints use the new streaming format.
>>
>> Regards,
>> Robert
>>
>>
>> ___
>> release mailing list
>> rele...@lists.opendaylight.org
>> https://lists.opendaylight.org/mailman/listinfo/release
> 



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

[controller-dev] CDS serialization format in Neon SR2/Sodium

[Robert Varga]

Hello everyone,

this is a notice that CDS/sal-distributed-datastore will ship an
upgraded data streaming format in Neon SR2 -- something we have not done
since Lithium.

This change impacts interoperability with previous versions (up to and
including Neon SR1):

1) each node will upgrade its local data replica to the new format as
soon as it performs recovery with Neon SR2 software. The only way to
undo this upgrade is to restore the node, using mechanisms like daexim.

2) mixed-version shards will remain operational for as long as no Neon
SR2 becomes the leader. Once this happens, Neon SR1-and-older slaves
will not be able to join.

3) Neon SR2 shards can still service old clients, i.e. applications
running on Neon SR1-or-older nodes.

4) Neon SR1 shards can service new clients, i.e. applications running on
Neon SR2 nodes.

5) Unversioned users, like netconf-topology-singleton request routing
and cluster-wide RPCs, are not affected by this change

This break in compatibility is redeemed by the new streaming format
being both faster and smaller -- as detailed here:
https://pantheon.tech/opendaylight-neon-sr1-sr2/

Sodium release train also has these changes, with the notable difference
that unversioned endpoints use the new streaming format.

Regards,
Robert




signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Guava-28 removed CheckedFuture

[Robert Varga]

On 01/07/2019 10:28, Faseela K wrote:
>I will try to take it up for genius, netvirt and serviceutils.

Hello Faseela,

based on the direction taken in
https://git.opendaylight.org/gerrit/82889, I think the best approach for
genius/netvirt would be:

1) eliminate netvirt's use of non-Typed*Transaction interfaces from
genius and those @Deprecated genius methods

2) deprecate genius.infra.Typed*Transaction, as it has an MD-SAL
counterpart in mdsal.binding.util

3) mass-migrate all users

This way we would end up with:
1) immediate performance gains (fewer DS transactions)
2) users concentrated on a well-known API
3) a simple flag day merge

Regards,
Robert

P.S.: controller deprecations are already in place and I am seeing all
yellow, this will make the code *way* more maintainable



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Guava-28 removed CheckedFuture

[Robert Varga]

On 01/07/2019 11:08, Robert Varga wrote:
> 
> On 01/07/2019 10:28, Faseela K wrote:
>> Robert,
>>Can you create a JIRA with the plan for deprecation and removal of 
>> controller APIs?
> Sure.
> 
> https://jira.opendaylight.org/browse/CONTROLLER-1902 tracks complete
> deprecation.

This has now been merged.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Notificatoins over websockets: Making JSON notifications to go just from one node in cluster

[Robert Varga]

On 01/07/2019 19:35, N Vivekanandan wrote:
> Hi Team,

Hello,

[+neutron-dev, see below]

> Can you please let us know if its possible to provide an enhanced API
> from RestConfImpl, where we can request that the
> datastore-change-notifications be JSON Wrapped and sent only from one
> node of an MD-SAL cluster?
> 
>  
> 
> For example, the port-status change listener subscribes to send
> websocket notifications on all the 3 nodes in our MD-SAL Cluster.

This is the problem: why do you register on all nodes? If you do, you
have three streams and it is *your* responsibility to
synchronize/deduplicate them.

> This results in same port-status notifications sent 3 times on every
> change in the status of the neutron port in the ODL Controller.
> 
>  
> 
> We wanted to send just “one port-status notification per status change
> on the port” towards the Openstack services from the ODL controller.
> 
>  
> 
> Can you please let us know if there is a simpler way that we would be
> able to send websocket notifications from only one MD-SAL cluster
> 
> node towards openstack within the RestConfImpl?
> 
>  
> 
> We thought of using EOS (EntityOwnershipService), but looks we have to
> resubscribe to the stream everytime on the new node which

Got a pointer to a patch?

> becomes an EOS leader due to elections in a running cloud, and this
> resubscribing to the stream may resend tons of notifications towards
> 
> Openstack on a scaled cloud with about 6000 ACTIVE ports.
> 
>  
> 
> https://github.com/opendaylight/neutron/blob/master/northbound-api/src/main/java/org/opendaylight/neutron/northbound/impl/PortStatusUpdateInitializer.java
> 

... I am not quite getting why neutron is talking to RESTCONF instead of
directly to MD-SAL here. Then, obviously, neutron can use
ClusterDataTreeChangeListener coupled with Cluster Singleton Service to
provide proper subscription service.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Guava-28 removed CheckedFuture

[Robert Varga]

On 01/07/2019 10:28, Faseela K wrote:
> Robert,
>Can you create a JIRA with the plan for deprecation and removal of 
> controller APIs?

Sure.

https://jira.opendaylight.org/browse/CONTROLLER-1902 tracks complete
deprecation.

https://jira.opendaylight.org/browse/CONTROLLER-1903 tracks removal.

>I will try to take it up for genius, netvirt and serviceutils.

Awesome, thanks!

Regards,
Robert

> Thanks,
> Faseela
> 
> -Original Message-----
> From: Robert Varga  
> Sent: Monday, July 1, 2019 1:56 PM
> To: Faseela K ; rele...@lists.opendaylight.org; 
> odlparent-...@lists.opendaylight.org; controller-dev@lists.opendaylight.org
> Cc: disc...@lists.opendaylight.org; t...@lists.opendaylight.org
> Subject: Re: [controller-dev] Guava-28 removed CheckedFuture
> 
> On 01/07/2019 10:13, Faseela K wrote:
>> Robert,
> 
> Hey Faseela,
> 
>>What is the timeline for finishing this migration?
> 
> I do not have a specific timeline and certainly I cannot commit to finishing 
> up all the patches.
> 
> As for controller API removal, I do want to completely deprecate them (i.e. 
> all interfaces/classes) in Sodium and remove them in Aluminium (specifically, 
> during its MRI window in April 2020).
> 
>>I hope this is the corresponding neutron patch?
>>  
>> https://protect2.fireeye.com/url?k=b90d94dd-e5844e9a-b90dd446-0cc47ad9
>> 3c18-4e437efcd33c907d=1=https%3A%2F%2Fgit.opendaylight.org%2Fgerri
>> t%2F%23%2Fc%2F82802%2F
> 
> This one: 
> https://protect2.fireeye.com/url?k=4729bdb2-1ba067f5-4729fd29-0cc47ad93c18-0a9c7e3316fc2735=1=https%3A%2F%2Fgit.opendaylight.org%2Fgerrit%2F80860
> 
> Regards,
> Robert
> 
>> Thanks,
>> Faseela
>>
>> -Original Message-
>> From: controller-dev-boun...@lists.opendaylight.org 
>>  On Behalf Of Robert 
>> Varga
>> Sent: Monday, July 1, 2019 1:36 PM
>> To: rele...@lists.opendaylight.org; 
>> odlparent-...@lists.opendaylight.org; 
>> controller-dev@lists.opendaylight.org
>> Cc: disc...@lists.opendaylight.org; t...@lists.opendaylight.org
>> Subject: [controller-dev] Guava-28 removed CheckedFuture
>>
>> Hello everyone,
>>
>> this is just a heads up that Guava 28 removed CheckedFuture:
>>
>> https://protect2.fireeye.com/url?k=b8e15d45-e435574c-b8e11dde-86740465
>> fc08-65a26a6532eb57e6=1=https%3A%2F%2Fgithub.com%2Fgoogle%2Fguava%
>> 2Freleases%2Ftag%2Fv28.0
>>
>> This means that controller-based MD-SAL APIs are now officially dead weight.
>>
>> While there is no immediate need to upgrade Guava in Magnesium, there is now 
>> a real need to get off of controller/sal-*-api -- most projects are already 
>> done, but there are still some left:
>>
>> - serviceutils
>> - bgpcep
>> - ovsdb
>> - neutron
>> - genius
>> - sfc
>> - netvirt
>>
>> neutron already has a proposed patch, bgpcep has a patch in progress, the 
>> rest seem to be subtly intertwined and will need some effort to migrate.
>>
>> Regards,
>> Robert
>>
> 



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Guava-28 removed CheckedFuture

[Robert Varga]

On 01/07/2019 10:13, Faseela K wrote:
> Robert,

Hey Faseela,

>What is the timeline for finishing this migration?

I do not have a specific timeline and certainly I cannot commit to
finishing up all the patches.

As for controller API removal, I do want to completely deprecate them
(i.e. all interfaces/classes) in Sodium and remove them in Aluminium
(specifically, during its MRI window in April 2020).

>I hope this is the corresponding neutron patch?
>  https://git.opendaylight.org/gerrit/#/c/82802/

This one: https://git.opendaylight.org/gerrit/80860

Regards,
Robert

> Thanks,
> Faseela
> 
> -Original Message-
> From: controller-dev-boun...@lists.opendaylight.org 
>  On Behalf Of Robert Varga
> Sent: Monday, July 1, 2019 1:36 PM
> To: rele...@lists.opendaylight.org; odlparent-...@lists.opendaylight.org; 
> controller-dev@lists.opendaylight.org
> Cc: disc...@lists.opendaylight.org; t...@lists.opendaylight.org
> Subject: [controller-dev] Guava-28 removed CheckedFuture
> 
> Hello everyone,
> 
> this is just a heads up that Guava 28 removed CheckedFuture:
> 
> https://protect2.fireeye.com/url?k=b8e15d45-e435574c-b8e11dde-86740465fc08-65a26a6532eb57e6=1=https%3A%2F%2Fgithub.com%2Fgoogle%2Fguava%2Freleases%2Ftag%2Fv28.0
> 
> This means that controller-based MD-SAL APIs are now officially dead weight.
> 
> While there is no immediate need to upgrade Guava in Magnesium, there is now 
> a real need to get off of controller/sal-*-api -- most projects are already 
> done, but there are still some left:
> 
> - serviceutils
> - bgpcep
> - ovsdb
> - neutron
> - genius
> - sfc
> - netvirt
> 
> neutron already has a proposed patch, bgpcep has a patch in progress, the 
> rest seem to be subtly intertwined and will need some effort to migrate.
> 
> Regards,
> Robert
> 



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

[controller-dev] Guava-28 removed CheckedFuture

[Robert Varga]

Hello everyone,

this is just a heads up that Guava 28 removed CheckedFuture:

https://github.com/google/guava/releases/tag/v28.0

This means that controller-based MD-SAL APIs are now officially dead weight.

While there is no immediate need to upgrade Guava in Magnesium, there is
now a real need to get off of controller/sal-*-api -- most projects are
already done, but there are still some left:

- serviceutils
- bgpcep
- ovsdb
- neutron
- genius
- sfc
- netvirt

neutron already has a proposed patch, bgpcep has a patch in progress,
the rest seem to be subtly intertwined and will need some effort to migrate.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] 答复: Problem: ODL Neon archetype repo missing?

[Robert Varga]

On 20/06/2019 18:41, Robert Varga wrote:
> 
> Okay, with Stephen's help I was able to get the project into a
> semi-reasonable state -- i.e. you can currently use 1.1.0-SNAPSHOT to
> target Neon GA and 1.2.0-SNAPSHOT to target current Sodium.
> 
> There are a couple more patches to completely catch up to Fluorine
> SR2/SR3 and Neon SR1/current, which we will continue to work on, issuing
> proper releases for that.
> 
> Going forward, though, we need a caretaker committer, who will keep an
> eye on version bumps and releases, to produce the equivalent of these
> patches:
> 
> https://git.opendaylight.org/gerrit/82593 (i.e. archetypes version bump)
> https://git.opendaylight.org/gerrit/82594 (i.e. post-SR bump)
> https://git.opendaylight.org/gerrit/82599 (i.e. documentation update)
> 
> and spin standalone releases.
> 
> Any takers?

[+ release, discuss, as per my AI]

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Following the ODL neon archetype repo problem

[Robert Varga]

On 24/06/2019 05:33, 金恺文 wrote:
> Hi all,
> 
> Thank you all very much for the quick response, now we are able to download 
> the archetype ;), but we found another issue: 
> 
> The Example project that we built with the version 1.1.0-SNAPSHOT archetype 
> has the odlparent of version 4.0.9 as maven dependency, but the neon sr1 
> release code has the odlparent of version 4.0.10 as maven dependency, and we 
> wonder why they have different versions of odlparent

Because 1.1.0-SNAPSHOT (and now 1.1.0) tracks Neon GA. 1.1.1 tracks Neon
SR1. 1.1.2-SNAPSHOT will track Neon SR2 development.

Catching up the project from both capability and documentation
perspective is going to need some more time.

Regards.
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] 答复: Problem: ODL Neon archetype repo missing?

[Robert Varga]

On 20/06/2019 16:32, Abhijit Kumbhare wrote:
> Sure - we can discuss it on today's call.
> 
> On Thu, Jun 20, 2019 at 3:38 AM Robert Varga  <mailto:n...@hq.sk>> wrote:
> 
> On 20/06/2019 04:39, 金恺文 wrote:
> > Hi everyone，
> 
> Hello,
> 
> > *Failed to execute goal
> > org.apache.maven.plugins:maven-archetype-plugin:3.1.1:generate
> > (default-cli) on project standalone-pom: The desired archetype
> does not
> > exist
> >
> 
> (org.opendaylight.archetypes:opendaylight-startup-archetype:1.1.0-SNAPSHOT)*
> >
> > I confirm that I have changed the settings.xml as required, so is
> there
> > any other repository where the archetypes are hosted?
> >
> > Hope someone could answer my amateur question J, thanks
> 
> The archetypes project is sorrily out of date to the point of its merge
> jobs not being able to complete, so the -SNAPSHOT versions were not
> refreshed and expired from nexus.

Okay, with Stephen's help I was able to get the project into a
semi-reasonable state -- i.e. you can currently use 1.1.0-SNAPSHOT to
target Neon GA and 1.2.0-SNAPSHOT to target current Sodium.

There are a couple more patches to completely catch up to Fluorine
SR2/SR3 and Neon SR1/current, which we will continue to work on, issuing
proper releases for that.

Going forward, though, we need a caretaker committer, who will keep an
eye on version bumps and releases, to produce the equivalent of these
patches:

https://git.opendaylight.org/gerrit/82593 (i.e. archetypes version bump)
https://git.opendaylight.org/gerrit/82594 (i.e. post-SR bump)
https://git.opendaylight.org/gerrit/82599 (i.e. documentation update)

and spin standalone releases.

Any takers?

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] 答复: Problem: ODL Neon archetype repo missing?

[Robert Varga]

On 20/06/2019 04:39, 金恺文 wrote:
> Hi everyone，

Hello,

> Sorry I put the wrong link in the previous mail, I followed the
> neon-version guide but gave the boron-version link
> 
>  
> 
> So here is my question after correction:
> 
>  
> 
> I was trying to create an Example project following this developer
> guide,
> https://docs.opendaylight.org/en/stable-neon/developer-guide/developing-apps-on-the-opendaylight-controller.html,
> but the archetype repository seemed not to exist.
> 
> I did
> 
> *mvn archetype:generate
> **-**DarchetypeGroupId**=**org**.**opendaylight**.**archetypes
> **-**DarchetypeArtifactId**=**opendaylight**-**startup**-**archetype \*
> 
> *-**DarchetypeCatalog**=**remote
> **-**DarchetypeVersion**=**1.0**.**0**-**SNAPSHOT*
> 
>  
> 
> Then I got this error from maven:
> 
> *Failed to execute goal
> org.apache.maven.plugins:maven-archetype-plugin:3.1.1:generate
> (default-cli) on project standalone-pom: The desired archetype does not
> exist
> (org.opendaylight.archetypes:opendaylight-startup-archetype:1.1.0-SNAPSHOT)*
> 
> I confirm that I have changed the settings.xml as required, so is there
> any other repository where the archetypes are hosted?
> 
> Hope someone could answer my amateur question J, thanks

The archetypes project is sorrily out of date to the point of its merge
jobs not being able to complete, so the -SNAPSHOT versions were not
refreshed and expired from nexus.

Abhijit, can we discuss the way forward there at today's TSC call?

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] 答复: 答复: 答复: 答复: Is Read from follower shard ok and openflowplugin master must be shard leader?

[Robert Varga]

On 07/06/2019 15:03, Yi Yang (杨燚)-云服务集团 wrote:
> Robert, let us use inventory as an example, assume I started 127 ODL nodes,
> how many leader shards can we have for it? I think it is one, so for this
> case, only one node, i.e. inventory leader shard node is handling all the
> inventory read and write (including serving for remote read and write), isn't
> it?

Right now you can have as many shards as there are models which define
root-level nodes. This is the default, and suffers exactly from the
'inventory leader' syndrome, where you just cannot partition the data to
fit the access pattern. I am sorry, but this is how shards were contributed.

> Please let me know how we can split inventory into many shards (I don't mean
> replica shards) in order that we can have any leader shards per requirement,
> for example, we have 100 openflowplugin clusters, every has its own inventory
> leader shard, that is what I want, can you tell me how I can do this in
> current ODL clustering implementation? You told me prefix-based shard, but it
> isn't ready for this, right?

So this is really about data partitioning, which is what shards are
supposed to be. The use case is that each OVS switch has some
master-related state and therefore, OFP needs to be able to tell CDS to
move the master to co-locate that state with where the OFP master lives.

This functionality is available with prefix-based shards, which I think
does provide compatibility APIs DOMStore, but I'd have to check.

Any testing feedback is appreciated.

> BTW, you Pantheon sales Martin Varga contacted me, he told me he would ask 
> you 
> to have a meeting with me about our requirement, I would like to have more 
> discussion with you, I'm not sure if you catch our pain point.

Let's follow up separately.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] 答复: 答复: 答复: Is Read from follower shard ok and openflowplugin master must be shard leader?

[Robert Varga]

On 05/06/2019 02:49, Yi Yang (杨燚)-云服务集团 wrote:
> Robert, thank you so much for your insightful answers, I'm wondering if we 
> can have a meeting to discuss this specially, last week, we discussed this in 
> openflowplugin weekly meeting, I believe you participated in ODL DDF and 
> joined discussion about :ODL scale" Luis presented, it will be great let us 
> have a meeting to focus on this.

Well, I believe in rough consensus and running code -- and I have not
seen anything aside from high-level slides so far.

> Abhijit, Anil, does my suggestion make sense? Robert is the strongest arguer 
> for akka-based ODL cluster.

I am not arguing either way. What I am doing here is dispelling
misconceptions and trying to understand the *exact* nature of the problem.

> Robert, but in ODL cluster, there is only one leader node no matter how many 
> nodes we have, I think it is bottleneck, isn't it?

That is not correct. Every Shard has its own leader and it is a matter
of how shards are laid out.

> In my mind, message queue is the only feasible way to synchronize data in 
> larger scale distributed application, I'm not sure if akka is using the same 
> way to handle data synchronization. I would like to get your idea about this. 
> I know akka uses gossip, but leader node will be responsible for 
> synchronizing data to all the other follower nodes, this is a big issue, in 
> message queue solution, message servers can handle this workload, data 
> producer just send data once, in current ODL cluster, I think, the leader 
> node will send N-1 times data to all the other follower nodes, please correct 
> me if I'm wrong.

I do not see how a message queue (directly) is solving the problem. At
the end of the day, every actor has a message queue, so what are we
really talking about?

This thread has been heavy on CDS criticism, without ever mentioning
what the applications are doing, how or why.

Before jumping to solutions, I *really* would like us understanding the
requirements and the problem we are faced with -- software engineering
is *engineering*, i.e. picking reasonable trade-offs from the solution
space for a given problem space and constraints.

Currently, there has been no data floated in this thread, so ... can we
follow UNPHAT, as per
https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb ?

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] 答复: 答复: Is Read from follower shard ok and openflowplugin master must be shard leader?

[Robert Varga]

On 04/06/2019 02:29, Yi Yang (杨燚)-云服务集团 wrote:
> Robert, we're talking about scalability, can you tell us how many nodes 
> current akka-base clustering can support at most?

Yi,

I think we have vocabulary (i.e. language) discrepancy. In order to be
clear:

- "performance" means how fast a system is when operating with a certain
working set

- "scalability" means how well a system is able to maintain performance
when the working set is increased. I think you may have meant this when
you asked about IMDT "efficiency", but I can't be sure.

In a potentially-distributed system, there are two distinct parts which
affect how the system can scale:

- "vertical scalability" means how well the system can be scaled by
increasing resources available to individual nodes

- "horizontal scalability" means how well the system can be scaled by
increasing the number of individual nodes

I think it is always more efficient use of resources to allocated them
to scaling vertically rather than horizontally -- each node
participating in a distributed system typically requires non-zero overhead.

The number of potential nodes is limited by what Akka can provide us
with -- which I see no problem with based on
https://www.lightbend.com/blog/running-a-2400-akka-nodes-cluster-on-google-compute-engine.

> Per my understanding, current ODL clustering is more like a disaster backup 
> solution for data store, I don't think it can work correctly if we have 128 
> nodes there.

I am not sure what that understanding is based on. CDS uses an
implementation of RAFT, which does not place artificial limits on the
number of participating nodes.

I do not see any design issue with deploying CDS on such a large number
of nodes. There may be bugs, but those are just bugs -- I do believe it
*will* work correctly.

> In cloud environment, tenants are dynamically creating and destroying VMs, 
> which will install and remove flows very often, openflow statistics is also a 
> not-small stress for openflow. Per current openflowplugin clustering, one ovs 
> node is connected to 3 odl nodes, these are permanent tcp connections, hoe 
> many ovs nodes can 3 odl nodes support at most? Anybody tested it, I think it 
> won't surpass 100.

That largely depends on what flows are loaded on the switches.

Yes, somebody tested it, and yes, it did surpass 100, thank you:
https://slides.com/dfarrell07/odl-perf/fullscreen#/1

> As I said, config inventory will have 2MB data in a 3 nodes environment, you 
> can evaluate how much data is there if we have 1 nodes, do you think 
> current ODL replication mechanism can work well?

As I wrote previously, this heavily depends on the structure of the
data, what the application does and how. It also depends on the software
being used.

To get definitive answers, I do suggest running some tests and
evaluating them.

> I know Pantheon has some commercial deployment in production environments, 
> can you tell us how many devices/nodes you can support at most in a 3 node 
> ODL cluster?

Not really, sorry.

Even if I could, the numbers depend on the particulars of a deployment
and I have precious little details about what is it exactly you are
doing and how -- and thus could not select the relevant data to share.

> Performance and scalability are two things, we always can get performance 
> improvement less or more by optimizing, but scalability is not so, we have to 
> redesign something to get scalability, any ODL developer ever considered how 
> ODL supports 1 nodes cloud environment? You are MDSAL experts, it will be 
> great if you can show us your insights about this here.

Yes, we have considered both horizontal and vertical scalability when we
architected the MD-SAL and yes, 10K (and more) nodes were considered and
taken into account.

Horizontal scalability usually means partitioning the working set in one
way or another -- how exactly you do that is part of the solution design.

Any decisions made here must be done with regard of the actual
application being run on the system, as data layout has strong interplay
with data access patterns.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

[controller-dev] 2019.06.04 Kernel Projects weekly meeting minutes

[Robert Varga]

Hello,

the minutes for this meeting are available at:

> [04/06/2019 18:26:34]  Meeting ended Tue Jun  4 16:26:33 2019 
> UTC.  Information about MeetBot at http://ci.openstack.org/meetbot.html . (v 
> 0.1.4)
> [04/06/2019 18:26:34]  Minutes:
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-06-04-16.09.html
> [04/06/2019 18:26:34]  Minutes (text): 
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-06-04-16.09.txt
> [04/06/2019 18:26:34]  Log:
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-06-04-16.09.log.html

Regards,
Robert
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--RmyWPuP73D1fJhzToaEsvbXCZlvJI1LkB
Content-Type: multipart/mixed; boundary="do4Z1Tgi0kc2IdOch42RWV1Ip31BI4rID";
 protected-headers="v1"
From: Robert Varga 
To: netconf-dev ,
 infrautils-...@lists.opendaylight.org,
 controller-dev ,
 mdsal-...@lists.opendaylight.org,
 yangtools-dev ,
 aaa-...@lists.opendaylight.org,
 odlparent-dev 
Message-ID: <81cba0e5-1d6f-7152-fff4-a767af46f...@hq.sk>
Subject: 2019.02.26 Kernel Projects weekly meeting minutes

--do4Z1Tgi0kc2IdOch42RWV1Ip31BI4rID
Content-Type: multipart/mixed;
 boundary="C6FF50AEC8D128829AEE3A18"
Content-Language: en-US

This is a multi-part message in MIME format.
--C6FF50AEC8D128829AEE3A18
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hello,

the minutes for this meeting are available at:

> [26/02/2019 18:58:58]  Meeting ended Tue Feb 26 17:58:58 2=
019 UTC.  Information about MeetBot at http://ci.openstack.org/meetbot.ht=
ml . (v 0.1.4)
> [26/02/2019 18:58:58]  Minutes:http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.html
> [26/02/2019 18:58:58]  Minutes (text): http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.txt
> [26/02/2019 18:58:58]  Log:http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.log.html

and also as an attachment to this message.

Regards,
Robert


--C6FF50AEC8D128829AEE3A18
Content-Type: text/html; charset=UTF-8;
 name="#opendaylight-meeting: Kernel Projects call.html"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="#opendaylight-meeting: Kernel Projects call.html"

PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMDEgVHJhbnNpdGlv
bmFsLy9FTiI+CjxodG1sPgo8aGVhZD4KPG1ldGEgaHR0cC1lcXVpdj0iQ29udGVudC10eXBl
IiBjb250ZW50PSJ0ZXh0L2h0bWw7Y2hhcnNldD1VVEYtOCI+Cjx0aXRsZT4jb3BlbmRheWxp
Z2h0LW1lZXRpbmc6IEtlcm5lbCBQcm9qZWN0cyBjYWxsPC90aXRsZT4KPHN0eWxlIHR5cGU9
InRleHQvY3NzIj4KLyogVGhpcyBpcyBmb3IgdGhlIC5odG1sIGluIHRoZSBIVE1MMiB3cml0
ZXIgKi8KYm9keSB7CiAgICBmb250LWZhbWlseTogSGVsdmV0aWNhLCBzYW5zLXNlcmlmOwog
ICAgZm9udC1zaXplOjE0cHg7Cn0KaDEgewogICAgdGV4dC1hbGlnbjogY2VudGVyOwp9CmEg
ewogICAgY29sb3I6bmF2eTsKICAgIHRleHQtZGVjb3JhdGlvbjogbm9uZTsKICAgIGJvcmRl
ci1ib3R0b206MXB4IGRvdHRlZCBuYXZ5Owp9CmE6aG92ZXIgewogICAgdGV4dC1kZWNvcmF0
aW9uOm5vbmU7CiAgICBib3JkZXItYm90dG9tOiAwOwogICAgY29sb3I6IzAwMDBCOTsKfQpo
ciB7CiAgICBib3JkZXI6IDFweCBzb2xpZCAjY2NjOwp9Ci8qIFRoZSAobmljaywgdGltZSkg
aXRlbSBwYWlycywgYW5kIG90aGVyIGJvZHkgdGV4dCB0aGluZ3MuICovCi5kZXRhaWxzIHsK
ICAgIGZvbnQtc2l6ZTogMTJweDsKICAgIGZvbnQtd2VpZ2h0OmJvbGQ7Cn0KLyogVGhlICdB
R1JFRUQ6JywgJ0lERUEnLCBldGMsIHByZWZpeCB0byBsaW5lcy4gKi8KLml0ZW10eXBlIHsK
ICAgIGZvbnQtc3R5bGU6IG5vcm1hbDsgICAgLyogdW4taXRhbGljcyBpdCAqLwogICAgZm9u
dC13ZWlnaHQ6IGJvbGQ7Cn0KLyogRXhhbXBsZTogY2hhbmdlIHNpbmdsZSBpdGVtIHR5cGVz
LiAgQ2FwaXRhbGl6ZWQgY29tbWFuZCBuYW1lLgovKiAuVE9QSUMgIHsgIGNvbG9yOm5hdnk7
ICB9ICovCi8qIC5BR1JFRUQgeyAgY29sb3I6bGltZTsgIH0gKi8KCjwvc3R5bGU+CjwvaGVh
ZD4KCjxib2R5Pgo8aDE+I29wZW5kYXlsaWdodC1tZWV0aW5nOiBLZXJuZWwgUHJvamVjdHMg
Y2FsbDwvaDE+CjxzcGFuIGNsYXNzPSJkZXRhaWxzIj4KTWVldGluZyBzdGFydGVkIGJ5IHJv
dmFyZ2EgYXQgMTc6MDY6MDMgVVRDCig8YSBocmVmPSJvcGVuZGF5bGlnaHQtbWVldGluZy1r
ZXJuZWxfcHJvamVjdHNfY2FsbC4yMDE5LTAyLTI2LTE3LjA2LmxvZy5odG1sIj5mdWxsIGxv
Z3M8L2E+KS48L3NwYW4+Cgo8YnI+PGJyPgoKCgo8aDM+TWVldGluZyBzdW1tYXJ5PC9oMz4K
PG9sPgo8bGk+PGIgY2xhc3M9IlRPUElDIj5vZGxwYXJlbnQtNS4wLjA8L2I+IDxzcGFuIGNs
YXNzPSJkZXRhaWxzIj4oPGEgaHJlZj0nb3BlbmRheWxpZ2h0LW1lZXRpbmcta2VybmVsX3By
b2plY3RzX2NhbGwuMjAxOS0wMi0yNi0xNy4wNi5sb2cuaHRtbCNsLTUnPnJvdmFyZ2E8L2E+
LCAxNzowNjowOCk8L3NwYW4+CjxvbCB0eXBlPSJhIj4KICA8bGk+PGEKICAgIGhyZWY9Imh0
dHBzOi8vamlyYS5vcGVuZGF5bGlnaHQub3JnL3Byb2plY3RzL09ETFBBUkVOVC92ZXJzaW9u
cy8xMDgyNSI+aHR0cHM6Ly9qaXJhLm9wZW5kYXlsaWdodC5vcmcvcHJvamVjdHMvT0RMUEFS
RU5UL3ZlcnNpb25zLzEwODI1PC9hPgogICAgaXMgdGhlIHRyYW

Re: [controller-dev] Questions Regarding Adding YANG 1.1 action support to controller

[Robert Varga]

On 04/06/2019 11:52, Emmett Cox wrote:
> Hi Everyone,

Hey Emmett,

> I'm currently working on making changes to the controller project to
> allow Yang 1.1 action support.
> 
> I've several questions regarding implementing the changes:
> 
> 1. In regards to where sal-remoterpc-connector is used/instantiated,
> does anyone know where/how it is utilized in opendaylight? I've search
> for references to the factory methods etc. used as part of the
> blueprints for the module within both the controller and mdsal project,
> and I've seen no mention of it anywhere. How is remoterpc-connector used
> alongside other modules? and where would I have to make additional
> changes in order to allow clustering support for remote actions, similar
> to remote rpcs?

The bundle is activated through its remote-rpc.xml blueprint. It is
packaged in odl-mdsal-broker. Aside from changes to blueprint I do not
believe anything else is needed.

> 2. Currently as part of these changes I've needed to use a class that
> functions in a similar way to DOMRpcIdentifier. I've been making use of
> the DOMActionInstance in any places where a domrpcidentifier would have
> been used for the action functionality - is this the correct class to be
> using, or is there a more appropriate class I could use to mirror
> DOMRpcIdentifier for actions?

DOMActionInstance is pretty much equivalent to DOMRpcIdentifier.Local,
except for the former's ability to be bound to multiple paths at the
same time (and hence availability listeners work slightly differently).

> 3.is there any data difference between normalizedNode vs ContainerNode?
> IE could I store normalized nodes and cast them to container nodes where
> needed without losing essential data? could be used to refactor some
> code to be more reusable.

ContainerNode is a NormalizedNode, aside from losing type safety nothing
is lost. Please consider using generics before resorting to explicit
casts, as they are hard to maintain...

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] 答复: Is Read from follower shard ok and openflowplugin master must be shard leader?

[Robert Varga]

On 31/05/2019 20:39, Anil Vishnoi wrote:
> Hi Yi,
> 
> Please see inline...
> 
> On Thu, May 30, 2019 at 5:04 PM Yi Yang (杨燚)-云服务集团
> mailto:yangy...@inspur.com>> wrote:

[trim]

> # Q2. Openflowplugin clustering also has master, per its document,
> only openflowplugin master node can do write operation against
> inventory data store, then what if this openflowplugin master node
> is follower shard?
> 
> OpenFlow plugin is driven by the devices connected to it, in the
> clustered setup. OpenFlow plugin allows you to connect your device to
> any of the controller node (one or more), and internally it will decide
> which node from the cluster will be the owner/master of the device using
> Cluster SIngleton Service + EOS. Once the owner/master is decided, that
> owner/master is the one allowed to write data to the "operational"
> inventory (plugin don't write to config inventory).

Note that there is an improvement possible to the latency of
master/backup selection here. It was presented by HPE way back a long
time ago, but it never made its way upstream.

[snip]

> # Q4. Anybody can recommend node number of a ODL cluster which will
> manage 1 compute/network nodes? I think leader nodes will have
> too high workload if number of ODL cluster node is too big so that
> it can’t do horizontal scale, per current default shard strategy,
> every node has all the data store, that looks more like data store
> replication, not distribute data store on all the nodes.
> 
> In my experience and opinion, ODL in clustered setup is not a solution
> here. As i mentioned above, with cluster setup i can think of two
> possible solution as i mentioned above. Deploying 20 cluster will be
> operational nightmare (E.g per cluster partition issues, device
> switching between cluster, device inventory data sharing across cluster
> on device switching etc). Apart from that you will need external
> mechanism to share the data between these clusters. And depends on your
> application, things can get even more complicated to maintain in
> production environment. If you go with the second option of 60 nodes in
> cluster, i am not even sure this cluster even will boot up properly :),
> let alone managing the devices. To make it work, you need to go with the
> prefix-based-sharding and cook a solution per device (per deivce shard,
> nodes where this shard can be replicated, making sure that device
> connection only switch to the node where the devie shard is replicated
> etc etc etc).

I think we need to drill down into assumptions and design a bit more:
- what is the mode of operation of OFP here? Does it include FRM?
- how many flows are expected to be programmed on each device?
- what is the expected stability of those flows?

Can you share details about your experience (ODL version, setup, numbers)?

At the end of the day, there is precious little hard data going around,
as the Performance Report is an orphan ever since Marcus left.
Refreshing that report is probably the low-hanging fruit here: Yi, are
you willing to organize and drive that effort?

> # Q5. Is it possible to run an asymmetric ODL cluster? I mean some
> nodes are full stack (there are netvirt, sfc, genius, etc), some
> nodes are southbound only (only install openflowplugin, ovsdb). I
> don’t think we must run other stuff in south bound device management
> nodes except southbound protocols.
> 
> I think you can do that, but if you want HA for your application and
> southbound plugins and also you want to run these in exlusion, 3 node
> cluster is not going to work (atleast you need 4 nodes in cluster). 

The number of total cluster nodes needs to be always odd, unless we are
talking externally managed active/passive deployment in which case in
needs to be 2 * n when n is an odd number larger than 2.

4 nodes is never a valid option. 3, 5, 6 (a/p), 7, 9, 10 (a/p), ... are
valid options.

> #Q7. Anybody can propose a good ODL clustering solution for a super
> scale data center which has 1 nodes?
> 
> In my experience, if you are looking for stable production environment
> with low operation cost (logistic, resource, support etc), ODL in
> "clustering" environment is probably not at-par solution. Luis and
> myself, shared some high level thoughts on how we can achieve this kind
> of scale using horizontally scalable system in the ONS summit. Here is
> the deck if you want to get more details.
> https://docs.google.com/presentation/d/1gDLHyyuh8VVRpzHpTq9GDkv4XKAe3EaSbm2uGJFTiO8/edit?usp=sharing

I do not want to sound dismissive, but those slides are general
guidelines without any validation -- there is no apples-to-apples
comparison between two implementations of the same use case, one
implemented as-is and one with the guidelines.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list

Re: [controller-dev] 答复: [mdsal-dev] 答复: Is Read from follower shard ok and openflowplugin master must be shard leader?

[Robert Varga]

On 03/06/2019 05:30, Yi Yang (杨燚)-云服务集团 wrote:
> Robert, thank you so much for your replies. Prefix-based shard was done by 
> CISCO guys in 2017,

Slight correction: Pantheon personnel on a contract.

> it seems nobody continues to do this since then, do you plan to roll it out 
> in 2019 because you're mdsal leader?

Well, there did not seem to be much interest nor support, so obviously
other things took precedence.

What we are talking about is not a single-person afternoon job and
unless an effort has committed engineering resources, I cannot promise
anything.

> About distributed shard, I mean data store of a module is separated into 
> several shards, every shard has different leader, so read write operations 
> are distributed into multiple nodes, cluster is not only for data 
> replication, but also for workload balance.

Yes, something in that direction is the idea.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

[controller-dev] Taking controller to MRI

[Robert Varga]

Hello everyone,

as noted at the DDF and https://jira.opendaylight.org/browse/TSC-207, I
would like to transition the controller project to being MRI.

There are multiple reasons for doing this:

- it is mandated by our goverernance since day one, it's just that our
processes were not up to par in the early days. That is no longer the
case, as the history of odlparent/yangtools/mdsal clearly shows.
- it will lower our infrastructure resource consumption (autorelease,
distcheck jobs)
- our codebase is very stable, with good mechanisms to support
versioning going back as far as Lithium, there simply is little pressure
to get changes out immediately
- being part of MRI cycle will improve ergonomics for downstreams, as we
will be entering the MRI version bump with controller fully ready

Unlike the previous MRI split-offs of odlparent/yangtools/mdsal, I do
not believe controller is ready to also transition to strict SemVer
rules -- we have not consolidated our codebase yet and we will need to
sweep our codebase at some point for defunct/unused remnants. I would
hate to see that effort being hampered by artificial rules.

That having been said, we should (and will, if I can help it) maintain
reasonable compatibility where our downstreams are concerned.

I am therefore raising the following vote for controller project
committers, please vote with usual +1/0/-1:

Do you agree to controller transitioning to being a Managed Release
Integrated project?

Thanks,
Robert, voting +1 :)



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] 答复: Is Read from follower shard ok and openflowplugin master must be shard leader?

[Robert Varga]

On 31/05/2019 02:03, Yi Yang (杨燚)-云服务集团 wrote:
> Also cc dev mailing list for getting more responses.
> 
>  
> 
> *发件人:*Yi Yang (杨燚)-云服务集团
> *发送时间:*2019年5月30日14:08
> *收件人:*'mdsal-...@lists.opendaylight.org'
> ;
> 'controller-dev@lists.opendaylight.org'
> ;
> 'openflowplugin-...@lists.opendaylight.org'
> 
> *抄送:*'robert.va...@pantheon.tech' ;
> 'tompante...@gmail.com' ;
> 'avish...@luminanetworks.com' ;
> 'abhijit.kumbh...@ericsson.com' 
> *主题:*Is Read from follower shard ok and openflowplugin master must be
> shard leader?
> *重要性:*高
> 
>  
> 
> Hi, folks
> 
>  
> 
> I have some questions about ODL clustering and openflowplugin
> clustering, look forward to getting your great help, thank you in advance.
> 
>  
> 
> # Q1. Is only leader node responsible for synchronizing data store to
> other followers for any shard?

Yes, as per RAFT.

> # Q2. Openflowplugin clustering also has master, per its document, only
> openflowplugin master node can do write operation against inventory data
> store, then what if this openflowplugin master node is follower shard?

It will talk to whoever the leader is.

> # Q3. Can we do more granular shard per openflow node(DPID) in
> inventory? I don’t think it makes sense that the inventory for one
> openflowplugin cluster is replicated to all the other openflowplugin
> clusters (assume there are many openflowplugin clusters because many
> south nodes/devices are there)

Architecturally, yes. The implementation has not been finished/hardened
enough for a rollout -- search for "prefix-based shard".

> # Q4. Anybody can recommend node number of a ODL cluster which will
> manage 1 compute/network nodes? I think leader nodes will have too
> high workload if number of ODL cluster node is too big so that it can’t
> do horizontal scale, per current default shard strategy, every node has
> all the data store, that looks more like data store replication, not
> distribute data store on all the nodes.

Not sure, it heavily depends on access patterns, etc. hence is more a
question to app engineering.

> # Q5. Is it possible to run an asymmetric ODL cluster? I mean some nodes
> are full stack (there are netvirt, sfc, genius, etc), some nodes are
> southbound only (only install openflowplugin, ovsdb). I don’t think we
> must run other stuff in south bound device management nodes except
> southbound protocols.

Infrastructure certainly allows for this.

> #Q6. I know data store read can be done in any node, but is it read from
> local shard in fact? Per document, it seems shard manager is doing this,
> if local shard is not leader, it will do this from remote shard leader.

Reads are always serviced from the leader, be it local or remote, as
that the only place we can reason about state.

There are asks to allow reading from follower, but to do that, someone
first must define what the semantic meaning of that data is supposed to
be -- a follower can be behind the leader, it could have stale/invalid
data, etc.

> #Q7. Anybody can propose a good ODL clustering solution for a super
> scale data center which has 1 nodes?

That depends on requirements.

> #Q8. Does shard size have any limitation? Per my evaluation from 3 nodes
> inventory, inventory for 2000 nodes will be more than 2G, for so big
> data set, can IMDS handle it very efficiently? I understand CDS is also
> using IMDS locally, right?

I am not sure what metric you mean under 'very efficiently'. CDS does
not use IMDS -- they are separate implementations of DOMStore
(MD-SAL-level construct). They both are using InMemoryDataTree
(yangtools-level construct).

> #Q9. Is it feasible to use distributed database as data store backend? I
> saw opencontrail/Tungsten Fabric is using Cassandra to save all the
> config data. It looks like a good idea to use existing database
> clustering solution, a big concern is it doesn’t support data change
> notification and listener, do we have some other way to do this for such
> databases?

That would be in scope of an integration plugin with such databases --
certainly alt-datastores project is looking into such things, so can
probably chime in.


> #Q10. Can we split data store of a module into more shards? I mean these
> shards include different data set, they form whole data store. I think
> this is a good way to do distributed data store and is more horizontally
> scalable. It will be better if every node which hosts data store shard
> can read it.

Is this a duplicate of Q3?

> #Q11. An application have one instance in every node, how does ODL
> decide which application instance to handle data change notification and
> data listener?

Depends on application integration.

For DataTreeChangeListener, only local events are delivered. For
ClusteredDataTreeChangeListener, all events are delivered.

DTCLs are delivered to all registrants.

> #12. How does an application use entity ownership service? What’s the
> difference between master, leader and EOS owner? I’m confused a bit.

EOS is a mediation 

[controller-dev] 2019.05.28 Kernel Projects weekly meeting minutes

[Robert Varga]

Hello,

the minutes for this meeting are available at:

> [28/05/2019 18:37:44]  Meeting ended Tue May 28 16:37:43 2019 
> UTC.  Information about MeetBot at http://ci.openstack.org/meetbot.html . (v 
> 0.1.4)
> [28/05/2019 18:37:44]  Minutes:
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-05-28-16.03.html
> [28/05/2019 18:37:44]  Minutes (text): 
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-05-28-16.03.txt
> [28/05/2019 18:37:44]  Log:
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-05-28-16.03.log.html

Regards,
Robert

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--RmyWPuP73D1fJhzToaEsvbXCZlvJI1LkB
Content-Type: multipart/mixed; boundary="do4Z1Tgi0kc2IdOch42RWV1Ip31BI4rID";
 protected-headers="v1"
From: Robert Varga 
To: netconf-dev ,
 infrautils-...@lists.opendaylight.org,
 controller-dev ,
 mdsal-...@lists.opendaylight.org,
 yangtools-dev ,
 aaa-...@lists.opendaylight.org,
 odlparent-dev 
Message-ID: <81cba0e5-1d6f-7152-fff4-a767af46f...@hq.sk>
Subject: 2019.02.26 Kernel Projects weekly meeting minutes

--do4Z1Tgi0kc2IdOch42RWV1Ip31BI4rID
Content-Type: multipart/mixed;
 boundary="C6FF50AEC8D128829AEE3A18"
Content-Language: en-US

This is a multi-part message in MIME format.
--C6FF50AEC8D128829AEE3A18
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hello,

the minutes for this meeting are available at:

> [26/02/2019 18:58:58]  Meeting ended Tue Feb 26 17:58:58 2=
019 UTC.  Information about MeetBot at http://ci.openstack.org/meetbot.ht=
ml . (v 0.1.4)
> [26/02/2019 18:58:58]  Minutes:http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.html
> [26/02/2019 18:58:58]  Minutes (text): http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.txt
> [26/02/2019 18:58:58]  Log:http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.log.html

and also as an attachment to this message.

Regards,
Robert


--C6FF50AEC8D128829AEE3A18
Content-Type: text/html; charset=UTF-8;
 name="#opendaylight-meeting: Kernel Projects call.html"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="#opendaylight-meeting: Kernel Projects call.html"

PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMDEgVHJhbnNpdGlv
bmFsLy9FTiI+CjxodG1sPgo8aGVhZD4KPG1ldGEgaHR0cC1lcXVpdj0iQ29udGVudC10eXBl
IiBjb250ZW50PSJ0ZXh0L2h0bWw7Y2hhcnNldD1VVEYtOCI+Cjx0aXRsZT4jb3BlbmRheWxp
Z2h0LW1lZXRpbmc6IEtlcm5lbCBQcm9qZWN0cyBjYWxsPC90aXRsZT4KPHN0eWxlIHR5cGU9
InRleHQvY3NzIj4KLyogVGhpcyBpcyBmb3IgdGhlIC5odG1sIGluIHRoZSBIVE1MMiB3cml0
ZXIgKi8KYm9keSB7CiAgICBmb250LWZhbWlseTogSGVsdmV0aWNhLCBzYW5zLXNlcmlmOwog
ICAgZm9udC1zaXplOjE0cHg7Cn0KaDEgewogICAgdGV4dC1hbGlnbjogY2VudGVyOwp9CmEg
ewogICAgY29sb3I6bmF2eTsKICAgIHRleHQtZGVjb3JhdGlvbjogbm9uZTsKICAgIGJvcmRl
ci1ib3R0b206MXB4IGRvdHRlZCBuYXZ5Owp9CmE6aG92ZXIgewogICAgdGV4dC1kZWNvcmF0
aW9uOm5vbmU7CiAgICBib3JkZXItYm90dG9tOiAwOwogICAgY29sb3I6IzAwMDBCOTsKfQpo
ciB7CiAgICBib3JkZXI6IDFweCBzb2xpZCAjY2NjOwp9Ci8qIFRoZSAobmljaywgdGltZSkg
aXRlbSBwYWlycywgYW5kIG90aGVyIGJvZHkgdGV4dCB0aGluZ3MuICovCi5kZXRhaWxzIHsK
ICAgIGZvbnQtc2l6ZTogMTJweDsKICAgIGZvbnQtd2VpZ2h0OmJvbGQ7Cn0KLyogVGhlICdB
R1JFRUQ6JywgJ0lERUEnLCBldGMsIHByZWZpeCB0byBsaW5lcy4gKi8KLml0ZW10eXBlIHsK
ICAgIGZvbnQtc3R5bGU6IG5vcm1hbDsgICAgLyogdW4taXRhbGljcyBpdCAqLwogICAgZm9u
dC13ZWlnaHQ6IGJvbGQ7Cn0KLyogRXhhbXBsZTogY2hhbmdlIHNpbmdsZSBpdGVtIHR5cGVz
LiAgQ2FwaXRhbGl6ZWQgY29tbWFuZCBuYW1lLgovKiAuVE9QSUMgIHsgIGNvbG9yOm5hdnk7
ICB9ICovCi8qIC5BR1JFRUQgeyAgY29sb3I6bGltZTsgIH0gKi8KCjwvc3R5bGU+CjwvaGVh
ZD4KCjxib2R5Pgo8aDE+I29wZW5kYXlsaWdodC1tZWV0aW5nOiBLZXJuZWwgUHJvamVjdHMg
Y2FsbDwvaDE+CjxzcGFuIGNsYXNzPSJkZXRhaWxzIj4KTWVldGluZyBzdGFydGVkIGJ5IHJv
dmFyZ2EgYXQgMTc6MDY6MDMgVVRDCig8YSBocmVmPSJvcGVuZGF5bGlnaHQtbWVldGluZy1r
ZXJuZWxfcHJvamVjdHNfY2FsbC4yMDE5LTAyLTI2LTE3LjA2LmxvZy5odG1sIj5mdWxsIGxv
Z3M8L2E+KS48L3NwYW4+Cgo8YnI+PGJyPgoKCgo8aDM+TWVldGluZyBzdW1tYXJ5PC9oMz4K
PG9sPgo8bGk+PGIgY2xhc3M9IlRPUElDIj5vZGxwYXJlbnQtNS4wLjA8L2I+IDxzcGFuIGNs
YXNzPSJkZXRhaWxzIj4oPGEgaHJlZj0nb3BlbmRheWxpZ2h0LW1lZXRpbmcta2VybmVsX3By
b2plY3RzX2NhbGwuMjAxOS0wMi0yNi0xNy4wNi5sb2cuaHRtbCNsLTUnPnJvdmFyZ2E8L2E+
LCAxNzowNjowOCk8L3NwYW4+CjxvbCB0eXBlPSJhIj4KICA8bGk+PGEKICAgIGhyZWY9Imh0
dHBzOi8vamlyYS5vcGVuZGF5bGlnaHQub3JnL3Byb2plY3RzL09ETFBBUkVOVC92ZXJzaW9u
cy8xMDgyNSI+aHR0cHM6Ly9qaXJhLm9wZW5kYXlsaWdodC5vcmcvcHJvamVjdHMvT0RMUEFS
RU5UL3ZlcnNpb25zLzEwODI1PC9hPgogICAgaXMgdGhlIHRyYW

[controller-dev] 2019.05.21 Kernel Projects weekly meeting minutes

[Robert Varga]

Hello,

the minutes for this meeting are available at:

> [21/05/2019 18:53:56]  Meeting ended Tue May 21 16:53:55 2019 
> UTC.  Information about MeetBot at http://ci.openstack.org/meetbot.html . (v 
> 0.1.4)
> [21/05/2019 18:53:56]  Minutes:
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-05-21-16.04.html
> [21/05/2019 18:53:56]  Minutes (text): 
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-05-21-16.04.txt
> [21/05/2019 18:53:56]  Log:
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-05-21-16.04.log.html

Sorry I have missed sending some of these out :( You can find the
previous ones at
http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/

Regards,
Robert
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--RmyWPuP73D1fJhzToaEsvbXCZlvJI1LkB
Content-Type: multipart/mixed; boundary="do4Z1Tgi0kc2IdOch42RWV1Ip31BI4rID";
 protected-headers="v1"
From: Robert Varga 
To: netconf-dev ,
 infrautils-...@lists.opendaylight.org,
 controller-dev ,
 mdsal-...@lists.opendaylight.org,
 yangtools-dev ,
 aaa-...@lists.opendaylight.org,
 odlparent-dev 
Message-ID: <81cba0e5-1d6f-7152-fff4-a767af46f...@hq.sk>
Subject: 2019.02.26 Kernel Projects weekly meeting minutes

--do4Z1Tgi0kc2IdOch42RWV1Ip31BI4rID
Content-Type: multipart/mixed;
 boundary="C6FF50AEC8D128829AEE3A18"
Content-Language: en-US

This is a multi-part message in MIME format.
--C6FF50AEC8D128829AEE3A18
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hello,

the minutes for this meeting are available at:

> [26/02/2019 18:58:58]  Meeting ended Tue Feb 26 17:58:58 2=
019 UTC.  Information about MeetBot at http://ci.openstack.org/meetbot.ht=
ml . (v 0.1.4)
> [26/02/2019 18:58:58]  Minutes:http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.html
> [26/02/2019 18:58:58]  Minutes (text): http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.txt
> [26/02/2019 18:58:58]  Log:http://meetings.ope=
ndaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight=
-meeting-kernel_projects_call.2019-02-26-17.06.log.html

and also as an attachment to this message.

Regards,
Robert


--C6FF50AEC8D128829AEE3A18
Content-Type: text/html; charset=UTF-8;
 name="#opendaylight-meeting: Kernel Projects call.html"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="#opendaylight-meeting: Kernel Projects call.html"

PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMDEgVHJhbnNpdGlv
bmFsLy9FTiI+CjxodG1sPgo8aGVhZD4KPG1ldGEgaHR0cC1lcXVpdj0iQ29udGVudC10eXBl
IiBjb250ZW50PSJ0ZXh0L2h0bWw7Y2hhcnNldD1VVEYtOCI+Cjx0aXRsZT4jb3BlbmRheWxp
Z2h0LW1lZXRpbmc6IEtlcm5lbCBQcm9qZWN0cyBjYWxsPC90aXRsZT4KPHN0eWxlIHR5cGU9
InRleHQvY3NzIj4KLyogVGhpcyBpcyBmb3IgdGhlIC5odG1sIGluIHRoZSBIVE1MMiB3cml0
ZXIgKi8KYm9keSB7CiAgICBmb250LWZhbWlseTogSGVsdmV0aWNhLCBzYW5zLXNlcmlmOwog
ICAgZm9udC1zaXplOjE0cHg7Cn0KaDEgewogICAgdGV4dC1hbGlnbjogY2VudGVyOwp9CmEg
ewogICAgY29sb3I6bmF2eTsKICAgIHRleHQtZGVjb3JhdGlvbjogbm9uZTsKICAgIGJvcmRl
ci1ib3R0b206MXB4IGRvdHRlZCBuYXZ5Owp9CmE6aG92ZXIgewogICAgdGV4dC1kZWNvcmF0
aW9uOm5vbmU7CiAgICBib3JkZXItYm90dG9tOiAwOwogICAgY29sb3I6IzAwMDBCOTsKfQpo
ciB7CiAgICBib3JkZXI6IDFweCBzb2xpZCAjY2NjOwp9Ci8qIFRoZSAobmljaywgdGltZSkg
aXRlbSBwYWlycywgYW5kIG90aGVyIGJvZHkgdGV4dCB0aGluZ3MuICovCi5kZXRhaWxzIHsK
ICAgIGZvbnQtc2l6ZTogMTJweDsKICAgIGZvbnQtd2VpZ2h0OmJvbGQ7Cn0KLyogVGhlICdB
R1JFRUQ6JywgJ0lERUEnLCBldGMsIHByZWZpeCB0byBsaW5lcy4gKi8KLml0ZW10eXBlIHsK
ICAgIGZvbnQtc3R5bGU6IG5vcm1hbDsgICAgLyogdW4taXRhbGljcyBpdCAqLwogICAgZm9u
dC13ZWlnaHQ6IGJvbGQ7Cn0KLyogRXhhbXBsZTogY2hhbmdlIHNpbmdsZSBpdGVtIHR5cGVz
LiAgQ2FwaXRhbGl6ZWQgY29tbWFuZCBuYW1lLgovKiAuVE9QSUMgIHsgIGNvbG9yOm5hdnk7
ICB9ICovCi8qIC5BR1JFRUQgeyAgY29sb3I6bGltZTsgIH0gKi8KCjwvc3R5bGU+CjwvaGVh
ZD4KCjxib2R5Pgo8aDE+I29wZW5kYXlsaWdodC1tZWV0aW5nOiBLZXJuZWwgUHJvamVjdHMg
Y2FsbDwvaDE+CjxzcGFuIGNsYXNzPSJkZXRhaWxzIj4KTWVldGluZyBzdGFydGVkIGJ5IHJv
dmFyZ2EgYXQgMTc6MDY6MDMgVVRDCig8YSBocmVmPSJvcGVuZGF5bGlnaHQtbWVldGluZy1r
ZXJuZWxfcHJvamVjdHNfY2FsbC4yMDE5LTAyLTI2LTE3LjA2LmxvZy5odG1sIj5mdWxsIGxv
Z3M8L2E+KS48L3NwYW4+Cgo8YnI+PGJyPgoKCgo8aDM+TWVldGluZyBzdW1tYXJ5PC9oMz4K
PG9sPgo8bGk+PGIgY2xhc3M9IlRPUElDIj5vZGxwYXJlbnQtNS4wLjA8L2I+IDxzcGFuIGNs
YXNzPSJkZXRhaWxzIj4oPGEgaHJlZj0nb3BlbmRheWxpZ2h0LW1lZXRpbmcta2VybmVsX3By
b2plY3RzX2NhbGwuMjAxOS0wMi0yNi0xNy4wNi5sb2cuaHRtbCNsLTUnPnJvdmFyZ2E8L2E+
LCAxNzowNjowOCk8L3NwYW4+CjxvbCB0eXBlPSJhIj4KICA8bGk+PGEKICAgIG

Re: [controller-dev] [netconf-dev] Help regarding Yang 1.1 actions

[Robert Varga]

On 13/05/2019 13:49, Ajay Deep Singh wrote:
> Hi Robert,

Hey Ajay,

> 
> Thank you for mail, we are going through all of them trying to understand 
> more on Odl.
> 
> Regarding adding support for Yang 1.1 Action in RestConf please correct my 
> understanding if its wrong as explained below, Going with you suggestion for 
> "RESTCONF the wiring from HTTP to DOMActionService is missing (that would sit 
> in restconf-nb-rfc8040)".  
> 
> Steps am following :
> 
> 1. Inject service "DOMActionService" in Rfc8040RestConfWiring.java in 
> restconf.nb.rfc8040 .

Almost correct. You will need the equivalent of
org.opendaylight.restconf.nb.rfc8040.handlers.RpcServiceHandler
specialized for DOMActionService and inject that.

> 2. In ServicesWrapper under restconf.nb.rfc8040.services.wrapper calling 
> domActionService.getExtensions().getInstance(DOMYangTextSourceProvider.class) 
> here we need to add new interface in MD-SAL named : 
> DOMYangActionTextSourceProvider extending DOMActionServiceExtension to 
> support new yang Model supporting action, if that the case we need to add 
> more code in MD-Sal is that am going in correct direction.?
> 3. If above understanding is correct few changes need to be added in 
> mdsal-dom-sip in abstract class  AbstractDOMSchemaService and more which I 
> need to figure out.

Wrong service :) DOMSchemaService takes care of propagating
SchemaContext, which services all other services -- you do not need to
muck with it, as all that would be required is already present in
SchemaContext (and the SchemaNode tree it holds).

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease fluorine-mvn35-openjdk8 failed to build sal-distributed-datastore from controller

[Robert Varga]

On 09/05/2019 03:25, Jenkins wrote:
> Attention controller-devs,
> 
> Autorelease fluorine-mvn35-openjdk8 failed to build sal-distributed-datastore 
> from controller in build
> 108. Attached is a snippet of the error message related to the
> failure that we were able to automatically parse as well as console logs. 
> 
> 
> Console Logs:
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-fluorine-mvn35-openjdk8/108

*sigh* this test is getting unstable. I'll try to look at what's going
on next week.

Regards,
Robert




signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [netconf-dev] Help regarding Yang 1.1 actions

[Robert Varga]

On 09/05/2019 13:27, Martin Sandberg wrote:
> Hi Ajay,
> 
>  
> 
> Sorry, I can’t really describe the signaling sequence more than that. I
> have basically no knowledge about the code, I have only tested ODL a
> bit, and observed the behavior I described below. I don’t actually know
> for a fact how ODL reads up the models from the node after the Hello
> message exchange, it’s just an assumption that a  on RFC 7895
> /modules-state is done, and then uploads each of those models. Hopefully
> others who knows the code much better can answer detailed questions.

Hello,

the issue here is discovery of what modules does a particular server
support, which we need to know in order to form our understanding of the
data that can be exchanged with it.

The baseline specification here is RFC6020, which requires all models to
be announced: https://tools.ietf.org/html/rfc6020#section-5.6.4

This obviously is inefficient and does not support transfer of schemas,
which is required for seamless connection establishment. An alternative
is RFC6022 (ietf-netconf-monitoring), which optional to implement.

This mistake was rectified in YANG 1.1 (RFC7950), which refers to
RFC7895 (ietf-yang-library) to provide capabilities similar to RFC6022.

Hence our connection bootstrap process occurs in two phases, where the
first phase bootstraps the basic understanding of models based on
advertisement in , which allows us to start talking to the
device -- i.e. interact with RFC6022/RFC7895, at which point we
establish the connection based on the combined understanding of the
device's schema.

> If you turn on detailed tracing, you can maybe see the exact signaling
> sequence.

Yup, the sequence can be inferred at TRACE level, where netconf becomes
very chatty about what it is doing.

> Since the trace is only a WARNing, I guess it shouldn’t be treated as
> something that is breaking anything, but I agree it could maybe have
> sufficed as a INFO trace.

Actually, the code should be re-visited to suppress the warning in case
of :yang-library(:1.1) capability presence, as that is a promise that
yang-library is available and the server is following RFC7950 specification.

If the capability is not present, the warning still has to be present
there, as in that case we are dealing with an inconsistent
RFC6020/RFC6022 server.

> I think our reason to only advertise the basic capabilities in the Hello
> message is to be able to protect access to more sensitive
> application-specific models via  on /modules-state by applying
> access control mechanisms on that part of the model.

That, and also to keep the  message small.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] org.opendaylight.controller:sal-binding-api ver. 1.9.1

[Robert Varga]

On 29/04/2019 06:03, ylhs...@itri.org.tw wrote:
> Hi controller-dev,
> 
>  
> 
> We (SNMP4SDN, self-managed project) needs the artifact
> org.opendaylight.controller:sal-binding-api ver. 1.9.1 for Neon SR1. So
> far in Nexus there is only ver. 1.9.0 [1], will you release ver. 1.9.1
> in Neon SR1? Just confirm with you in case we will fail to make it.

Yes, that artifact will appear once MSI projects release Neon SR1 ...

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [netconf-dev] Help regarding Yang 1.1 actions

[Robert Varga]

On 25/04/2019 12:11, Emmett Cox wrote:
> 
> 
> Hi everyone,

Hello,

> 
> Myself and several members of my team are investigating what changes we 
> would need to implement to allow restconf, mdsal and netconf to support 
> Yang 1.1 actions, both in a clustered and non-clustered way.

YANG Tools and MD-SAL support is there, on both DOM and Binding layers,
supported by

org.opendaylight.mdsal.dom.api.DOMActionService
org.opendaylight.mdsal.dom.api.DOMActionProviderService
org.opendaylight.mdsal.binding.api.ActionService
org.opendaylight.mdsal.binding.api.ActionProviderService

and their respective JVM-local implementations.

> To that end we've been trying to find documentation or guides regarding 
> the various ODL modules relating to restconf, mdsal and netconf and 
> determine which modules we need to change to get our use case working.
> 
> Would any of you be able to point us in the direction of anything we 
> could use to understand the modules, and where we could make a start 
> regarding the changes we'll need to make?

I am not sure about clustering support, that would be part of
controller's sal-remoterpc-connector -- which needs to hook onto the
DOMActionService/DOMActionProviderService instances available in Service
Registry. I think the gossiping protocol should support advertizing
actions without a change, but that needs to be checked out.

For NETCONF, I think the only piece missing is the translator in ...
sal-netconf-connector (and probably mdsal-netconf-connector, I always
confuse the two).

For RESTCONF, the old spec does not support actions, so there's nothing
to be done for that. For the RFC8040 implementation, I think the wiring
from HTTP to DOMActionService is missing (that would sit in
restconf-nb-rfc8040).

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease fluorine-mvn35-openjdk8 failed to build sal-distributed-datastore from controller

[Robert Varga]

On 09/04/2019 03:22, Jenkins wrote:
> Attention controller-devs,
> 
> Autorelease fluorine-mvn35-openjdk8 failed to build sal-distributed-datastore 
> from controller in build
> 78. Attached is a snippet of the error message related to the
> failure that we were able to automatically parse as well as console logs. 
> 
> 
> Console Logs:
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-fluorine-mvn35-openjdk8/78

> testOwnerChangesOnPeerAvailabilityChanges(org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipShardTest)
>   Time elapsed: 8.15 sec  <<< FAILURE!
> org.junit.ComparisonFailure: getRaftState expected:<[]Leader> but 
> was:<[Pre]Leader>
>   at org.junit.Assert.assertEquals(Assert.java:115)
>   at 
> org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipShardTest.lambda$testOwnerChangesOnPeerAvailabilityChanges$4(EntityOwnershipShardTest.java:656)

Looks like the minion was way too slow and state settle in 5 seconds
(PreLeader indicates things were on their way).

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Netvirt 3 node CSIT HA Tests - failure debugging

[Robert Varga]

On 13/03/2019 08:38, Faseela K wrote:
> +Robert
> 
> Robert,

Hello,

> 
>     Jaya is trying the below steps in a 3 node ODL Cluster :
> 
>  1. Take Down ODL1 and ODL2
>  1. Check all VM connectivities are intact even if 2 nodes are down
>  2. Bring Up ODL1 and ODL2, sequentially as below
> 
> a.  ClusterManagement.Start Single Member    /1    msg=up: ODL3, down:
> ODL1, ODL2    wait_for_sync=False/
> 
> b.  ClusterManagement.Start Single Member    /2    msg=up: ODL1, ODL3,
> down: ODL2/
> 
>  3. Take Down ODL2 and ODL3, sequentially
> 
> Once step 3 is performed, many of the subsequent tests fail, before we
> dig into the details of the specific test cases that fail, wanted to
> know whether the above procedure of

I am not sure what you mean. With two out of three nodes out the cluster
cannot make any progress...

> Bringing UP and Bringing Down ODL nodes are supported, and expected to
> work, or do we have to add any additional delay?

It certainly is supported and is expected to work. Note that it can take
upto 2 x leader election timeout (not sure what it's set to, can be
seconds to tens of seconds) for CDS to stabilized after such an excercise.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Cluster problem

[Robert Varga]

On 08/03/2019 09:05, Luis Gomez wrote:
> Hi controller experts,
> 
> I see sporadically  this problem in one of our cluster deployments (Fluorine 
> SR1):
> 
> 1) Operational DS gets locked in member-1 (leader of 
> shard-default-operational) after stopping member-2:
> 
> 2019-03-01T14:49:05,189 | INFO  | 
> opendaylight-cluster-data-akka.actor.default-dispatcher-19 | RpcRegistry  
> | 225 - org.opendaylight.controller.sal-clustering-commons - 
> 1.8.1 | Actor termination 
> Terminated(Actor[akka.tcp://opendaylight-cluster-data@172.18.1.12:2550/user/rpc/broker#-938891863])
>  received
> ...
> 2019-03-01T14:49:16,900 | WARN  | epollEventLoopGroup-11-7 | 
> TransactionContextWrapper| 233 - 
> org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | Failed to 
> acquire enqueue operation permit for transaction 
> member-1-datastore-operational-fe-8-chn-4-txn-23-0 on shard default
> 2019-03-01T14:49:16,901 | WARN  | epollEventLoopGroup-11-8 | 
> TransactionContextWrapper| 233 - 
> org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | Failed to 
> acquire enqueue operation permit for transaction 
> member-1-datastore-operational-fe-8-chn-5-txn-24-0 on shard default
> …
> 2019-03-01T14:50:06,051 | WARN  | epollEventLoopGroup-11-4 | 
> TransactionContextWrapper| 233 - 
> org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | Failed to 
> acquire enqueue operation permit for transaction 
> member-1-datastore-operational-fe-8-chn-21-txn-2-0 on shard default

This is backpressure being applied on frontend ...

> 2019-03-01T14:50:06,385 | WARN  | epollEventLoopGroup-11-2 | 
> TransactionContextWrapper| 233 - 
> org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | Failed to 
> acquire enqueue operation permit for transaction 
> member-1-datastore-operational-fe-8-chn-32-txn-2-0 on shard default
> 
> 2) A few seconds later I see A DS transaction getting stuck so maybe this 
> explain why DS oper DS is locked:
> 
> 019-03-01T14:49:24,032 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-70 | ShardDataTree 
>| 233 - org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | 
> member-1-shard-default-operational: Current transaction 
> member-1-datastore-operational-fe-8-txn-1101-0 has timed out after 15707 ms 
> in state COMMIT_PENDING
> 2019-03-01T14:49:24,033 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-70 | ShardDataTree 
>| 233 - org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | 
> member-1-shard-default-operational: Transaction 
> member-1-datastore-operational-fe-8-txn-1101-0 is still committing, cannot 
> abort
> 2019-03-01T14:49:38,333 | WARN  | 
> opendaylight-cluster-data-akka.actor.default-dispatcher-46 | 
> LocalThreePhaseCommitCohort  | 233 - 
> org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | Failed to 
> prepare transaction member-1-datastore-operational-fe-8-txn-1101-0 on backend
> 2019-03-01T14:49:44,032 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-36 | ShardDataTree 
>| 233 - org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | 
> member-1-shard-default-operational: Current transaction 
> member-1-datastore-operational-fe-8-txn-1101-0 has timed out after 2 ms 
> in state COMMIT_PENDING
> 2019-03-01T14:49:44,033 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-36 | ShardDataTree 
>| 233 - org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | 
> member-1-shard-default-operational: Transaction 
> member-1-datastore-operational-fe-8-txn-1101-0 is still committing, cannot 
> abort
> 2019-03-01T14:49:59,033 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-59 | ShardDataTree 
>| 233 - org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | 
> member-1-shard-default-operational: Current transaction 
> member-1-datastore-operational-fe-8-txn-1101-0 has timed out after 15000 ms 
> in state COMMIT_PENDING
> 2019-03-01T14:49:59,033 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-59 | ShardDataTree 
>| 233 - org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | 
> member-1-shard-default-operational: Transaction 
> member-1-datastore-operational-fe-8-txn-1101-0 is still committing, cannot 
> abort
> 2019-03-01T14:50:14,033 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-58 | ShardDataTree 
>| 233 - org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | 
> member-1-shard-default-operational: Current transaction 
> member-1-datastore-operational-fe-8-txn-1101-0 has timed out after 15000 ms 
> in state COMMIT_PENDING
> 2019-03-01T14:50:14,033 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-58 | ShardDataTree 
>| 233 - org.opendaylight.controller.sal-distributed-datastore - 1.8.1 | 
> member-1-shard-default-operational: 

Re: [controller-dev] Compilation errors -- mdsal

[Robert Varga]

On 06/03/2019 10:01, Vikram Darsi wrote:
> Hi Team
> 
>  
> 
> Getting compilation errors on tagged version *of mdsal project*:
> 
> git checkout tags/release/oxygen-sr4 -b oxygen-sr4

[...]

> [ERROR] Failed to execute goal
> org.eclipse.xtend:xtend-maven-plugin:2.14.0:compile (default) on project
> mdsal-binding-java-api-generator: Execution default of goal
> org.eclipse.xtend:xtend-maven-plugin:2.14.0:compile failed: An API
> incompatibility was encountered while executing
> org.eclipse.xtend:xtend-maven-plugin:2.14.0:compile:
> java.lang.NoSuchMethodError:
> org.eclipse.jdt.internal.compiler.batch.FileSystem.getClasspath(Ljava/lang/String;Ljava/lang/String;ZLorg/eclipse/jdt/internal/compiler/env/AccessRuleSet;Ljava/lang/String;Ljava/util/Map;)Lorg/eclipse/jdt/internal/compiler/batch/FileSystem$Classpath;
> 
> [ERROR] -
> 
> [ERROR] realm =    plugin>org.eclipse.xtend:xtend-maven-plugin:2.14.0
> 
> [ERROR] strategy =
> org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
> 
> [ERROR] urls[0] =
> file:/root/.m2/repository/org/eclipse/xtend/xtend-maven-plugin/2.14.0/xtend-maven-plugin-2.14.0.jar
> 
> [ERROR] urls[1] =
> file:/root/.m2/repository/org/eclipse/platform/org.eclipse.equinox.common/3.10.0/org.eclipse.equinox.common-3.10.0.jar

Upstream issue, resolved on stable/oxygen branch. The release tag is
fixed, obviously and besides ... why would you need to rebuild it? The
artifacts are in nexus...

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] DOMRpcImplementationNotAvailableException

[Robert Varga]

On 04/03/2019 14:22, Vikram Darsi wrote:
> Hi Robert
> 
> We are using Boron-SR4 (Pretty old ☹), here is the exception: 
> https://pastebin.com/REP0ZuSx, Will the DOMRpcRouter/table on the problematic 
> node will eventually get updated with the missed RPCRegistration events? From 
> the logs it did not happen event after a long time.

It should, but there have been bugs in that area of code, which were
rectified by a rather thorough rewrite in Carbon.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] DOMRpcImplementationNotAvailableException

[Robert Varga]

On 04/03/2019 13:37, Vikram Darsi wrote:
> Hi Team

Hello,

> 1. We have a RoutedRpc registered on one of the clustered nodes using
> ClusterSingletonService.
> 
> 2. ClusterSingletonService is up and is shown available on member-2
> (verified ->
> http://{{ipaddr}}:8181/restconf/operational/entity-owners:entity-owners)
> 
> 3. While invoking API on registered routed rpc from member-1, we are
> getting DOMRpcImplementationNotAvailableException
> 
>    invoke same API from member-3, invocation is successful.
> 
> 4. What is wrong with member-1? if this the case, what is the way to
> recover?

Well.. the exception usually includes a hint as to what is going on.
Depending on software version, which you do not mention here, this could
either be a deployment issue, a bug or expected behavior.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease sodium-mvn35-openjdk11 failed to build sal-akka-segmented-journal from controller

[Robert Varga]

On 04/03/2019 12:56, Robert Varga wrote:
> 
> On 04/03/2019 01:51, Jenkins wrote:
>> Attention controller-devs,
>>
>> Autorelease sodium-mvn35-openjdk11 failed to build 
>> sal-akka-segmented-journal from controller in build
>> 27. Attached is a snippet of the error message related to the
>> failure that we were able to automatically parse as well as console logs. 
>>
>>
>> Console Logs:
>> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-sodium-mvn35-openjdk11/27
> Interesting, this test seems to be consistently failing with CME on
> JDK11, but passes on JDK8. I will investigate.

https://git.opendaylight.org/gerrit/80665

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease sodium-mvn35-openjdk11 failed to build sal-akka-segmented-journal from controller

[Robert Varga]

On 04/03/2019 01:51, Jenkins wrote:
> Attention controller-devs,
> 
> Autorelease sodium-mvn35-openjdk11 failed to build sal-akka-segmented-journal 
> from controller in build
> 27. Attached is a snippet of the error message related to the
> failure that we were able to automatically parse as well as console logs. 
> 
> 
> Console Logs:
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-sodium-mvn35-openjdk11/27

Interesting, this test seems to be consistently failing with CME on
JDK11, but passes on JDK8. I will investigate.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Missing sal-binding-api ver 1.9.0

[Robert Varga]

On 04/03/2019 05:14, ylhs...@itri.org.tw wrote:
> Hi controller-dev folk,
> 
>  
> 
> Excuse me. The snmp4sdn project depends on sal-binding-api ver.
> 1.9.0-SNAPSHOT [1], however for Neon release, ver. 1.9.0 would be
> required, but we can’t find the 1.9.0 artifact in nexus [2] so that we
> can’t complete the release process. Do you plan to release it?

Hello Christine,

These artifacts are part of MSI projects and therefore will be released
with the autorelease build -- which is currently in the sign-off stage.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] OFP cluster test with "tell-based"

[Robert Varga]

On 26/02/2019 21:52, Luis Gomez wrote:
> FYI tell-based test with latest master:
> 
> https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-3node-gate-clustering-only-sodium/9/
> 
> I still see topology link information is not cleared after node/link down in 
> some random tests.

There does not seem to be anything out of the ordinary from CDS
perspective, but there are some OLFE/validation errors.

Can someone from the OFP team take a look, please?

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

[controller-dev] 2019.02.26 Kernel Projects weekly meeting minutes

[Robert Varga]

Hello,

the minutes for this meeting are available at:

> [26/02/2019 18:58:58]  Meeting ended Tue Feb 26 17:58:58 2019 
> UTC.  Information about MeetBot at http://ci.openstack.org/meetbot.html . (v 
> 0.1.4)
> [26/02/2019 18:58:58]  Minutes:
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-02-26-17.06.html
> [26/02/2019 18:58:58]  Minutes (text): 
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-02-26-17.06.txt
> [26/02/2019 18:58:58]  Log:
> http://meetings.opendaylight.org/opendaylight-meeting/2019/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2019-02-26-17.06.log.html

and also as an attachment to this message.

Regards,
Robert

Title: #opendaylight-meeting: Kernel Projects call





#opendaylight-meeting: Kernel Projects call

Meeting started by rovarga at 17:06:03 UTC
(full logs).





Meeting summary

odlparent-5.0.0 (rovarga, 17:06:08)

  https://jira.opendaylight.org/projects/ODLPARENT/versions/10825
is the tracker (rovarga,
17:06:16)
  most of the targeted issues are in
review (rovarga,
17:06:27)
  there are couple of patches we want in 4.0.10,
will wait for them to land before branching (rovarga,
17:06:44)
  not sure about
https://jira.opendaylight.org/browse/ODLPARENT-176 (rovarga,
17:08:26)
  the bundle name is useful in general
(rovarga,
17:08:37)
  and this can always be tuned in a particular
deployment (rovarga,
17:08:49)
  most notably, high-log-deployments will
probably end up using logstash or something of that kind to capture
the logs (and sift through them) (rovarga,
17:09:17)


karaf-4.2.3 (rovarga, 17:16:27)

  karaf-4.2.3 bumps javax.annotation version to
1.3.0 (rovarga,
17:16:41)
  and multipatch fails of the dreaded package
resolution conflicts (rovarga,
17:19:08)
  we may need to postpone 4.2.4 upgrade until we
have sorted out oour JSR305 story, at least in APIs (rovarga,
17:19:28)


RFC8040 and filter (rovarga, 17:19:43)

  the question is whether the new RESTCONF
supports https://tools.ietf.org/html/rfc8040#section-4.8.4
filters (rovarga,
17:20:42)
  RFC8040 seems to specify filters only for
notification streams, not for individual GETs (rovarga,
17:26:34)
  since they are XPath based, we are pretty sure
they are not implemented for GETs even as an extension (rovarga,
17:26:54)


multipatch issues (rovarga, 17:27:22)

  the problem is with topic=fooBar, where the
latest tag is checked out and the patches are checked out on
it (rovarga,
17:28:54)
  for odlparent master patches cannot been
cherry-picked on top of odlparent-4.0.9 tag (rovarga,
17:29:12)
  https://jenkins.opendaylight.org/releng/view/integration/job/integration-multipatch-test-sodium/9/
(rovarga,
17:29:59)
  the best thing would be to introduce topic:foo
to perform a checkout (rovarga,
17:33:24)
  ACTION: LuisGomez
will try to come up with a solution for odlparent only (rovarga,
17:34:13)


tell-based protocol (rovarga, 17:36:10)

  ofp testing is still stuck on topology links
not being torn down (rovarga,
17:36:41)
  functional testing has only this issue
outstanding (rovarga,
17:37:46)
  https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-3node-clustering-perf-bulkomatic-only-neon/
(LuisGomez,
17:39:06)
  we have a failure with performance jobs, but
that may need tweaking (rovarga,
17:40:58)
  it seems that switching to tell-based protocol
by default is feasible in Sodium timeframe (rovarga,
17:45:04)


NETCONF scale testing (rovarga, 17:47:46)

  jmorvay asks about scaling of SB devices
(rovarga,
17:47:59)
  for a simple test tool, I could attach 10K
sessions with 965MiB retained heap (rovarga,
17:48:27)
  for stable/neon (rovarga,
17:48:37)
  with sshd-core-2.2.0 (staged at
https://git.opendaylight.org/gerrit/#/c/76300/) I could attach 10K
sessions with 744MiB retained heap (rovarga,
17:49:27)
  Bala reports similar results with 10GiB heap
and a real device (rovarga,
17:49:57)
  real device == ~70 YANG models (rovarga,
17:50:14)
  https://jira.opendaylight.org/browse/NETCONF-590 tracks
transport layer rework (rovarga,
17:53:02)
  which blocks
https://jira.opendaylight.org/browse/NETCONF-571 (rovarga,
17:53:38)
  which is about 50% of our current static
steady-state memory overhead (not counting SchemaContext
overheads) (rovarga,
17:56:28)


anyxml handling in get-config (rovarga, 17:57:47)

  Bala reports that it did not seem to work in
Carbon (rovarga,
17:57:59)
  it may be a missing bit in yangtools codecs,
where we do not handle one of the directions (rovarga,
17:58:30)
  will follow up after testing a 

Re: [controller-dev] OFP cluster test with "tell-based"

[Robert Varga]

On 19/02/2019 19:50, Luis Gomez wrote:
> 
> 
>> On Feb 19, 2019, at 6:16 AM, Robert Varga > <mailto:n...@hq.sk>> wrote:
>>
>> On 19/02/2019 02:11, Luis Gomez wrote:
>>>
>>>
>>>> On Feb 13, 2019, at 2:22 AM, Robert Varga >>> <mailto:n...@hq.sk>> wrote:
>>>>
>>>> On 12/02/2019 19:44, Luis Gomez wrote:
>>>>> Hi everybody,
>>>>>
>>>>> FYI I have just tried OFP cluster test with "tell-based" protocol:
>>>>>
>>>>> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-only-neon/180/robot-plugin/log.html.gz
>>>>>
>>>>> My observations:
>>>>>
>>>>> 1) node/port down events do not clear links in topology, this is
>>>>> why all topology check test fail.
>>>>
>>>> I think this is related to the transactions not commit in 5 seconds,
>>>> hence masters are not created.
>>>
>>> Any workaround for this?
>>
>> Not sure... if we have messed up accounding (below), we may end up
>> reporting things out of whack.

Alright, we can ditch the builder debugs, as everything there works as
it is supposed to.

The reason for non-removal of the links is captured in
https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/openflowplugin-csit-3node-clustering-only-sodium/1/odl_3/odl3_karaf.log.gz,
I think, and it is a VerifyException.

Filed to https://jira.opendaylight.org/browse/CONTROLLER-1885.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] OFP cluster test with "tell-based"

[Robert Varga]

On 19/02/2019 02:11, Luis Gomez wrote:
> 
> 
>> On Feb 13, 2019, at 2:22 AM, Robert Varga  wrote:
>>
>> On 12/02/2019 19:44, Luis Gomez wrote:
>>> Hi everybody,
>>>
>>> FYI I have just tried OFP cluster test with "tell-based" protocol:
>>>
>>> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-only-neon/180/robot-plugin/log.html.gz
>>>
>>> My observations:
>>>
>>> 1) node/port down events do not clear links in topology, this is why all 
>>> topology check test fail.
>>
>> I think this is related to the transactions not commit in 5 seconds,
>> hence masters are not created.
> 
> Any workaround for this?

Not sure... if we have messed up accounding (below), we may end up
reporting things out of whack.

> 
>>
>>> 2) some WARNs are flooding the log:
>>>
>>> 2019-02-12T00:26:30,055 | WARN  | 
>>> opendaylight-cluster-data-shard-dispatcher-33 | 
>>> FrontendClientMetadataBuilder| 223 - 
>>> org.opendaylight.controller.sal-distributed-datastore - 1.9.0.SNAPSHOT | 
>>> member-1-shard-inventory-operational: Unknown history for aborted 
>>> transaction member-1-datastore-operational-fe-0-txn-30-2, ignoring
>>>
>>> 2019-02-12T00:26:30,056 | WARN  | 
>>> opendaylight-cluster-data-shard-dispatcher-33 | 
>>> FrontendClientMetadataBuilder| 223 - 
>>> org.opendaylight.controller.sal-distributed-datastore - 1.9.0.SNAPSHOT | 
>>> member-1-shard-inventory-operational: Unknown history for aborted 
>>> transaction member-2-datastore-operational-fe-0-txn-19-1, ignoring
>>>
>>> 2019-02-12T00:26:30,056 | WARN  | 
>>> opendaylight-cluster-data-shard-dispatcher-33 | 
>>> FrontendClientMetadataBuilder| 223 - 
>>> org.opendaylight.controller.sal-distributed-datastore - 1.9.0.SNAPSHOT | 
>>> member-1-shard-inventory-operational: Unknown history for aborted 
>>> transaction member-3-datastore-operational-fe-0-txn-7-1, ignoring
>>
>> This is interesting, as it starts happening for the same transaction on
>> all shard members and these are standalone transactions, for which the
>> history should always be there.
>>
>> Can you re-run the test with debug on
>> org.opendaylight.controller.cluster.datastore.FrontendClientMetadataBuilder,
>> please?
> 
> Here it is: 
> https://jenkins.opendaylight.org/sandbox/job/openflowplugin-csit-3node-clustering-only-neon/1

Thanks, this actually provides a lead: everything works with normal
transaction chains, yet breaks down with single transactions.

Since we have module-based shards in play and multi-shard commits, the
cookie inside LocalHistoryIdentifier becomes significant in lookup --
and the single history is hard-wired to not have a cookie.

https://git.opendaylight.org/gerrit/80392 does that.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] OFP cluster test with "tell-based"

[Robert Varga]

On 12/02/2019 19:44, Luis Gomez wrote:
> Hi everybody,
> 
> FYI I have just tried OFP cluster test with "tell-based" protocol:
> 
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-only-neon/180/robot-plugin/log.html.gz
> 
> My observations:
> 
> 1) node/port down events do not clear links in topology, this is why all 
> topology check test fail.

I think this is related to the transactions not commit in 5 seconds,
hence masters are not created.

> 2) some WARNs are flooding the log:
> 
> 2019-02-12T00:26:30,055 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-33 | FrontendClientMetadataBuilder 
>| 223 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-1-shard-inventory-operational: Unknown history for 
> aborted transaction member-1-datastore-operational-fe-0-txn-30-2, ignoring
> 
> 2019-02-12T00:26:30,056 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-33 | FrontendClientMetadataBuilder 
>| 223 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-1-shard-inventory-operational: Unknown history for 
> aborted transaction member-2-datastore-operational-fe-0-txn-19-1, ignoring
> 
> 2019-02-12T00:26:30,056 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-33 | FrontendClientMetadataBuilder 
>| 223 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-1-shard-inventory-operational: Unknown history for 
> aborted transaction member-3-datastore-operational-fe-0-txn-7-1, ignoring

This is interesting, as it starts happening for the same transaction on
all shard members and these are standalone transactions, for which the
history should always be there.

Can you re-run the test with debug on
org.opendaylight.controller.cluster.datastore.FrontendClientMetadataBuilder,
please?

> 3) The cluster perf test does not pass: 
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-perf-bulkomatic-only-neon/180/robot-plugin/log.html.gz
> 
> I do not know if we still pursue to switch the cluster protocols, at least 
> after this test it does not seem an straight forward change.

I'd like to be able to ditch the old one, but it seems we need to shake
some bugs out :(

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] many logs about log index out-of-sync

[Robert Varga]

On 09/01/2019 01:41, Tom Pantelis wrote:
> > Those messages can happen if a follower gets behind the leader,
> especially if it gets isolated and AppendEntries
> > messages from the leader get backed up and then the follower gets
> a bunch of messages quickly when it reconnects to the
> > leader. Looks like it eventually caught up to the leader although,
> from the output, the distro you're testing with is
> > missing https://git.opendaylight.org/gerrit/#/c/78929/ which
> should speed up the sync process in that case and alleviate
> > most of those messages.
> 
> Thanks Tom, makes sense then as this is on one controller that came
> up last after
> bouncing them all.
> 
> 
> yeah it eventually synced but it went down an inefficient path that
> resulted in 1176 messages from the leader to sync it. Actually that
> patch I mentioned above was for a different case and wouldn't help in
> this case. I have an idea where we can optimize it to eliminate those
> extra messages and make it faster. 

https://git.opendaylight.org/gerrit/79334 should help with that (sorry,
it got lost for a year).

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] Unknown history for purged transaction member-2-datastore-operational-fe-0-chn-12-txn-0-0, ignoring

[Robert Varga]

On 26/11/2018 10:02, Faseela K wrote:
> https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/rpkgenius-csit-3node-gate-only-neon/6/odl_1/odl1_karaf.log.gz

Thanks. odl3 is the leader, showing:

> 2018-11-26T08:54:03,409 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-41 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Committed transaction 
> member-2-datastore-operational-fe-0-txn-146-0
> 2018-11-26T08:54:03,409 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-41 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Purged transaction 
> member-2-datastore-operational-fe-0-txn-145-0
> 2018-11-26T08:54:03,409 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-41 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Committed transaction 
> member-2-datastore-operational-fe-0-chn-3-txn-0-0
> 2018-11-26T08:54:03,417 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-23 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Closed history 
> LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-operational,
>  generation=0}, history=3, cookie=0}
> 2018-11-26T08:54:03,418 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-23 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Purged history 
> LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-operational,
>  generation=0}, history=3, cookie=0}
> 2018-11-26T08:54:03,418 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-23 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Purged transaction 
> member-2-datastore-operational-fe-0-txn-146-0
> 2018-11-26T08:54:03,418 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-23 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Unknown history for 
> purged transaction member-2-datastore-operational-fe-0-chn-3-txn-0-0, ignoring

and

> 2018-11-26T08:54:04,046 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-20 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Created local history 
> LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-operational,
>  generation=0}, history=4, cookie=0}
> 2018-11-26T08:54:04,047 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-20 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Purged transaction 
> member-2-datastore-operational-fe-0-txn-177-0
> 2018-11-26T08:54:04,050 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-44 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Created local history 
> LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-operational,
>  generation=0}, history=5, cookie=0}
> 2018-11-26T08:54:04,050 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-44 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Closed history 
> LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-operational,
>  generation=0}, history=4, cookie=0}
> 2018-11-26T08:54:04,051 | DEBUG | 
> opendaylight-cluster-data-shard-dispatcher-44 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Purged history 
> LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-operational,
>  generation=0}, history=4, cookie=0}
> 2018-11-26T08:54:04,051 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-44 | FrontendClientMetadataBuilder 
>| 232 - org.opendaylight.controller.sal-distributed-datastore - 
> 1.9.0.SNAPSHOT | member-3-shard-default-operational: Unknown history for 
> commited transaction member-2-datastore-operational-fe-0-chn-4-txn-0-0, 
> ignoring
> 2018-11-26T08:54:04,058 | WARN  | 
> opendaylight-cluster-data-shard-dispatcher-45 | FrontendClientMetadataBuilder 
>| 232 - 

Re: [controller-dev] [mdsal-dev] Unknown history for purged transaction member-2-datastore-operational-fe-0-chn-12-txn-0-0, ignoring

[Robert Varga]

On 23/11/2018 17:23, Faseela K wrote:
> Do you still want me to reproduce the issue with the DEBUGs mentioned? Or 
> already identified the
> Root cause?

Please try to reproduced it, I have only a theory at this point, those
debugs will hopefully give us a smoking gun.

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] Unknown history for purged transaction member-2-datastore-operational-fe-0-chn-12-txn-0-0, ignoring

[Robert Varga]

On 23/11/2018 16:11, Robert Varga wrote:
> On 23/11/2018 15:29, Faseela K wrote:
>> https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/rpkgenius-csit-3node-gate-only-neon/15/odl_2/odl2_karaf.log.gz
>>
>> [sandbox log, will get deleted in another 24hours, I guess]
> This looks weird, as if we lost some journal records for create, or got
> some early closures.

Tom,

I just dug around the code, and it seems CloseTransactionChain is being
broadcast to all nodes, which dates back to
https://git.opendaylight.org/gerrit/#/c/10833/.

On the backend side, we treat the same on all shards -- i.e. by call
into ShardDataTree, which closes stuff it finds and then calls into
Shard.replicatePayload(). I am pretty sure that is wrong, as I am not
seeing a guard for that happening only on the leader and thus we are
probably screwing up journal consistency.

So if journal index accounding and payload ID tracking is screwed up, we
can end up firing wrong events when consensus is reached...

Can you take a look?

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] Unknown history for purged transaction member-2-datastore-operational-fe-0-chn-12-txn-0-0, ignoring

[Robert Varga]

On 23/11/2018 15:29, Faseela K wrote:
> https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/rpkgenius-csit-3node-gate-only-neon/15/odl_2/odl2_karaf.log.gz
> 
> [sandbox log, will get deleted in another 24hours, I guess]

This looks weird, as if we lost some journal records for create, or got
some early closures. There are some dead letters involving
CloseTransactionReply just around that time, which may be related.

Can you replicate it with

> log:set DEBUG 
> org.opendaylight.controller.cluster.datastore.FrontendClientMetadataBuilder

please?

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] Unknown history for purged transaction member-2-datastore-operational-fe-0-chn-12-txn-0-0, ignoring

[Robert Varga]

On 23/11/2018 15:20, Faseela K wrote:
> I did not understand much of it :)
> But I am seeing this in genius 3 node CSIT karaf logs, Does this mean we are 
> doing something wrong in  our datastore operations, which we have to fix?

Got a pointer?

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] Unknown history for purged transaction member-2-datastore-operational-fe-0-chn-12-txn-0-0, ignoring

[Robert Varga]

On 23/11/2018 09:05, Faseela K wrote:
> Hello mdsal-dev,
> 
> Can someone please tell me when exactly this warning will come?
> 
>  
> 
>  
> 
> *2018-11-23T04:37:13,831 | WARN  |
> opendaylight-cluster-data-shard-dispatcher-28 |
> FrontendClientMetadataBuilder    | 232 -
> org.opendaylight.controller.sal-distributed-datastore - 1.9.0.SNAPSHOT |
> member-2-shard-default-operational: Unknown history for purged
> transaction member-2-datastore-operational-fe-0-chn-12-txn-0-0, ignoring*

sal-distributed-datastore: -mdsal-dev, +controller-dev.

This occurs when a history (a.k.a transaction chain) purge record is
received through the journal, but the shard has no knowledge of it.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

[controller-dev] [yangtools-dev] MoM of today's Kernel Projects call

[Robert Varga]

Hello,

we had a rather longer Kernel Projects' call today, discussing some
impeding changes to better support downstreams. Minutes are available at:

> [20/11/2018 19:16:22]  Meeting ended Tue Nov 20 18:16:22 2018 
> UTC.  Information about MeetBot at http://ci.openstack.org/meetbot.html . (v 
> 0.1.4)
> [20/11/2018 19:16:22]  Minutes:
> http://meetings.opendaylight.org/opendaylight-meeting/2018/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2018-11-20-17.03.html
> [20/11/2018 19:16:22]  Minutes (text): 
> http://meetings.opendaylight.org/opendaylight-meeting/2018/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2018-11-20-17.03.txt
> [20/11/2018 19:16:22]  Log:
> http://meetings.opendaylight.org/opendaylight-meeting/2018/kernel_projects_call/opendaylight-meeting-kernel_projects_call.2018-11-20-17.03.log.html

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] new PingPongDataBroker interface for simpler annotation instead of XML based dependency injection?

[Robert Varga]

On 13/11/2018 17:45, Michael Vorburger wrote:
> Hello,
> 
> In https://git.opendaylight.org/gerrit/#/c/77640/
> for https://jira.opendaylight.org/browse/OPNFLWPLUG-1046 the question
> has come up how to correctly wire the ping pong DataBroker when
> following the https://wiki.opendaylight.org/view/Using_Blueprint
> documentation.
> 
> One of the possible options, which would seem simplest and clearest to
> me, would be to introduce a new interface PingPongDataBroker extends
> DataBroker, and bind to and @Inject that. Would the controller/mdsal
> projects be open for a change proposing this, or is that a stupid idea?

It is an implementation, I do not believe each implementation should
have its own interface.

The proper way is via an implementation property, just like we're doing
it now.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease oxygen failed to build sal-distributed-datastore from controller

[Robert Varga]

On 08/11/2018 02:09, Jenkins wrote:
> Attention controller-devs,
> 
> Autorelease oxygen failed to build sal-distributed-datastore from controller 
> in build
> 477. Attached is a snippet of the error message related to the
> failure that we were able to automatically parse as well as console logs. 
> 
> 
> Console Logs:
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-oxygen/477

This is the third time Oxygen autorelease has failed on CDS unit tests,
which are (inherently) time-sensitive. Each time it was a different test
and while we can increase timeouts
(https://git.opendaylight.org/gerrit/77597), these tests have been
stable for a long time -- which, coupled with other performance-related
questions cropping up, is leading me to ask:

What is happening to our build infrastructure lately?
Are we sizing the VMs differently?
Can we get a statement from our cloud provider, please?

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Entity ownership for non-voting member

[Robert Varga]

On 07/11/2018 20:36, Ajay Lele wrote:
> 
> 
> On Wed, Nov 7, 2018 at 3:58 AM Robert Varga  <mailto:n...@hq.sk>> wrote:
> 
> On 07/11/2018 02:07, Ajay Lele wrote:
> > Hi Controller-devs,
> >
> > [0] changed EOS behavior such that a non-voting member cannot become
> > entity owner. But there are some situations where we want to allow
> this
> > e.g. when BGP speaker config is local to node and not replicated in
> > cluster. I think the behavior should be that non-voting member
> will not
> > become entity owner *provided* one or more voting candidates are
> > available. Thoughts?
> 
> This is a sticky topic. Non-voting members are meant to be used for
> geo-redundancy only, in which case non-voting side should be really
> passive.
> 
> Can you describe the deployment scenario in more detail?
> 
> 
> One scenario is BGP route injection using application RIB. Injected
> route will be withdrawn when BGP connection over which it is advertised
> goes down. To prevent downtime window when primary to DR cutover
> happens, BGP connection is established from non-voting member from DR
> site as well. This is accomplished by moving BGP speaker data to a
> separate shard which is not replicated.

I see, so this is used for planned maintenance only, right?

If that is the case, I think it would make sense to make the selection
policy switchable at runtime, such that under normal operation only
voting members are considered, but in preparation for a switchover, the
policy can be adjusted. Can you file an improvement issue against
controller, please?

Tom: do you have an opinion on how best implement this? It feels like we
want to have a per-entity behavior table, so that BGP would switch to
non-voting, but other services would not...

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease oxygen failed to build sal-distributed-datastore from controller

[Robert Varga]

On 05/11/2018 18:14, Sam Hague wrote:
> Is this any way related to Lori's connection failures in the lisp IT.
> Likely a stretch but they had connection issues also.

Could be, if the VMs are experiencing hiccups. Then again it might have
beenm a one-off... Oxygen has been just sitting there for a looong time.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease oxygen failed to build sal-distributed-datastore from controller

[Robert Varga]

On 05/11/2018 13:32, Tom Pantelis wrote:
> Results :
> 
> Failed tests: 
>   ModuleShardBackendResolverTest.testRefreshBackendInfo:138
> assertion failed: timeout (3 seconds) during expectMsgClass waiting
> for class
> org.opendaylight.controller.cluster.access.commands.ConnectClientRequest
> 
> 
> I have never seen this test fail. I don't have much familiarity with
> this code (Robert wrote it) and I wouldn't be able to look into it till
> later next week at the earliest. Hopefully it doesn't somehow become
> frequent now - AFAIK there's been no changes in quite some time so most
> likely it's a rare timing issue in the UT.

Yup, I do not see a reason why this should start failing now. Let's see
what the next run does.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Karaf error yangtool

[Robert Varga]

On 01/11/2018 16:29, benamrane fouad wrote:
> Hi all,

Hello,

> I would like to reproduce the hello world application in odl using
> latest versions, but I got an error after creating the yang rpc model.
> 
> org.apache.felix.resolver.reason.ReasonException: Unable to resolve
> org.opendaylight.mdsal.yang-binding/0.14.0.SNAPSHOT: missing requirement
> [org.opendaylight.mdsal.yang-binding/0.14.0.SNAPSHOT]
> osgi.wiring.package;
> filter:="(&(osgi.wiring.package=org.opendaylight.yangtools.util)(version>=2.0.0)(!(version>=3.0.0)))"
>     at
> org.apache.felix.resolver.Candidates$MissingRequirementError.toException(Candidates.java:1343)
>     at
> org.apache.felix.resolver.ResolverImpl.doResolve(ResolverImpl.java:420)
>     at org.apache.felix.resolver.ResolverImpl.resolve(ResolverImpl.java:378)
>     at org.apache.felix.resolver.ResolverImpl.resolve(ResolverImpl.java:332)
>     at
> org.apache.karaf.features.internal.region.SubsystemResolver.resolve(SubsystemResolver.java:257)
>     at
> org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:388)
>     at
> org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1025)
>     at
> org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$13(FeaturesServiceImpl.java:964)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>     at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>     at java.lang.Thread.run(Thread.java:748)
> 
> My environment and build command
> java 1.8.0_181
> maven 3.5.4
> to create hello project I setup the maven archetype:
> mvn archetype:generate -DarchetypeGroupId=org.opendaylight.archetypes
> -DarchetypeArtifactId=opendaylight-startup-archetype
> -DarchetypeCatalog=remote -DarchetypeVersion=1.1.0-SNAPSHOT

Archetype has not been updated in the Neon MRI window:
https://jira.opendaylight.org/browse/ARCH-3.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease neon failed to build sal-binding-it from controller

[Robert Varga]

On 24/10/2018 09:48, Ariel Adam wrote:
> Tom, any idea what this failure is?
> 
> Karaf started in 1s. Bundle stats: 13 active, 13 total
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 
> 181.838 s <<< FAILURE! - in 
> org.opendaylight.controller.test.sal.binding.it.RoutedServiceIT
> [ERROR] 
> testServiceRegistration(org.opendaylight.controller.test.sal.binding.it.RoutedServiceIT)
>   Time elapsed: 181.831 s  <<< ERROR!
> java.rmi.NotBoundException: 2837944f-d160-4b17-a124-be922dbaf3bd

  

The usual error reported when karaf fails to start under pax-exam,
exactly like SFT.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] aaa build failure in proposed topic:neon-mri ("Weather Item" TSC-132)

[Robert Varga]

On 23/09/2018 17:33, Michael Vorburger wrote:
> On Sun, Sep 23, 2018 at 3:21 PM Tom Pantelis  > wrote:
> 
> On Sun, Sep 23, 2018 at 7:26 AM Michael Vorburger
> mailto:vorbur...@redhat.com>> wrote:
> 
> Dear maintainers of project aaa,
> 
> While verifying the proposed cross-projects changes on managed
> topic:neon-mri together, your project failed to build; please
> see
> 
> https://jenkins.opendaylight.org/releng/view/integration/job/integration-multipatch-test-neon/26/console.
> 
> IMHO this is blocking topic:neon-mri / TSC-132 and one of us
> should see how we can sort this out:
> 
> Running org.opendaylight.odlparent.featuretest.SingleFeatureTest
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 315.015 sec <<< FAILURE! - in
> org.opendaylight.odlparent.featuretest.SingleFeatureTest
> 
> installFeatureCatchAndLog(org.opendaylight.odlparent.featuretest.SingleFeatureTest)[repoUrl:
> 
> file:/w/workspace/integration-multipatch-test-neon/patch_tester/aaa/features/odl-aaa-password-service/target/feature/feature.xml,
> Feature: odl-aaa-password-service 0.9.0.SNAPSHOT]  Time elapsed:
> 314.722 sec  <<< ERROR!
> org.awaitility.core.ConditionTimeoutException: Condition with
> alias 'checkBundleDiagInfos' didn't complete within 300 seconds
> because lambda expression in
> org.opendaylight.odlparent.bundlestest.lib.TestBundleDiag:
> expected system either ready with all bundles Active, or
> Stopping or Failure (but not still booting in GracePeriod,
> Waiting, Starting, Unknown;but just Resolved and some
> exceptional Installed OK) but was  Resolved=5, Unknown=0, GracePeriod=1, Waiting=0, Starting=0,
> Active=101, Stopping=0, Failure=0}
> 1. NOK
> org.opendaylight.aaa.password-service-impl:0.9.0.SNAPSHOT: OSGi
> state = Active, Karaf bundleState = GracePeriod, due to: Blueprint
> 9/23/18 10:55 AM
> Missing dependencies: 
> 
> (&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://opendaylight.org/xmlns/blueprint/v1.0.0))
>  
> >.
> 
> 
> This is b/c https://git.opendaylight.org/gerrit/#/c/74964/ moved the
> aaa-password-service BP xml file under OSGI-INF/blueprint. However
> the feature does not pull in the ODL blueprint bundles, either
> directly or indirectly via odl-mdsal-broker-local. 
> So it either needs to pull in odl-mdsal-broker-local or we create a
> feature for the ODL blueprint bundle. For the short-term, that patch
> doesn't need to
> move  the BP xml file for the MRI version bumps so we could put it
> back under org/opendaylight/blueprint for now and address it in
> another patch.
> 
> 
> I see. For the very short-term and to unblock topic:neon-mri (I'm
> curious to see how far we can get the multipatch job to progress by all
> working together this week!) I agree and too would go for the latter
> and leave them in org/opendaylight/blueprint instead of moving them
> to OSGI-INF/blueprint in c/74964 (NB not
> just password-service-blueprint.xml but all BP XML).
> 
> Robert, as the author of c/74964 would you like to amend it to do so? If
> you don't have time but confirm that you agree this is what should be
> done, then I'm happy to do this as well, in order to unblock.

Done.

> But given that we want to converge on OSGI-INF/blueprint (and explicitly
> ask projects in the migration documentation on
> https://wiki.opendaylight.org/view/Neon_platform_upgrade#Blueprint_declarations
> ...) I think it would be useful to do this uniformely soon-ish, so let's
> make a plan for that as well, in parallel to fixing the short term? I
> should be easy enough to raise a change in controller to have a new
> odl-blueprint feature if that's what we want (and I'm happy to), but...
> do we really want to? Would you then want to add that explicitly
> to odl-aaa-password-service, and elsewhere where we hit this problem? I
> don't really understand how it's possible for a bundle to use ODL
> extensions in its BP XML yet not already depend on the feature that
> currently brings it along (odl-mdsal-broker-local, from wha you write) -
> what am I missing? You wouldn't want to just make
> odl-aaa-password-service dependant on odl-mdsal-broker-local ?

Tom's email has the details. Yes, we want a new feature, no
odl-mdsal-broker-local is an overkill, yes, we want to move the
blueprints (because that change is good), no we do not need to do in in
the MRI window :)

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease neon failed to build sal-inmemory-datastore from controller

[Robert Varga]

On 20/09/2018 10:34, Ariel Adam wrote:
> Tom, looks like a compilation failure:
> 
> *00:28:35* [INFO] 
> -
> *00:28:35* [INFO] 
> -
> *00:28:35* [ERROR] COMPILATION ERROR : *00:28:35* [INFO] 
> -
> *00:28:35* [ERROR]
> /w/workspace/autorelease-release-neon/controller/opendaylight/md-sal/sal-inmemory-datastore/src/main/java/org/opendaylight/controller/md/sal/dom/store/impl/InMemoryDOMDataStore.java:[38,20]
> no suitable constructor found for
> InMemoryDOMDataStore(java.lang.String,org.opendaylight.mdsal.common.api.LogicalDatastoreType,java.util.concurrent.ExecutorService,int,boolean)
> 
> 
> What do you think?

autorelease.git is not updated to pull correct revision of mdsal.git --
trailing by two patches.

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [opendaylight-dev] [release] [OpenDaylight TSC] [releng] Fluorine branch cutting

[Robert Varga]

On 13/08/18 14:06, Anil Belur wrote:
> 
> https://git.opendaylight.org/gerrit/#/c/75151/
> 
> 
> cool, now we just need to get autorelease to actually pass again... ;-)
> I'm still, right now, seeing various failures
> on 
> https://jenkins.opendaylight.org/releng/job/genius-validate-autorelease-neon/8/console
> from the following projects:
> 

00:04:11.596 Entering 'mdsal'
00:04:11.607 mdsal 3e5d53eefead8a109f72b9d658432050d01e8cf1 Validate

mdsal is not getting updated to the proper commit, which should be
3346408b80c3dd860d42f18469c7e35aa01cfee3

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Does commiting a transaction with no operations have any noteworthy (real life) overhead compared to cancelling it?

[Robert Varga]

On 27/07/18 13:34, Michael Vorburger wrote:
> What I am wondering if such a short cut optimization has any real value,
> and if it does if it shouldn't go into core MD SAL instead
> of ManagedNewTransactionRunner.

I do not believe it is worth the complexity it incurs. We used to
shortcut transactions which were known to have no effect and we were
glad to ditch it.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] Does commiting a transaction with no operations have any noteworthy (real life) overhead compared to cancelling it?

[Robert Varga]

On 27/07/18 11:41, Anil Vishnoi wrote:
> 
> My initial reaction is that such an optimization in
> ManagedNewTransactionRunner is probably pointless as whatever
> happens behind the scenes on a commit is surely already smart enough
> by itself for a submit on an empty transaction to basically be a low
> overhead NOOP anyway?
> 
> ​Or if transaction API can expose some api like isEmpty() (just
> example), that can come bit handy here?

I don't think the benefit of such a method justifies additional state
tracking required to support it.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] Does commiting a transaction with no operations have any noteworthy (real life) overhead compared to cancelling it?

[Robert Varga]

On 27/07/18 13:33, Michael Vorburger wrote:
> On Fri, Jul 27, 2018 at 12:21 PM Robert Varga  <mailto:n...@hq.sk>> wrote:
> 
> On 27/07/18 11:29, Michael Vorburger wrote:
> > Hello, Tom, Robert, Stephen,
> >
> > Does commiting a transaction with no put/.../merge/delete
> operations on
> > that Tx have any noteworthy (real life) overhead compared to
> cancelling it?
> >
> 
> At the very least it requires queueing, as it needs to be ordered among
> transactions. Cancel is distinctly different.
> 
> 
> but if there are no operations, it could just shortcut to cancel, no? 
> 
> Would this perhaps be a worthwhile enhancement (for Neon obviously) in
> core MD SAL (instead of ManagedNewTransactionRunner) ?

They are different operations, their outcome is different, as is their
interface contract. No, we should not be taking any shortcuts here.

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] Does commiting a transaction with no operations have any noteworthy (real life) overhead compared to cancelling it?

[Robert Varga]

On 27/07/18 11:29, Michael Vorburger wrote:
> Hello, Tom, Robert, Stephen,
> 
> Does commiting a transaction with no put/.../merge/delete operations on
> that Tx have any noteworthy (real life) overhead compared to cancelling it?
> 

At the very least it requires queueing, as it needs to be ordered among
transactions. Cancel is distinctly different.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Project docs in Fluorine

[Robert Varga]

On 19/07/18 01:20, Luis Gomez wrote:
> To all project committers/PTLs,
> 
> If your project docs:

Actually, we have a 3) in mdsal and controller (at least) and
openflowplugin (perhaps):

3) We have some .adoc files which need conversion and proper filing :(

Can somebody lend a hand to deal with

> nite@nitebug : ~/odl/autorelease on master $ find . -name \*.adoc | fgrep src/
> ./controller/opendaylight/md-sal/sal-distributed-datastore/src/site/asciidoc/distributed-data-store.adoc
> ./openflowplugin/applications/bulk-o-matic/src/site/asciidoc/bulk-o-matic.adoc
> ./mdsal/binding2/mdsal-binding2-spec/src/site/asciidoc/binding-2.adoc
> ./mdsal/docs/src/main/asciidoc/contributor/introduction.adoc
> ./mdsal/docs/src/main/asciidoc/developer/analysis/binding-v2.adoc
> ./mdsal/docs/src/main/asciidoc/developer/design/yang-1-1.adoc
> ./mdsal/docs/src/main/asciidoc/developer/introduction.adoc
> ./mdsal/singleton-service/mdsal-singleton-common-api/src/site/asciidoc/cluster-wide-services.adoc
> ./mdsal/src/site/asciidoc/architecture.adoc
> ./mdsal/src/site/asciidoc/conceptual-data-tree.adoc
> ./mdsal/src/site/asciidoc/overview.adoc
> ./mdsal/src/site/asciidoc/release-notes.adoc
> ./mdsal/src/site/asciidoc/resources.adoc

?

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease oxygen failed to build sal-cluster-admin-impl from controller

[Robert Varga]

On 20/07/18 16:54, Daniel Farrell wrote:
> 
> I feel like blaming infrastructure being "slow" is too easy an
> excuse for issues. If the software was run in a customer production
> environment I suspect telling the customer that their hardware is
> too slow and is not the same hardware as the developer's laptop it
> would not be a solution the customer would be happy with.
> 
> 
> +1000
> 
> Our code and tests need to be robust enough to handle diverse
> infrastructure. Bugs like this might be highlighted by infra
> variability, but they are still bugs in code/tests.
> 
> Not picking on TomP or Controller here, this is a general ODL culture
> problem of blaming the infra first and until Thanh/Jamo/et al prove
> otherwise.

I am not taking sides here, but when we are talking distributed systems,
timing *is* critical -- a system slow to respond is indiscernible from a
failed system and it *will* get isolated.

The trade-off here is tolerance of bumps vs. reaction to failures -- and
this *is* something you tweak for a deployment and yes, this is where
the vendor typically does get to dictate required performance
characteristics and/or provides very stringent tuning parameters. And
yes, as soon as you contact support, it will ask for those parameters
(and bunch more) and will (very politely) ask you to fix your
configuration if there is any deviation from recommended values.

If there are stressors we experience during autorelease build we need to
understand what they are so we can account for them: as Tom noted, the
deadline is 'a tad too low' -- which works out to a trade-off question
(I am exaggerating to make a point):

Would you be okay with the test having a deadline on the order of 10
minutes before it declares the test failed?

Which boils down to: how much of a hiccup are we supposed to tolerate?

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease fluorine failed to build sal-core-compat from controller

[Robert Varga]

On 19/07/18 03:04, Jenkins wrote:
> Attention controller-devs,
> 
> Autorelease fluorine failed to build sal-core-compat from controller in build
> 150. Attached is a snippet of the error message related to the
> failure that we were able to automatically parse as well as console logs. 
> 
> 
> Console Logs:
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-fluorine/150

autorelease got caught up in an API-incompatible change between mdsal
and controller due to unfortunate timing. This is already cleared up.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

[controller-dev] Understading CDS

[Robert Varga]

Hello Josh, everyone,

when trying to understand what CDS does and how it does it, there are
concepts and technologies that must be understood -- all relating to
distributed systems and state management theory.

Specific topics:
- Actor systems, with Akka being an implementation
- Akka Clustering
- Akka Persistence
- The RAFT algorithm (and distributed consensus in general, like 3PC)
- Multiversion Concurrency Control (as a solution to the problem of
concurrency control)

All of these are things that cannot be explained in minutes and all have
bearing on architecture of CDS as well as trade-offs taken in its design
and implementation.

If we try to have a conversation about the CDS without sharing this
common knowledge, that conversation will be utterly inefficient with
frequent and long digressions into those topics -- which is something I
(and I suspect Tom) can ill afford.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] SystemReadyService fails, system does not work

[Robert Varga]

On 03/07/18 13:43, Josh Hershberg wrote:
> A number of us have seen the exception below over the past few days.
> I've opened this bug to track it, anyone know anything?

Missing port of CONTROLLER-1590, flushed out by controller's switch to
mdsal components, already fixed.

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Serviceutils distro failure

[Robert Varga]

On 26/06/18 13:40, Faseela K wrote:
> Thanks a lot Michael.
> 
> This looks like an MDSAL bug, that even if the yang modules are
> separated by namespaces, there is some bug somewhere that it is not
> honored properly.
> 
>  
> 
> I was able to change the module names in serviceutils(so that it looks
> different from genius), and then when I install the feature, I am not
> hitting the error.
> 
>  
> 
> @Robert : Can we get someone to take a look at
> https://jira.opendaylight.org/browse/MDSAL-354
> 
>  And fix, else we have no other option than to workaround by renaming
> the yang modules in serviceutils.

Module names form a global flat namespace, as per
https://tools.ietf.org/html/rfc7950#section-6.2.1 and therefore module
names within a system must be unique and furthermore module
name/namespace mapping must be a bijection.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [infrautils-dev] Sharding evolution

[Robert Varga]

Hello Muthu,

There are only two ways in which a transaction can fail aside from 'datastore 
is busted':
- invalid data being written
- conflicting activity outside of the causality chain

The first one is an obvious coding error and I don't quite see how you'd design 
a recovery strategy whose complexity does not exceed complexity of the normal 
path.

The second one stems from application design: why is the application not 
designed in a conflict - free manner? And when a conflict occurs, how do you 
know it's nature and how to reconcile it?

You certainly can redo a failed transaction: it is only a matter holding on to 
the inputs, i.e. DTCL view is immutable.

Nevertheless if it's performance you are after conflicts should happen once in 
a blue moon...

Sent from my BlackBerry - the most secure mobile device - via the Orange Network

  Original Message  
From: muthukumara...@ericsson.com
Sent: June 9, 2018 10:10 AM
To: n...@hq.sk; faseel...@ericsson.com; vishnoia...@gmail.com
Cc: infrautils-...@lists.opendaylight.org; 
controller-dev@lists.opendaylight.org; genius-...@lists.opendaylight.org
Subject: RE: [controller-dev] [infrautils-dev] Sharding evolution

Transaction Chains is also useful in context of ensuring that last txn is 
completed before next is executed so that subsequent txn can see the changes 
made by previous one (of course within single subtree) more efficiently. And 
also enables single-writer discipline

@Robert,

In context of Txn Chain, if 10 txns are submitted and failure occurs at 5th 
txn, the chain would provide a failure callback.
Most rampant pattern part for apps would be submitting txns to the chain from 
DTCLs or CDTCLs. Assuming, 10 change notifications resulted in 10 chain txn 
submits and chain fails the 5th txn due to valid reasons, now, apps have lost 
the context of 5 txns which failed.

In such scenarios, what would be a better approach for apps to perform any 
compensatory actions for failed transactions in context of using chain?

Regards
Muthu

-Original Message-
From: controller-dev-boun...@lists.opendaylight.org 
[mailto:controller-dev-boun...@lists.opendaylight.org] On Behalf Of Robert Varga
Sent: Saturday, June 09, 2018 6:25 AM
To: Faseela K ; Anil Vishnoi 
Cc: infrautils-...@lists.opendaylight.org; controller-dev 
; genius-...@lists.opendaylight.org
Subject: Re: [controller-dev] [infrautils-dev] Sharding evolution

On 09/06/18 02:06, Faseela K wrote:
> [Changed the subject]
> 
>  
> 
> Anil, now you can ask ;)
> 
>  
> 
> https://wiki.opendaylight.org/view/Genius:Sharding_evolution
> 

MD-SAL long-term design:
https://wiki.opendaylight.org/view/MD-SAL:Boron:Conceptual_Data_Tree

Make sure to align your thinking with that... Splitting lists at MD-SAL runs 
into the problem of consistent hashing and scatter/gather operations:
- given a key I must know which shard it belongs to (and that determination has 
to be *quick*)
- anything crossing shards is subject to coordination, which is a *lot* less 
efficient than single-shard commits

If it's performance you are after:
- I cannot stress the importance of TransactionChains enough: if you cannot do 
them, you need to go back to the drawing board, as causality and shared fate 
*must* be properly expressed
- Avoid cross-shard transactions at (pretty much) all cost. I know of
*no* reason to commit to inventory and topology at the same time - if you have 
a use case which cannot be supported without it, please do describe it (and 
explain why it cannot be done)
- No synchronous operations, anywhere
- Producers (tx.submit() are just one side of the equation, consumers
(DTCL) are equally important

Regards,
Robert

___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [infrautils-dev] Sharding evolution

[Robert Varga]

On 09/06/18 02:06, Faseela K wrote:
> [Changed the subject]
> 
>  
> 
> Anil, now you can ask ;)
> 
>  
> 
> https://wiki.opendaylight.org/view/Genius:Sharding_evolution
> 

MD-SAL long-term design:
https://wiki.opendaylight.org/view/MD-SAL:Boron:Conceptual_Data_Tree

Make sure to align your thinking with that... Splitting lists at MD-SAL
runs into the problem of consistent hashing and scatter/gather operations:
- given a key I must know which shard it belongs to (and that
determination has to be *quick*)
- anything crossing shards is subject to coordination, which is a *lot*
less efficient than single-shard commits

If it's performance you are after:
- I cannot stress the importance of TransactionChains enough: if you
cannot do them, you need to go back to the drawing board, as causality
and shared fate *must* be properly expressed
- Avoid cross-shard transactions at (pretty much) all cost. I know of
*no* reason to commit to inventory and topology at the same time - if
you have a use case which cannot be supported without it, please do
describe it (and explain why it cannot be done)
- No synchronous operations, anywhere
- Producers (tx.submit() are just one side of the equation, consumers
(DTCL) are equally important

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] OK to resurrect c/64522 to first move infrautils.DiagStatus integration for datastore from genius to controller, and then improve it for GENIUS-138 ?

[Robert Varga]

On 08/06/18 23:02, Anil Vishnoi wrote:
> 
> All yang notifications are local - not sure what you mean by routed. 
> 
> ​ I mean, like we route RPC's, i was wondering if you are building
> something that will route the yang notification as well to other node.​
>  


Not until someone implements
https://jira.opendaylight.org/browse/CONTROLLER-917 (and corresponding
classifier so we do not end up flooding all nodes with things that are
supposed to be node-local).

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] OK to resurrect c/64522 to first move infrautils.DiagStatus integration for datastore from genius to controller, and then improve it for GENIUS-138 ?

[Robert Varga]

On 07/06/18 19:14, Michael Vorburger wrote:
> Robert,
> 
> just to avoid any misunderstandings and unnecessary extra work to throw
> away, may we double check and confirm that we correctly understand your
> comment in  https://jira.opendaylight.org/browse/GENIUS-138 to mean that
> we are past the "dependency of a mature project on an incubation
> project" objection and you are now OK with that
> we resurrect https://git.opendaylight.org/gerrit/#/c/64522/, to first
> move infrautils.DiagStatus integration for datastore from genius to
> controller? We would then improve it, in controller instead of genius,
> for the improvement proposed in issue GENIUS-138.

Hello Michael,

my point is duplicity of information, i.e. I very much want to have a
single point of source of the data, with others being adaptations of
that source.

If you are saying that DiagStatus can supplant the information provided
in JMX, fine. If not, than DiagStatus should rely on that JMX endpoint
-- I think the complexity you implied in GENIUS-138 does not exist, but
there was reaction around that point.

So rather than rolling in yet another framework, let's have the
discussion: in what ways it is superior to JMX?

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] yang notifications to CDS to emit interesting state/status changes

[Robert Varga]

On 07/06/18 21:20, Faseela K wrote:
> Yes, we are currently using EOS in such scenarios. But is there a way to
> specify that I want my entity owner to be on “default-config shard leader”?
> 
> That’s why I was asking whether the new notifications you are going to
> add will help in implementing such functionalities.
> 

No, service placement policies are currently not implemented. This
really is something which should be sitting outside of EOS (and CDS),
should monitor both and push down selection decisions.

The reason for that is that the policy you are describing is simple, but
the problem domain is quite huge -- because right next request in this
area is to provide service load balancing :-)

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [yangtools-dev] patch-test triggers to run csit on patches

[Robert Varga]

On 05/06/18 16:28, Sam Hague wrote:
> All the projects in the to: list have the ability to run genius and
> netvirt csit on any patch using a gerrit comment. The job is non-voting
> and only reports back status. If you do run the csit and notice a
> non-success, please ping the genius and netvirt lists to look at the job
> so the team can fix any issues found.
> 
> The format of the comment is:
> test--. 
> 
> For example, on a controller patch, you would add this comment to run
> genius csit: test-controller-genius. This is shown in controller
> patch: https://git.opendaylight.org/gerrit/#/c/72650/.
> 
> For aaa to run netvirt csit, test-aaa-netvirt.

This is awesome :) although yangtools would also need a replacement of
jars, so I am not sure how useful it is at this time :)

Thanks,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Build passing for proposed topic:CONTROLLER-1802 ("Weather Item" CONTROLLER-1802)

[Robert Varga]

Merged, I'll close the weather item once it's clear it does not break AR.

Regards,
Robert

On 04/05/18 14:38, Robert Varga wrote:
> Yeah, it is on my list, probably sometime on Sunday.. M
> 
> Sent from my BlackBerry - the most secure mobile device - via the Orange
> Network
> *From:* vorbur...@redhat.com
> *Sent:* May 4, 2018 14:26
> *To:* n...@hq.sk; tompante...@gmail.com
> *Cc:* controller-dev@lists.opendaylight.org;
> mdsal-...@lists.opendaylight.org; t...@lists.opendaylight.org
> *Subject:* Re: Build passing for proposed topic:CONTROLLER-1802
> ("Weather Item" CONTROLLER-1802)
> 
> 
> Robert & Tom,
> 
> On Thu, May 3, 2018 at 7:00 PM, Michael Vorburger <vorbur...@redhat.com
> <mailto:vorbur...@redhat.com>>wrote:
> 
> Dear maintainers of projects controller, mdsal,
> 
> While verifying the proposed cross-projects changes on managed
> topic:[( ${gerritTopicName} )], all of your projects and their
> downstreams passed, when built together:
> 
>    
> 
> https://jenkins.opendaylight.org/releng/job/integration-multipatch-test-fluorine/25/
> 
> <https://jenkins.opendaylight.org/releng/job/integration-multipatch-test-fluorine/25/>
> 
> Please decide whether you need CSIT/s to run for this.  If not, get
> together and organize a "simultaneous merge" of all of these changes:
> 
>     https://git.opendaylight.org/gerrit/#/q/topic:CONTROLLER-1802
> <https://git.opendaylight.org/gerrit/#/q/topic:CONTROLLER-1802>
> 
> 
> based on this (build passing, no FB impact anywhere), would either of
> you who are both committers on both controller and mdsal like to go
> ahead and (at the same time)
> merge https://git.opendaylight.org/gerrit/#/c/66360/ and 
> https://git.opendaylight.org/gerrit/#/c/66361/
> ? Perhaps first thing on Monday, just in case - not to spoil anyone's 
> week-end.
> 
> Tx,
> M.
> --
> Michael Vorburger, Red Hat
> vorbur...@redhat.com <mailto:vorbur...@redhat.com> | IRC: vorburger
> @freenode | ~ = http://vorburger.ch <http://vorburger.ch/>
> 
>  
> 
> If you would like to reach a human for any questions about this,
> please do not reply to this email, but reach out to the Reporter of
> JIRA issue https://jira.opendaylight.org/browse/TSC-100
> <https://jira.opendaylight.org/browse/TSC-100>.
> 
> Yours sincerely,
> The ODL Bot <https://github.com/vorburger/opendaylight-bot
> <https://github.com/vorburger/opendaylight-bot>>
> 
> 



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Build passing for proposed topic:CONTROLLER-1802 ("Weather Item" CONTROLLER-1802)

[Robert Varga]

  Yeah, it is on my list, probably sometime on Sunday.. MSent from my BlackBerry - the most secure mobile device - via the Orange Network   From: vorbur...@redhat.comSent: May 4, 2018 14:26To: n...@hq.sk; tompante...@gmail.comCc: controller-dev@lists.opendaylight.org; mdsal-...@lists.opendaylight.org; t...@lists.opendaylight.orgSubject: Re: Build passing for proposed topic:CONTROLLER-1802 ("Weather Item" CONTROLLER-1802)  Robert & Tom,On Thu, May 3, 2018 at 7:00 PM, Michael Vorburger  wrote:Dear maintainers of projects controller, mdsal,While verifying the proposed cross-projects changes on managed topic:[( ${gerritTopicName} )], all of your projects and their downstreams passed, when built together:    https://jenkins.opendaylight.org/releng/job/integration-multipatch-test-fluorine/25/Please decide whether you need CSIT/s to run for this.  If not, get together and organize a "simultaneous merge" of all of these changes:    https://git.opendaylight.org/gerrit/#/q/topic:CONTROLLER-1802based on this (build passing, no FB impact anywhere), would either of you who are both committers on both controller and mdsal like to go ahead and (at the same time) merge https://git.opendaylight.org/gerrit/#/c/66360/ and https://git.opendaylight.org/gerrit/#/c/66361/ ? Perhaps first thing on Monday, just in case - not to spoil anyone's  week-end.Tx,M.--Michael Vorburger, Red Hatvorbur...@redhat.com | IRC: vorburger @freenode | ~ = http://vorburger.ch If you would like to reach a human for any questions about this, please do not reply to this email, but reach out to the Reporter of JIRA issue https://jira.opendaylight.org/browse/TSC-100.Yours sincerely,The ODL Bot 


___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Giving Binding Specification V1 a bit of TLC

[Robert Varga]

On 05/03/18 16:52, Robert Varga wrote:
> Hello everyone,

Hello again,

as per today's TSC call, with regards to upcoming weather items from
MD-SAL, here is an update to the plan:

> we are still ways off of having a smooth transition plan from Binding V1
> to Binding V2 and a large body of models are getting standardized, some
> of which trigger V1 deficiencies.
> 
> Contingent on favorable outcome of the discussion here:
> https://lists.opendaylight.org/pipermail/mdsal-dev/2018-March/001517.html,

This was presented and discussed at a TSC call, with overall decision to
move forward and has been fully executed.

> it would be possible to fix majority of these with very few changes,
> which will break pretty much everyone out there, but is ways that are:
> - obvious compilation breakages
> - can be fixed mechanically
> 
> Specifically I mean the following changes (warning, technical details
> ahead):
> 
> 1) Map identity statements to interfaces, not abstract classes

This item has been completed the weekend after ONS.

> 
> RFC6020 specified single inheritance for identities, which meant mapping
> them to abstract classes worked as expected. RFC7950 changed the
> underlying meta model and allowed identities to have multiple base
> statements -- leading to multiple inheritance. Java does not support
> multiple inheritance of state, hence we currently cannot represent this
> YANG 1.1 construct in generated code.
> 
> The change from abstract class to interface is fully source-compatible
> for all use cases that are supported. Invalid uses (like subclassing a
> generated identity) will break and will require a mechanical source code
> update.
> 
> 
> 2) Replace org.opendaylight.yangtools.yang.binding.Identifiable.getKey()

This is https://jira.opendaylight.org/browse/TSC-101, i.e. this is not
immediately pending, until there is will to combine it with TSC-99,
below. It is expected to be more invasive than TSC-99 in downstreams.

> 
> This is the most major pain in models, as the name chosen clashes with
> the namespace used by generated code for getters. This means any
> construct like:
> 
> list foo {
> key ...;
> leaf key { ... }
> }
> 
> leads either to a codegen failure on ClassCastException or results in
> code which cannot be compiled.
> 
> The solution is to rename getKey() to key(). As all user-governed names
> in generated code is prefixed (with "is", "get" and similar), this will
> permanently solve this particular problem. Unfortunately it will also
> break all code which deals with getting/putting elements into keyed lists.

This also applies to getAugmentation(), where the change is from
'getAugmentation()' to 'augmentation()' (for lack of imagination).

An example of downstream impact can be seen in BGPCEP, which is heavily
modularized using augmentations, in the unfinished patch:
https://git.opendaylight.org/gerrit/#/c/71259/.

> The fix is a mechanical one and probably scriptable (but I am not much
> for writing scripts).
> 
> 
> 3) Change the signature of generated RPC invocation methods

This is https://jira.opendaylight.org/browse/TSC-99, which we would like
to merge soon. The exact date is to be discussed at the TSC call on May 3rd.

> This change has two facets, the first of which is convenience to callers
> of RPCs: these are currently defined as returning a
> java.util.concurrent.Future. The MD-SAL core implementation and all RPC
> implementations are using subclasses of ListenableFuture while
> asynchronous end users tend to JdkFutureAdapters or straightout casts to
> change the Future into a ListenableFuture.
> 
> The other facet is the request and response type when the RPC
> declaration in YANG does not have an input or output statement:
> 
> rpc foo { };
> rpc bar { input { } };
> rpc bar { output { } };
> 
> Such RPCs are mapped to either not having an input argument or returning
> a Future. This is a design mistake, as YANG specification calls
> for input/output statements to be implicitly present -- they are valid
> targets for "augment" statement irrespective of whether they were
> explicitly declared. While we can compile such models, all of the
> information in such augmentations is inaccessible to Binding users, as
> there is no anchor representation class generated for input/output
> statements.
> 
> I propose this to be fixed three changes:
> a) always generate Input/Output statements for RPCs
> b) always take an Input as the argument of generated method
> c) always return ListenableFuture from generated method

After going through the implementation phase, delivery in these three
steps would not be feasible due to internal structure of codegen :(
Hence the fix is struct

Re: [controller-dev] Help needed in RPC input format

[Robert Varga]

On 19/04/18 14:45, Gobinath . wrote:
> Hi All,
> 
>  
> 
> I’m developing a new Routed RPC in openflowplugin project which uses
> “NodeContextReference” in the input of the RPC(which have been used in
> many routed rpcs too).
> 
> Could you suggest how to frame the input body for such RPCs which use
> such references in the input.
> 
>  
> 
> *_Yang File:_*
> 
> *module **openflow-upgrade *{
>     *namespace
> **"urn:opendaylight:params:xml:ns:yang:openflowplugin:app:openflow-upgrade:service"*;
>     *prefix **openflow-upgrade*;
> 
>     *import **opendaylight-inventory *{*prefix **inv*; *revision-date
> **"2013-08-19"*;}
> 
>    *rpc **finish-upgrade-reconciliation *{
>     *description **"Stops the upgrade reconciliation of a
> node"*;
>     *input *{
>     *uses **"inv:node-context-ref"*;

defines a 'node' leaf, which is an instance-identifer.

>     *leaf **nodeId *{
>     *description **"Node for which the upgrade
> reconciliation has to be stopped"*;
>     *type **uint64*;
>     }
>     }
> 
>     *output *{
>  *leaf **result *{
>  *type **boolean*;
>  }
>     }
>  }
> }
> 
>  
> 
> The plain format
> 
> {
> 
> "input" :  {
> 
> "nodeId": "openflow:141097977278828"


there is no 'node' leaf present.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] Web Framework Discussion (continued from Kernel Call)

[Robert Varga]

On 11/04/18 17:50, Ryan Goulding wrote:
> Hi All,

Hello Ryan,

> During the Kernel call this past Tuesday, we talked about attempting an
> isolated transition of AAA restful web services from Jersey 1 to Jersey
> 2.  I attempted this change yesterday, and was able to partially convert
> (I just temporarily removed non-essential code that would've required
> overhaul).  However, when I compiled NETCONF next to test RESTCONF, I
> quickly realized that:
> 
> 1) jersey-2.26 won't behave well, since it relies on javax.ws.rs-api 2.1
> and jersey 1.17 relies on javax.ws.rs-api 2.0.1.  This leads to a Uses
> constraint violation since the dependency is provided via two chains
> (and two different versions too!).

Well, that upgrade has to wait for Neon then -- we simply cannot take
much more churn this low in the project.

> 2) jersey-2.25 won't work for a similar reason.  Even though it relies
> on the older javax.ws.rs-api 2.0.1 which is currently in place, jersey
> 1.17 repackages javax.ws.rs-api.  This means that utilizing the off the
> shelf javax.ws.rs-api 2.0.1 causes another Uses constraint violation,
> since the dependency is provided via upstream properly and jersey 1.17
> in a repackaged form.
> 
> I am starting to really agree with the sentiment that we should just
> stick to only one implementation across the board.  Additionally, I
> believe that isolating this in an API (utility or not) will help the
> transition since there will be a single point to toggle the
> implementations.  We may want to also discuss the drawbacks of jersey
> 2.  Namely, it appears to require a ton of overhead dependencies and
> starts a bit slower in newer versions.  Maybe that is fine, but we
> should fully understand the tradeoffs before investing more time.  We
> should also settle on what the intended version should be for jersey 2
> if we go that route, since jersey-2.26 is a lot different than even
> jersey-2.25.

Agreed, Tom's utility approach is probably the most sensible one. I took
a very simple stab at that with https://git.opendaylight.org/gerrit/70919.

I do belive using this you can eliminate any and all references to
jersey in shiro-impl.

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [mdsal-dev] shaded.org.eclipse.aether.resolution.ArtifactResolutionException: Error resolving artifact org.opendaylight.mdsal.model:ietf-ip-2014-06-16:jar:2014.06.16.13.0-SNAPSHOT

[Robert Varga]

On 11/04/18 14:54, Michael Vorburger wrote:
> Hello,
> 
> I'm hitting the error shown below on a local genius build from master -
> anyone knows what that is?
> 
> Did anything recently change re.
> mdsal.model:ietf-ip-2014-06-16:jar:2014.06.16.13.0-SNAPSHOT ?

It's a brand spanking new artifact (just went in today). It is nexus, so
perhaps it was a timing thing?

Regards,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev