On Tue, Jun 26, 2018 at 12:52 PM, Vishal Thapar <vtha...@redhat.com> wrote:
> Hi Michael, > > I haven't checked but when you say even srm wasn't present does this mean > the Genius one was missing too? We can probably look at log collected and > figure out a subset of the context that we want in logs and just capture > that. > no I shold clarify: "serviceutils.srm" is not available in that "runtimeTypes" of that BindingRuntimeContext, but "genius.srm[.rpcs.rev170711]" is. FYI anyone following this, check last comment on https://jira.opendaylight.org/browse/MDSAL-354 re. reproducing this with a simpler SFT without all of integration/distribution. > > Regards, > Vishal. > > On Tue, Jun 26, 2018 at 4:11 PM, Michael Vorburger <vorbur...@redhat.com> > wrote: > >> On Tue, Jun 26, 2018 at 12:03 PM, Faseela K <faseel...@ericsson.com> >> wrote: >> >>> Tried building a distribution with the MDSAL logging patches, and >>> attaching the karaf logs. >>> >> >> I've also attached it to https://jira.opendaylight.org/browse/MDSAL-354 >> >> >>> The log is huge with the new debugs added ;) >>> >> >> agreed that while this may be useful to understnd the current problem, >> this is perhaps excessive. >> >> >>> I am analyzing the logs to see whether there are any errors which I can >>> easily figure out. >>> >> >> as far as I can tell, il, it would seem that neither "serviceutils" nor >> "srm" is available in that "runtimeTypes" of the BindingRuntimeContext, >> which is weird/wrong? Why that didn't get loaded is probably the crux to >> figuring this out. It's of course defined in the srm-api bundle, and I >> don't see any error messaging saying that had any trouble... hm. >> >> >>> Thanks, >>> >>> Faseela >>> >>> >>> >>> *From:* Michael Vorburger [mailto:vorbur...@redhat.com] >>> *Sent:* Thursday, June 21, 2018 10:51 PM >>> *To:* Vishal Thapar <vtha...@redhat.com> >>> *Cc:* Faseela K <faseel...@ericsson.com>; controller-dev < >>> controller-dev@lists.opendaylight.org>; mdsal-...@lists.opendaylight.org; >>> Stephen Kitt <sk...@redhat.com>; serviceutils-...@lists.opendaylight.org >>> >>> *Subject:* Re: Serviceutils distro failure >>> >>> >>> >>> On Thu, Jun 21, 2018 at 6:32 PM, Vishal Thapar <vtha...@redhat.com> >>> wrote: >>> >>> Hi Michael, >>> >>> >>> >>> I was able to reproduce in my local distro build, >>> >>> >>> >>> Cool! Can you describe the exact steps one has to take? Like literally >>> "for dummies", describe for any interested in locally reproducing this >>> exactly what you did... ;-) >>> >>> >>> >>> so let me know what logging to enable, I can take a shot. >>> >>> >>> >>> If you could locally "mvn clean install" rebuild with the patches from >>> https://git.opendaylight.org/gerrit/#/q/topic:MDSAL-354, and share a >>> new log which includes that - that would be very intersting! (There >>> will be [probably MUCH] more after that "IllegalStateException: Schema for >>> interface org.opendaylight.yang.gen.v1.urn.opendaylight.serviceutils.s >>> rm.rpcs.rev170711.SrmRpcsService is not available." ) >>> >>> >>> >>> http://paste.openstack.org/show/i0tBL9XMbgob0iqPkQvd/ >>> >>> >>> >>> right; that's exactly the same as originally, below. >>> >>> >>> >>> Regards, >>> >>> Vishal. >>> >>> >>> >>> On Thu, Jun 21, 2018 at 9:32 PM, Michael Vorburger <vorbur...@redhat.com> >>> wrote: >>> >>> On Thu, Jun 21, 2018 at 5:42 PM, Faseela K <faseel...@ericsson.com> >>> wrote: >>> >>> Hi, >>> >>> The issue cannot be reproduced locally, even when I try to install >>> odl-integration-all from distribution. >>> >>> But whenever I download the distribution built on Jenkins[0] from my >>> patch to enable serviceutils in distribution, the feature:install fails >>> with the same error as below. >>> >>> >>> >>> OK but so if it's only possible to reproduce this locally with a binary >>> blob that we cannot trust on how it was actually built (which is quite >>> concerning!), and if I understand correctly what you are saying we cannot >>> reproduce this in a local bulid where we could include logging / debug >>> patches, then as a next step I would suggest we wait for the additional >>> logging I've just proposed via https://git.opendaylight.org/g >>> errit/#/q/topic:MDSAL-354 to be merged to master, and then reproduce >>> this on a new distribution job run including those patches, and look at >>> those logs and take it from there. Sounds like a plan? >>> >>> >>> >>> Thanks, >>> >>> Faseela >>> >>> >>> >>> [0] https://nexus.opendaylight.org/content/repositories/opendayl >>> ight.snapshot/org/opendaylight/integration/integration/distr >>> ibution/opendaylight/0.9.0-SNAPSHOT/opendaylight-0.9.0-20 >>> 180621.134056-7.zip >>> >>> >>> >>> *From:* Michael Vorburger [mailto:vorbur...@redhat.com] >>> *Sent:* Thursday, June 21, 2018 6:56 PM >>> *To:* controller-dev <controller-dev@lists.opendaylight.org>; >>> mdsal-...@lists.opendaylight.org >>> *Cc:* Stephen Kitt <sk...@redhat.com>; serviceutils-dev@lists.openday >>> light.org; Vishal Thapar <vtha...@redhat.com>; Faseela K < >>> faseel...@ericsson.com> >>> *Subject:* Re: Serviceutils distro failure >>> >>> >>> >>> On Thu, Jun 21, 2018 at 1:41 PM, Michael Vorburger <vorbur...@redhat.com> >>> wrote: >>> >>> +controller-dev & mdsal-dev: >>> >>> >>> >>> On Thu, Jun 21, 2018 at 4:57 AM, Vishal Thapar <vtha...@redhat.com> >>> wrote: >>> >>> Hi Michael, Stephen, >>> >>> >>> >>> We are unable to enable serviceutils in distro due to failure in distro >>> job. I tried looking at logs, it shows something wrong with srm-shell, but >>> it works locally, even with clean m2, so not sure what are we missing. Any >>> inputs? >>> >>> >>> >>> *18:40:41* 2018-06-20T18:40:23,484 | ERROR | Blueprint Extender: 3 | >>> BlueprintContainerImpl | 82 - org.apache.aries.blueprint.core - >>> 1.8.3 | Unable to start blueprint container for bundle >>> org.opendaylight.serviceutils.srm-shell/0.2.0.SNAPSHOT due to unresolved >>> dependencies >>> [(&(|(type=default)(!(type=*)))(objectClass=org.opendaylight.mdsal.dom.api.DOMSchemaService))] >>> >>> >>> >>> https://jenkins.opendaylight.org/releng/job/distribution-che >>> ck-fluorine/69/consoleFull >>> >>> >>> >>> For background, this is with https://git.opendaylight. >>> org/gerrit/#/c/73212/, currently reverted on master. >>> >>> >>> >>> It passes locally (I just tried), so I suspect another timing related >>> issue - somehow the build in distribution is too slow perhaps. >>> >>> >>> >>> I had to refresh my own memory by looking at the code in >>> odlparent:bundles-test-lib and infrautils:ready, so just as a refresher: >>> This isn't actually "timing out", in this case. I was wrong to suggest that >>> "somehow the build in distribution is too slow perhaps" - there are no >>> timeouts to increase to get this to pass. It's *NOT* waiting those >>> hard-coded max. 5 minutes. What's happening here is that one of the bundles >>> is in "bundleState = Failure" (grep that log for that), and that fairly >>> quickly and early on - there are only 17s between the following two key log >>> messages: >>> >>> >>> >>> 2018-06-20T18:35:25,908 | INFO | awaitility[checkBundleDiagInfos] | >>> SystemReadyImpl | 350 - >>> org.opendaylight.infrautils.ready-impl >>> - 1.4.0.SNAPSHOT | checkBundleDiagInfos: Elapsed time 2s, remaining time >>> 297s, diag: Booting {Installed=0, Resolved=4, Unknown=0, GracePeriod=117, >>> Waiting=0, Starting=0, Active=485, Stopping=0, Failure=0} >>> >>> >>> >>> 2018-06-20T18:35:42,573 | ERROR | SystemReadyService-0 | >>> SystemReadyImpl | 350 - >>> org.opendaylight.infrautils.ready-impl >>> - 1.4.0.SNAPSHOT | Failed, some bundles did not start (SystemReadyListeners >>> are not called) >>> >>> org.opendaylight.odlparent.bundlestest.lib.SystemStateFailureException: >>> diag failed; some bundles failed to start >>> >>> diag: Failure {Installed=0, Resolved=4, Unknown=0, GracePeriod=54, >>> Waiting=0, Starting=0, Active=547, Stopping=0, Failure=1} >>> >>> >>> >>> The bundle that fails is in Blueprint initialization is the (new) >>> serviceutils:srm-impl, due to that weird schema not found - so figuring >>> that one really is the key to resolving this. >>> >>> >>> >>> Nothing new - just reconfirming previous message, after having stared >>> more at the logs. >>> >>> >>> >>> Tom or Robert, if you can help clarify how these schema are normally >>> found by the BindingToNormalizedNodeCodec, and what could cause one to >>> not be found, that we interesting... ;-) otherwise I guess we can have some >>> fun for the next weeks to learn more about this controller and mdsal code! >>> >>> >>> >>> The error shown about (unresolved dependencies ... DOMSchemaService) is >>> probably just an effect, not a cause. This which appears earlier in that >>> https://jenkins.opendaylight.org/releng/job/distribution-che >>> ck-fluorine/69/consoleFull log seems to be more interesting: >>> >>> >>> >>> 2018-06-20T18:35:42,573 | ERROR | SystemReadyService-0 | TestBundleDiag >>> | 350 - org.opendaylight.infrautils.ready-impl - >>> 1.4.0.SNAPSHOT | NOK org.opendaylight.serviceutils.srm-impl:0.2.0.SNAPSHOT: >>> OSGi state = Active, Karaf bundleState = Failure, due to: Declarative >>> Services >>> >>> Blueprint >>> >>> 6/20/18 6:35 PM >>> >>> Exception: >>> >>> Unable to initialize bean .component-2 >>> >>> org.osgi.service.blueprint.container.ComponentDefinitionException: Unable >>> to initialize bean .component-2 >>> >>> at >>> org.apache.aries.blueprint.container.BeanRecipe.runBeanProcInit(BeanRecipe.java:738) >>> >>> (...) >>> >>> Caused by: >>> org.osgi.service.blueprint.container.ComponentDefinitionException: Error >>> processing "rpc-implementation" for class >>> org.opendaylight.serviceutils.srm.impl.SrmRpcProvider >>> >>> at >>> org.opendaylight.controller.blueprint.ext.RpcImplementationBean.init(RpcImplementationBean.java:69) >>> >>> (...) >>> >>> Caused by: java.lang.IllegalStateException: Schema for interface >>> org.opendaylight.yang.gen.v1.urn.opendaylight.serviceutils.srm.rpcs.rev170711.SrmRpcsService >>> is not available. >>> >>> at >>> com.google.common.base.Preconditions.checkState(Preconditions.java:585) >>> >>> at >>> org.opendaylight.mdsal.binding.dom.adapter.BindingToNormalizedNodeCodec.getModuleBlocking(BindingToNormalizedNodeCodec.java:303) >>> >>> >>> >>> This doesn't look great, right? How can a Schema for a generated code >>> interface just be missing like this? But only in distribution, and not >>> locally reproducible? >>> >>> >>> >>> There is also an occurrence of https://jira.opendaylight.o >>> rg/browse/NETCONF-534 - not sure how worrying that should be? >>> >>> >>> >>> I've looked for (bundle / feature) "reload" in that log, but can't spot >>> anything (but could be missing it). >>> >>> >>> >>> Tx, >>> >>> M. >>> >>> -- >>> >>> Michael Vorburger, Red Hat >>> vorbur...@redhat.com | IRC: vorburger @freenode | ~ = >>> http://vorburger.ch >>> >>> >
_______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/controller-dev
