Hello,

we are facing a weird problem in OpenDaylight, trying to upgrade from
4.2.2 to 4.2.6.

The problem manifests as Akka's failure to find a class via reflection
when a one-shot installation of the entire stack (which is a tree of
features) is performed.

I was able to narrow it down to what looks like a failure to wire
optional dependencies during feature:install and those taking effect
only after the next install.

This is using
https://nite.hq.sk/distribution.opendaylight-karaf-1.13.0-SNAPSHOT.tar.gz,
as otherwise the distro is quite cumbersome to build due to multiple
projects being involved.

1) feature:install odl-akka-persistence-2.5

> 2019-06-21T13:27:33,725 | INFO  | pipe-feature:install 
> odl-akka-persistence-2.5 | FeaturesServiceImpl              | 9 - 
> org.apache.karaf.features.core - 4.2.6 | Adding features: 
> odl-akka-persistence-2.5/[5.0.1.SNAPSHOT,5.0.1.SNAPSHOT]
> 2019-06-21T13:27:34,124 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Changes to perform:
> 2019-06-21T13:27:34,124 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   Region: root
> 2019-06-21T13:27:34,124 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |     Bundles to 
> install:
> 2019-06-21T13:27:34,124 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.google.guava/guava/27.1-jre
> 2019-06-21T13:27:34,124 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.google.guava/failureaccess/1.0.1
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe.akka/akka-actor_2.12/2.5.23
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe.akka/akka-osgi_2.12/2.5.23
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe.akka/akka-persistence_2.12/2.5.23
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe.akka/akka-protobuf_2.12/2.5.23
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe.akka/akka-slf4j_2.12/2.5.23
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe/config/1.3.3
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.fusesource.leveldbjni/leveldbjni-all/1.8-odl
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.scala-lang.modules/scala-java8-compat_2.12/0.8.0
> 2019-06-21T13:27:34,125 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.scala-lang/scala-library/2.12.8
> 2019-06-21T13:27:34,126 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.scala-lang/scala-reflect/2.12.8
> 2019-06-21T13:27:34,126 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> wrap:mvn:com.google.errorprone/error_prone_annotations/2.3.3
> 2019-06-21T13:27:34,126 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> wrap:mvn:com.google.guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava
> 2019-06-21T13:27:34,126 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> wrap:mvn:com.google.j2objc/j2objc-annotations/1.1
> 2019-06-21T13:27:34,126 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> wrap:mvn:org.checkerframework/checker-qual/2.5.8
> 2019-06-21T13:27:34,126 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> wrap:mvn:org.codehaus.mojo/animal-sniffer-annotations/1.17
> 2019-06-21T13:27:34,126 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> wrap:mvn:org.iq80.leveldb/leveldb/0.10
> 2019-06-21T13:27:34,127 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Installing bundles:
> 2019-06-21T13:27:34,127 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.google.guava/guava/27.1-jre
> 2019-06-21T13:27:34,138 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.google.guava/failureaccess/1.0.1
> 2019-06-21T13:27:34,140 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe.akka/akka-actor_2.12/2.5.23
> 2019-06-21T13:27:34,155 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe.akka/akka-osgi_2.12/2.5.23
> 2019-06-21T13:27:34,157 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe.akka/akka-persistence_2.12/2.5.23
> 2019-06-21T13:27:34,164 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe.akka/akka-protobuf_2.12/2.5.23
> 2019-06-21T13:27:34,168 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe.akka/akka-slf4j_2.12/2.5.23
> 2019-06-21T13:27:34,170 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe/config/1.3.3
> 2019-06-21T13:27:34,173 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.fusesource.leveldbjni/leveldbjni-all/1.8-odl
> 2019-06-21T13:27:34,182 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.scala-lang.modules/scala-java8-compat_2.12/0.8.0
> 2019-06-21T13:27:34,189 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.scala-lang/scala-library/2.12.8
> 2019-06-21T13:27:34,215 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.scala-lang/scala-reflect/2.12.8
> 2019-06-21T13:27:34,235 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap:mvn:com.google.errorprone/error_prone_annotations/2.3.3
> 2019-06-21T13:27:34,236 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap:mvn:com.google.guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava
> 2019-06-21T13:27:34,238 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap:mvn:com.google.j2objc/j2objc-annotations/1.1
> 2019-06-21T13:27:34,239 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap:mvn:org.checkerframework/checker-qual/2.5.8
> 2019-06-21T13:27:34,242 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap:mvn:org.codehaus.mojo/animal-sniffer-annotations/1.17
> 2019-06-21T13:27:34,244 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap:mvn:org.iq80.leveldb/leveldb/0.10
> 2019-06-21T13:27:34,298 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Starting bundles:
> 2019-06-21T13:27:34,301 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.config/1.3.3
> 2019-06-21T13:27:34,301 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.fusesource.leveldbjni.leveldbjni-all/1.8.0
> 2019-06-21T13:27:34,308 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.google.guava.failureaccess/1.0.1
> 2019-06-21T13:27:34,308 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.google.guava/27.1.0.jre
> 2019-06-21T13:27:34,310 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_org_iq80_leveldb_leveldb_0.10_leveldb-0.10.jar/0.0.0
> 2019-06-21T13:27:34,310 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.scala-lang.scala-library/2.12.8.v20181128-140630-VFINAL-38cd84d
> 2019-06-21T13:27:34,311 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.scala-lang.modules.scala-java8-compat/0.8.0
> 2019-06-21T13:27:34,312 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_org_checkerframework_checker-qual_2.5.8_checker-qual-2.5.8.jar/0.0.0
> 2019-06-21T13:27:34,313 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.scala-lang.scala-reflect/2.12.8.v20181128-140630-VFINAL-38cd84d
> 2019-06-21T13:27:34,314 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.protobuf/2.5.23
> 2019-06-21T13:27:34,314 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.osgi/2.5.23
> 2019-06-21T13:27:34,314 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_com_google_errorprone_error_prone_annotations_2.3.3_error_prone_annotations-2.3.3.
> jar/0.0.0
> 2019-06-21T13:27:34,315 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_com_google_j2objc_j2objc-annotations_1.1_j2objc-annotations-1.1.jar/0.0.0
> 2019-06-21T13:27:34,315 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_org_codehaus_mojo_animal-sniffer-annotations_1.17_animal-sniffer-annotations-1.17.
> jar/0.0.0
> 2019-06-21T13:27:34,315 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.slf4j/2.5.23
> 2019-06-21T13:27:34,315 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.actor/2.5.23
> 2019-06-21T13:27:34,316 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.persistence/2.5.23
> 2019-06-21T13:27:34,317 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_com_google_guava_listenablefuture_9999.0-empty-to-avoid-conflict-with-guava_listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar/0.0.0
> 2019-06-21T13:27:34,317 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Done.

At this point everything required for
com.typesafe.akka.persistence/2.5.23 is expected to be wired up and
ready to use.

Examining 'headers', 'imports' and 'exports' is finding nothing amiss.

2) feature:install odl-mdsal-clustering-commons

> 2019-06-21T13:28:24,570 | INFO  | pipe-feature:install 
> odl-mdsal-clustering-commons | FeaturesServiceImpl              | 9 - 
> org.apache.karaf.features.core - 4.2.6 | Adding features: 
> odl-mdsal-clustering-commons/[1.10.0.SNAPSHOT,1.10.0.SNAPSHOT]
> 2019-06-21T13:28:24,838 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Changes to perform:
> 2019-06-21T13:28:24,839 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   Region: root
> 2019-06-21T13:28:24,839 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |     Bundles to 
> install:
> 2019-06-21T13:28:24,839 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.esotericsoftware/kryo/4.0.2
> 2019-06-21T13:28:24,839 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.esotericsoftware/minlog/1.3.1
> 2019-06-21T13:28:24,839 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.esotericsoftware/reflectasm/1.11.8
> 2019-06-21T13:28:24,839 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.rabbitmq/amqp-client/4.4.1
> 2019-06-21T13:28:24,840 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe.akka/akka-cluster_2.12/2.5.23
> 2019-06-21T13:28:24,840 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe.akka/akka-remote_2.12/2.5.23
> 2019-06-21T13:28:24,840 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe.akka/akka-stream_2.12/2.5.23
> 2019-06-21T13:28:24,840 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:com.typesafe/ssl-config-core_2.12/0.3.7
> 2019-06-21T13:28:24,840 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:io.aeron/aeron-client/1.15.3
> 2019-06-21T13:28:24,840 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:io.aeron/aeron-driver/1.15.3
> 2019-06-21T13:28:24,841 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:io.dropwizard.metrics/metrics-core/4.0.5
> 2019-06-21T13:28:24,841 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:io.dropwizard.metrics/metrics-graphite/4.0.5
> 2019-06-21T13:28:24,841 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:io.dropwizard.metrics/metrics-healthchecks/4.0.5
> 2019-06-21T13:28:24,841 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:io.dropwizard.metrics/metrics-jmx/4.0.5
> 2019-06-21T13:28:24,841 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:io.dropwizard.metrics/metrics-jvm/4.0.5
> 2019-06-21T13:28:24,842 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:javax.servlet/javax.servlet-api/3.1.0
> 2019-06-21T13:28:24,842 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.agrona/agrona/0.9.33
> 2019-06-21T13:28:24,842 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.antlr/antlr4-runtime/4.7.2
> 2019-06-21T13:28:24,842 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.apache.commons/commons-lang3/3.9
> 2019-06-21T13:28:24,842 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:io.netty/netty/3.10.6.Final
> 2019-06-21T13:28:24,842 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.ow2.asm/asm/5.2
> 2019-06-21T13:28:24,843 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.objenesis/objenesis/2.6
> 2019-06-21T13:28:24,843 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.controller/sal-akka-raft/1.10.0-SNAPSHOT
> 2019-06-21T13:28:24,843 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.controller/sal-akka-segmented-journal/1.10.0-SNAPSHOT
> 2019-06-21T13:28:24,843 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.controller/sal-clustering-commons/1.10.0-SNAPSHOT
> 2019-06-21T13:28:24,843 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/concepts/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,843 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/odlext-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,843 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/openconfig-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,844 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/rfc6241-model-api/1.0.3-SNAPSHOT
> 2019-06-21T13:28:24,844 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/rfc6536-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,844 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/rfc7952-data-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,844 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/rfc7952-data-util/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,844 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/rfc7952-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,844 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/rfc8040-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,845 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/rfc8528-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,845 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/util/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,845 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-common/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,845 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-data-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,845 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-data-impl/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,845 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-data-transform/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,846 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-data-util/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,846 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,846 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-model-util/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,846 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-parser-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,846 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.opendaylight.yangtools/yang-xpath-api/1.0.3-SNAPSHOT
> 2019-06-21T13:28:24,846 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.reactivestreams/reactive-streams/1.0.2
> 2019-06-21T13:28:24,847 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:org.scala-lang.modules/scala-parser-combinators_2.12/1.1.2
> 2019-06-21T13:28:24,847 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |       
> mvn:tech.pantheon.triemap/triemap/1.0.4
> 2019-06-21T13:28:24,847 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Stopping bundles:
> 2019-06-21T13:28:24,848 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_org_iq80_leveldb_leveldb_0.10_leveldb-0.10.jar/0.0.0
> 2019-06-21T13:28:24,849 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.persistence/2.5.23

This is unexpected -- these bundles were already installed and are not
being touched. Refresh logs are probably going to explain this...

> 2019-06-21T13:28:24,850 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Installing bundles:
> 2019-06-21T13:28:24,851 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.esotericsoftware/kryo/4.0.2
> 2019-06-21T13:28:24,854 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.esotericsoftware/minlog/1.3.1
> 2019-06-21T13:28:24,856 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.esotericsoftware/reflectasm/1.11.8
> 2019-06-21T13:28:24,858 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.rabbitmq/amqp-client/4.4.1
> 2019-06-21T13:28:24,862 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe.akka/akka-cluster_2.12/2.5.23
> 2019-06-21T13:28:24,868 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe.akka/akka-remote_2.12/2.5.23
> 2019-06-21T13:28:24,881 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe.akka/akka-stream_2.12/2.5.23
> 2019-06-21T13:28:24,905 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:com.typesafe/ssl-config-core_2.12/0.3.7
> 2019-06-21T13:28:24,908 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:io.aeron/aeron-client/1.15.3
> 2019-06-21T13:28:24,910 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:io.aeron/aeron-driver/1.15.3
> 2019-06-21T13:28:24,913 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:io.dropwizard.metrics/metrics-core/4.0.5
> 2019-06-21T13:28:24,915 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:io.dropwizard.metrics/metrics-graphite/4.0.5
> 2019-06-21T13:28:24,917 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:io.dropwizard.metrics/metrics-healthchecks/4.0.5
> 2019-06-21T13:28:24,919 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:io.dropwizard.metrics/metrics-jmx/4.0.5
> 2019-06-21T13:28:24,920 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:io.dropwizard.metrics/metrics-jvm/4.0.5
> 2019-06-21T13:28:24,922 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:javax.servlet/javax.servlet-api/3.1.0
> 2019-06-21T13:28:24,923 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.agrona/agrona/0.9.33
> 2019-06-21T13:28:24,927 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.antlr/antlr4-runtime/4.7.2
> 2019-06-21T13:28:24,930 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.apache.commons/commons-lang3/3.9
> 2019-06-21T13:28:24,934 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:io.netty/netty/3.10.6.Final
> 2019-06-21T13:28:24,944 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.ow2.asm/asm/5.2
> 2019-06-21T13:28:24,946 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.objenesis/objenesis/2.6
> 2019-06-21T13:28:24,949 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.controller/sal-akka-raft/1.10.0-SNAPSHOT
> 2019-06-21T13:28:24,953 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.controller/sal-akka-segmented-journal/1.10.0-SNAPSHOT
> 2019-06-21T13:28:24,956 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.controller/sal-clustering-commons/1.10.0-SNAPSHOT
> 2019-06-21T13:28:24,959 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/concepts/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,960 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/odlext-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,961 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/openconfig-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,963 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/rfc6241-model-api/1.0.3-SNAPSHOT
> 2019-06-21T13:28:24,964 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/rfc6536-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,965 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/rfc7952-data-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,966 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/rfc7952-data-util/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,967 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/rfc7952-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,968 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/rfc8040-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,969 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/rfc8528-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,970 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/util/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,971 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-common/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,972 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-data-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,974 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-data-impl/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,977 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-data-transform/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,978 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-data-util/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,980 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-model-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,982 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-model-util/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,984 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-parser-api/3.0.3-SNAPSHOT
> 2019-06-21T13:28:24,985 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.opendaylight.yangtools/yang-xpath-api/1.0.3-SNAPSHOT
> 2019-06-21T13:28:24,986 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.reactivestreams/reactive-streams/1.0.2
> 2019-06-21T13:28:24,987 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:org.scala-lang.modules/scala-parser-combinators_2.12/1.1.2
> 2019-06-21T13:28:24,989 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> mvn:tech.pantheon.triemap/triemap/1.0.4
> 2019-06-21T13:28:24,992 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Refreshing bundles:
> 2019-06-21T13:28:24,992 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |     
> com.typesafe.akka.persistence/2.5.23 (Should be wired to: 
> org.fusesource.leveldbjni.leveldbjni-all/1.8.0 (through 
> [com.typesafe.akka.persistence/2.5.23] osgi.wiring.package; 
> resolution:=optional; 
> filter:="(&(osgi.wiring.package=org.fusesource.leveldbjni)(version>=1.8.0)(!(version>=2.0.0)))"))
> 2019-06-21T13:28:24,992 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |     
> wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_org_iq80_leveldb_leveldb_0.10_leveldb-0.10.jar/0.0.0
>  (Should be wired to: org.fusesource.leveldbjni.leveldbjni-all/1.8.0 (through 
> [wrap_file__home_nite_odl_controller_karaf_distribution.opendaylight-karaf-1.13.0-SNAPSHOT_system_org_iq80_leveldb_leveldb_0.10_leveldb-0.10.jar/0.0.0]
>  osgi.wiring.package; resolution:=optional; 
> filter:="(osgi.wiring.package=org.iq80.leveldb)"))

... but it does not. persistence is being refreshed to be wired to an
optional package, which was already installed and is not being touched
in any way.

> 2019-06-21T13:28:25,095 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Starting bundles:
> 2019-06-21T13:28:25,104 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.concepts/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,105 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-common/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,105 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> tech.pantheon.triemap/1.0.4
> 2019-06-21T13:28:25,105 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.util/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,106 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-xpath-api/1.0.3.SNAPSHOT
> 2019-06-21T13:28:25,106 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-model-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,107 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.rfc7952-model-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,107 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.scala-lang.modules.scala-parser-combinators/1.1.2
> 2019-06-21T13:28:25,107 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.sslconfig/0.3.7
> 2019-06-21T13:28:25,108 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.reactivestreams.reactive-streams/1.0.2
> 2019-06-21T13:28:25,108 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.stream/2.5.23
> 2019-06-21T13:28:25,110 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   org.agrona/0.9.33
> 2019-06-21T13:28:25,111 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> io.aeron.client/1.15.3
> 2019-06-21T13:28:25,111 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-data-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,111 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.rfc7952-data-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,112 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.odlext-model-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,112 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-model-util/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,112 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-data-util/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,113 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.rfc7952-data-util/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,113 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> io.dropwizard.metrics.core/4.0.5
> 2019-06-21T13:28:25,114 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.rabbitmq.client/4.4.1
> 2019-06-21T13:28:25,114 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> io.dropwizard.metrics.graphite/4.0.5
> 2019-06-21T13:28:25,115 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> io.dropwizard.metrics.jvm/4.0.5
> 2019-06-21T13:28:25,115 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> io.dropwizard.metrics.healthchecks/4.0.5
> 2019-06-21T13:28:25,115 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.antlr.antlr4-runtime/4.7.2
> 2019-06-21T13:28:25,116 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-data-impl/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,170 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-data-transform/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,171 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.openconfig-model-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,171 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> io.aeron.driver/1.15.3
> 2019-06-21T13:28:25,172 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> javax.servlet-api/3.1.0
> 2019-06-21T13:28:25,173 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.jboss.netty/3.10.6.Final
> 2019-06-21T13:28:25,178 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.remote/2.5.23
> 2019-06-21T13:28:25,179 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> io.dropwizard.metrics.jmx/4.0.5
> 2019-06-21T13:28:25,180 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.esotericsoftware.minlog/1.3.1
> 2019-06-21T13:28:25,180 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.objectweb.asm/5.2.0
> 2019-06-21T13:28:25,181 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.esotericsoftware.reflectasm/1.11.8
> 2019-06-21T13:28:25,181 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.objenesis/2.6.0
> 2019-06-21T13:28:25,181 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.esotericsoftware.kryo/4.0.2
> 2019-06-21T13:28:25,182 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.apache.commons.lang3/3.9.0
> 2019-06-21T13:28:25,182 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.rfc8040-model-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,183 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> com.typesafe.akka.cluster/2.5.23
> 2019-06-21T13:28:25,183 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.rfc6536-model-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,184 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.rfc6241-model-api/1.0.3.SNAPSHOT
> 2019-06-21T13:28:25,184 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.rfc8528-model-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,184 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.yangtools.yang-parser-api/3.0.3.SNAPSHOT
> 2019-06-21T13:28:25,185 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.controller.sal-akka-segmented-journal/1.10.0.SNAPSHOT
> 2019-06-21T13:28:25,185 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.controller.sal-akka-raft/1.10.0.SNAPSHOT
> 2019-06-21T13:28:25,186 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 |   
> org.opendaylight.controller.sal-clustering-commons/1.10.0.SNAPSHOT
> 2019-06-21T13:28:25,186 | INFO  | features-3-thread-1 | FeaturesServiceImpl   
>            | 9 - org.apache.karaf.features.core - 4.2.6 | Done.

After this is done, everything works as expected. Installing further
features from the stack do not trigger any further refreshes and
everything Just Works (tm). This can be confirmed by installing
'odl-mdsal-distributed-datastore'.

Also, installing any feature, even completely unrelated to
odl-akka-persistence-2.5, like odl-antlr4, triggers the refresh above,
which brings the system into a working state.

The same result is not obtained if there is only a single
'feature:install odl-mdsal-distributed-datastore' run -- i.e. the system
will creash with:

> Uncaught error from thread 
> [opendaylight-cluster-data-akka.actor.default-dispatcher-18]: 
> org/iq80/leveldb/DBFactory, shutting down JVM since 
> 'akka.jvm-exit-on-fatal-error' is enabled for 
> ActorSystem[opendaylight-cluster-data]
> java.lang.NoClassDefFoundError: org/iq80/leveldb/DBFactory
>         at java.lang.Class.getDeclaredConstructors0(Native Method)
>         at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
>         at java.lang.Class.getDeclaredConstructors(Class.java:2020)
>         at akka.util.Reflect$.findConstructor(Reflect.scala:92)
>         at 
> akka.persistence.Persistence.akka$persistence$Persistence$$createPlugin(Persistence.scala:408)
>         at 
> akka.persistence.Persistence$PluginHolderExtensionId.createExtension(Persistence.scala:443)
>         at 
> akka.persistence.Persistence$PluginHolderExtensionId.createExtension(Persistence.scala:433)
>         at 
> akka.actor.ActorSystemImpl.registerExtension(ActorSystem.scala:1006)
>         at akka.actor.ExtensionId.apply(Extension.scala:79)
>         at akka.actor.ExtensionId.apply$(Extension.scala:78)
>         at 
> akka.persistence.Persistence$PluginHolderExtensionId.apply(Persistence.scala:433)
>         at akka.persistence.Persistence.pluginHolderFor(Persistence.scala:391)
>         at 
> akka.persistence.Persistence.journalConfigFor(Persistence.scala:338)
>         at akka.persistence.Eventsourced.$init$(Eventsourced.scala:97)
>         at 
> akka.persistence.AbstractPersistentActor.<init>(PersistentActor.scala:454)
>         at 
> org.opendaylight.controller.cluster.access.client.AbstractClientActor.<init>(AbstractClientActor.java:28)
>         at 
> org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientActor.<init>(AbstractDataStoreClientActor.java:33)
>         at 
> org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientActor.<init>(DistributedDataStoreClientActor.java:26)
>         at 
> org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientActor.lambda$props$c06de021$1(DistributedDataStoreClientActor.java:39)
>         at akka.actor.CreatorConsumer.produce(IndirectActorProducer.scala:82)
>         at akka.actor.Props.newActor(Props.scala:212)
>         at akka.actor.ActorCell.newActor(ActorCell.scala:648)
>         at akka.actor.ActorCell.create(ActorCell.scala:674)
>         at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:547)
>         at akka.actor.ActorCell.systemInvoke(ActorCell.scala:569)
>         at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:293)
>         at akka.dispatch.Mailbox.run(Mailbox.scala:228)
>         at akka.dispatch.Mailbox.exec(Mailbox.scala:241)
>         at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>         at 
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>         at 
> akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>         at 
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> Caused by: java.lang.ClassNotFoundException: org.iq80.leveldb.DBFactory 
> cannot be found by com.typesafe.akka.persistence_2.5.23
>         at 
> org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:484)
>         at 
> org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:395)
>         at 
> org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:387)
>         at 
> org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:150)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         ... 32 more

This works reliably with karaf-4.2.2 and fails reliably with
karaf-4.2.6. Any ideas as to what may be causing this?

Thanks,
Robert

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to