That's really a great improvement. Thanks for that ! Regards JB
On 12/06/2017 08:12 PM, Grzegorz Grzybek wrote:
Hello again I'd like to summary recent additions related to my changes: - feature:install and feature:repo-add won't install/add blacklisted feature/repository - etc/org.apache.karaf.features.cfg may use property placeholders and these will be resolved from bundle context properties and from etc/versions.properties (configurable). The idea is to have e.g.,: <bundleReplacements> <bundle replacement="mvn:commons-io/commons-io/${version.commons-io}" /> </bundleReplacements> and just list all the versions we need in etc/versions.properties. So when fixed bundle comes out, it's a matter of changing property in single file and all features will use overriden version. - implemented overriding feature definitions (replace entire feature definition) - implemented overriding dependency="true|false" flag in bundles. If we know that some external features XML should use this flag and it doesn't, we can configure it externally without trying to have our PR merged in upstream project. - profiles can use "blacklisted.bundle.xxx", "blacklisted.feature.xxx" and "blacklisted.repository.xxx" properties (KARAF-5339) - features specified in karaf-maven-plugin:assembly configuration may use wildcards (KARAF-5273) I still hope to get some comments/review ;) I know it's not quick fix. If you think I went in wrong direction, please let me know. best regards Grzegorz Grzybek 2017-12-02 21:27 GMT+01:00 Grzegorz Grzybek <gr.grzy...@gmail.com>:Hello I want to report about my progress with https://issues.apache.org/ jira/browse/KARAF-5376 (Processor mechanism for feature definitions (a.k.a. "better overrides")). *Goal* - to replace (or provide more flexible alternative) of "overrides" and "blacklisting" mechanisms *Idea* - Karaf features come (normally) from features XML files, according to XML Schema specified by Karaf itself. My idea was that we can "intercept" the process of loading the repositories and alter the definitions. Effectively we could then affect the definitions which may be out of our influence, or use some bundles we'd rather like to replace with others. Some feature repositories include other repositories (e.g., Camel 2.19.2 features referencing CXF 3.1.11 features) and we'd rather like to avoid it. *Problems with blacklisting and overrides* - blacklisting is purely static thing - applied at distro creation time. Overrides are a bit awkward to use ("range" clause) and have some implicit behavior (requiring runtime access to resource, to verify version and symbolic name). *What I did* In short words - I've created a mechanism steered by etc/org.apache.karaf.features.xml (name matching etc/org.apache.karaf.features.cfg PID) that looks roughly like this: <featuresProcessing xmlns="http://karaf.apache. org/xmlns/features-processing/v1.0.0" xmlns:f="http://karaf.apache.org/xmlns/features/v1.5.0";> <blacklistedRepositories> <repository>mvn:org.hibernate/hibernate-validator-osgi- karaf-features/[5,*)/xml/features</repository> <!-- ... --> </blacklistedRepositories> <blacklistedFeatures> <feature>*jetty*</feature> <feature version="[2,3)">*jclouds*</feature> <!-- ... --> </blacklistedFeatures> <blacklistedBundles> <bundle>mvn:commons-logging/*</bundle> </blacklistedBundles> <overrideBundleDependency> <!-- Override "dependency" flag for all bundles of all features for given repository URI(s) --> <repository uri="mvn:org.ops4j.pax.cdi/pax-cdi-features/*/xml/features" dependency="true" /> <repository uri="mvn:*/xml/features" dependency="true" /> <!-- Override "dependency" flag for all bundles of given feature(s) --> <feature name="jclouds*" version="[1,3)" dependency="true" /> <!-- Override "dependency" flag for given bundle(s) --> <bundle uri="mvn:javax.annotation/javax.annotation-api/*" dependency="true" /> ... </overrideBundleDependency> <bundleReplacements> <bundle originalUri="mvn:commons-beanutils/commons-beanutils/[ 1.9,2)" replacement="mvn:commons-beanutils/commons-beanutils/1.9.3" /> <bundle replacement="mvn:commons-collections/commons-collections/3.2.2" /> <bundle originalUri="mvn:org.eclipse.jetty.orbit/javax.servlet/[3, 4)" replacement="mvn:org.apache.geronimo.specs/geronimo-servlet_3.0_spec/1.0" mode="maven" /> <!-- ... --> </bundleReplacements> <featureReplacements> <replacement mode="replace"> <feature name="pax-jsf-resources-support" description="Provide sharing of resources according to Servlet 3.0 for OSGi bundles and JSF" version="6.0.7"> <f:feature version="[6.0,6.1)">pax-jsf-support</f:feature> <f:bundle dependency="true">mvn:org.ops4j.pax.web/pax-web- resources-extender/6.0.7</f:bundle> <f:bundle>mvn:org.ops4j.pax.web/pax-web-resources-jsf/6.0. 7</f:bundle> </feature> </replacement> <!-- ... --> </featureReplacements> </featuresProcessing> Description: - blacklistedRepositories, blacklistedFeatures and blacklistedBundles mean what they mean - mark (instead of removing) given item kind from JAXB model of loaded features XML. This information is preserved at runtime, and for example is displayed with "features:list" command - overrideBundleDependency (not yet implemented) - is a method of overriding "dependency="true|false"" flags in feature definitions - bundleReplacements (implemented) - is an extension of "overrides" mechanism. In "osgi" mode, it roughly works like before - requires access to runtime ResourceImpl, to check headers and compare versions and symbolic names. In "maven" mode it's just static change of given bundle URI - any feature declaring <bundle>originalUri</bundle> will be seen as declaring <bundle>replacement</bundle>. This allows for replacement of e.g., SMX specs bundle with javax.* bundle (or vice versa). - featureReplacements (not yet implemented) - is a way of changing (or altering) any features by name (or pattern) and version simply by providing different definition of given feature. This could be useful if we as Karaf users know "better" than authors of original features, for whom OSGi and Karaf were rather an afterthought. Both bundle/feature/repository blacklisting and bundle/feature override use two new classes to "match" candidate to replace/override: - org.apache.karaf.features.LocationPattern - matches URI (of bundle or repository) in two modes - for non mvn: URI, it just uses a string with possible "*" glob. For mvn: URI, it splits the URI into well known components (groupId, artifactId, version, type and classifier). For version it can use version range (e.g, "[2.1,2.2)") and for other components it can use "*" glob. - org.apache.karaf.features.FeaturePattern - matches feature by name (possibly containing "*" glob) and version range (org.apache.felix.utils. version.VersionRange) Universally applicable: FeaturesProcessor generally postprocesses org.apache.karaf.features.internal.model.Features instance(s) and marks items as blacklisted if they match given pattern and changes the model (according to overrides/replacements). The behavior after processing JAXB model of features is exactly the same both when using FeaturesServiceImpl at runtime and when using profile Builder during custom assembly creation. For example changed flow of karaf-maven-plugin:asembly invocation looks like this: [INFO] --- karaf-maven-plugin:4.2.0-SNAPSHOT:assembly (default-assembly) @ test-karaf-plugins --- [INFO] Using repositories: [INFO] https://repo.maven.apache.org/maven2@id=central ... [INFO] Custom startup KAR found: mvn:grgr.test.karaf.plugins/ my-kar/0.1.0.BUILD-SNAPSHOT/kar ... [INFO] Unzipping kars [INFO] processing KAR: mvn:grgr.test.karaf.plugins/ my-kar/0.1.0.BUILD-SNAPSHOT/kar [INFO] found repository: mvn:grgr.test.karaf.plugins/ my-kar/0.1.0.BUILD-SNAPSHOT/xml/features [INFO] Found features processor configuration: ../classes/etc/org.apache. karaf.features.xml ... [INFO] Loading profiles from: [INFO] file:///home/ggrzybek/sources/_testing/grgr-test-karaf- plugins/src/main/profiles/1 [INFO] jar:mvn:test/profiles/42!/profiles/2 [INFO] Found profiles: p1a, p1b-another, p2b, p2c-two, p2c-one-two, p2a ... [INFO] Generating features processor configuration: etc/org.apache.karaf.features.xml [INFO] Startup stage [INFO] Loading startup repositories [INFO] Resolving startup features and bundles [INFO] Features: undertow, my-startup-feature ... [INFO] skipping blacklisted maven bundle: mvn:org.ops4j.pax.tipi/org. ops4j.pax.tipi.undertow.websocket-jsr/1.3.25.1 ... [INFO] Boot stage [INFO] Loading boot repositories [INFO] adding feature repository: mvn:org.ops4j.pax.web/pax-web- features/6.0.7/xml/features [INFO] adding feature repository: mvn:grgr.test.karaf.plugins/ my-kar/0.1.0.BUILD-SNAPSHOT/xml/features [INFO] Feature undertow/1.3.25.1 is defined as a boot feature ... [INFO] adding overriden maven artifact: mvn:org.jboss.logging/jboss-logging/3.3.1.Final (original location: mvn:org.jboss.logging/jboss-logging/3.3.0.Final) ... [INFO] skipping blacklisted maven artifact: mvn:org.ops4j.pax.tipi/org.ops4j.pax.tipi.undertow.websocket-jsr/1.3.25.1 [INFO] adding overriden maven artifact: mvn:javax.servlet/javax.servlet-api/3.1-b09 (original location: mvn:javax.servlet/javax.servlet-api/3.1.0) ... [INFO] Install stage [INFO] Loading installed repositories overrides and blacklisting information may come from karaf-maven-plugin configuration (in POM), from externally provided org.apache.karaf.features.xml (e.g., in resource KAR or in POM: <configuration>/<featuresProcessing>) or from profiles. There are still few things to polish, but I'll appreciate any feedback to check if I misunderstood something or there were other plans related to these mechanisms. The changes are in https://github.com/apache/ karaf/commits/KARAF-5376-overrides_v2 branch. best regards Grzegorz Grzybek
-- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com
