[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-24 Thread Grzegorz Grzybek (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139985#comment-16139985
 ] 

Grzegorz Grzybek commented on KARAF-5314:
-

[~ch...@die-schneider.net], I think this may be related:
{noformat}
[ERROR] Failed to execute goal 
org.apache.karaf.tooling:karaf-maven-plugin:4.2.0-SNAPSHOT:assembly 
(process-resources) on project jboss-fuse-karaf: Unable to build assembly: 
invalid version "3.2.0-SNAPSHOT": non-numeric "0-SNAPSHOT": For input string: 
"0-SNAPSHOT" -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal 
org.apache.karaf.tooling:karaf-maven-plugin:4.2.0-SNAPSHOT:assembly 
(process-resources) on project jboss-fuse-karaf: Unable to build assembly
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
at 
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at 
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at 
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at 
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at 
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Unable to build 
assembly
at org.apache.karaf.tooling.AssemblyMojo.execute(AssemblyMojo.java:283)
at 
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 20 more
Caused by: java.lang.IllegalArgumentException: invalid version 
"3.2.0-SNAPSHOT": non-numeric "0-SNAPSHOT"
at org.osgi.framework.Version.parseInt(Version.java:170)
at org.osgi.framework.Version.(Version.java:134)
at 
org.apache.karaf.features.internal.service.FeatureReq.exactVersion(FeatureReq.java:69)
at 
org.apache.karaf.features.internal.service.FeatureReq.range(FeatureReq.java:64)
at 
org.apache.karaf.features.internal.service.FeatureReq.(FeatureReq.java:45)
at 
org.apache.karaf.profile.assembly.FeatureSelector.getMatching(FeatureSelector.java:77)
at 
org.apache.karaf.profile.assembly.FeatureSelector.addFeatures(FeatureSelector.java:59)
at 
org.apache.karaf.profile.assembly.FeatureSelector.addFeatures(FeatureSelector.java:66)
at 
org.apache.karaf.profile.assembly.FeatureSelector.addFeatures(FeatureSelector.java:66)
at 
org.apache.karaf.profile.assembly.FeatureSelector.getMatching(FeatureSelector.java:53)
at org.apache.karaf.profile.assembly.Builder.bootStage(Builder.java:878)
at 
org.apache.karaf.profile.assembly.Builder.doGenerateAssembly(Builder.java:673)
at 
org.apache.karaf.profile.assembly.Builder.generateAssembly(Builder.java:446)
at 
org.apache.karaf.tooling.AssemblyMojo.doExecute(AssemblyMojo.java:521)
at org.apache.karaf.tooling.AssemblyMojo.execute(AssemblyMojo.java:277)
... 22 more
Caused by: java.lang.NumberFormatException: For input string: "0-SNAPSHOT"
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at org.osgi.framework.Version.parseInt(Version.java:168)
... 36 more
{noformat}

I'm using SNAPSHOT features for CXF 3.2.0-SNAPSHOT...

> The performance of profile builder used by karaf maven plugin has reduced

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread Vinay Shankar (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135419#comment-16135419
 ] 

Vinay Shankar commented on KARAF-5314:
--

[~ch...@die-schneider.net] - According to the schedule for releases on the 
karaf website, the 4.1.3 was supposed to be released on 8/10. Is there a new 
tentative date you could tell me so that I can watch out for the release and 
upgrade the version in our project?

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
>Assignee: Christian Schneider
> Fix For: 4.2.0, 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread Christian Schneider (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135366#comment-16135366
 ] 

Christian Schneider commented on KARAF-5314:


Tthe backport to 4.1.x is done now.

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
>Assignee: Christian Schneider
> Fix For: 4.2.0, 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135363#comment-16135363
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit cc758635a4a20fc4f68b3ca1c8d557ae17705c3a in karaf's branch 
refs/heads/karaf-4.1.x from [~ch...@die-schneider.net]
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=cc75863 ]

[KARAF-5314] Extract addFeatures into class FeatureSelector


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.2.0
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135364#comment-16135364
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit b22e3c44cdd09edbf6f1bff4199826fc98a3f405 in karaf's branch 
refs/heads/karaf-4.1.x from [~ch...@die-schneider.net]
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=b22e3c4 ]

[KARAF-5314] Only recurse into dependencies once per feature


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.2.0
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135362#comment-16135362
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 4e8393552a2724ea0f416a055026f0ae9704ad50 in karaf's branch 
refs/heads/karaf-4.1.x from Vinay Shankar (e24113)
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=4e83935 ]

KARAF-5314 Added a features cache that increases the performance of the 
filtering of the required features from the available features.

The performance of the filtering logic decresed when the java streams API was 
used in 4.1 compared to for loops in 4.0. I reverted the filtering logic to use 
for loops and also introduced a features cache. This increased the performance 
by a huge margin, almost 20X faster when there are ~900 available features and 
~300 required features from a highly complex and huge feature dependency tree.


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.2.0
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread Vinay Shankar (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135253#comment-16135253
 ] 

Vinay Shankar commented on KARAF-5314:
--

[~ch...@die-schneider.net] - It would be great if you could backport the 
changes to 4.1.x. 

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.2.0
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread Vinay Shankar (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135240#comment-16135240
 ] 

Vinay Shankar commented on KARAF-5314:
--

[~ch...@die-schneider.net] - My original email to the mailing list. Was not 
sent on Friday for some reason. 

_"We have a highly complex and huge feature dependency tree due to the number 
of OSGI bundles (components).
When we were using 4.0.9, the plugin would take aroung 5min to build the 
distribution.
We upgraded to karaf 4.1.1 and the build time has increased significantly.
Now that we refactored our feature dependency tree a bit and added more feature 
files, the build time takes around 13min (7min with 4.0.9).
I decided to debug and I noticed a performance bottleneck in the Builder class 
of the profile module.
I have created a ticket for the same 
https://issues.apache.org/jira/browse/KARAF-5314.
Looking in the code, there has been a change in 4.1.1 to use Java 8 Streams API 
to filter the required features from the Set of all available features. A “for” 
loop was being used in 4.0.9.
I also noticed that there was no cache being used. Each time a required feature 
was found in the available set of features, it was not being cached (to avoid 
checking the set of available features again if the same feature was found 
later in the dependency tree).
I have implemented the caching and I tested the code against our feature 
dependency tree by pointing to the SNAPSHOT version of the plugin I built 
locally.
The build only takes 40 seconds to complete now. It is a huge improvement from 
13min.
Considering that we are a big team and each of us build locally and we also 
build on Bamboo servers for our CI and deployment on multiple environments, 
this will be a real time saver for us.
I have pushed the fix to my forked repository and created a pull request for 
the same https://github.com/apache/karaf/pull/337.
I have also attached the patch file here. Please take a look and review.
I was also wondering (if the review is successful and fix merged to master), 
when the next release would be?
We would like to have the fix released in 4.1.3."_

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.2.0
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135239#comment-16135239
 ] 

ASF GitHub Bot commented on KARAF-5314:
---

Github user vinayshankar closed the pull request at:

https://github.com/apache/karaf/pull/337


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.2.0
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread Christian Schneider (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135226#comment-16135226
 ] 

Christian Schneider commented on KARAF-5314:


Currently the fix will go into karaf 4.2.0. I hope we are doing this release 
soon.
If there is need I can backport for 4.1.x. which is normally released about 
once a month.

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.2.0
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread Vinay Shankar (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135210#comment-16135210
 ] 

Vinay Shankar commented on KARAF-5314:
--

[~ch...@die-schneider.net]

Looks good. The build now takes 48 seconds. Definitely much better than the 
original code. 

I also do not see a difference between the Streams and for loop in this 
scenario. However, I would suggest using Streams API carefully as in the 
original code it did make a difference and the reason behind me mentioning it 
in my ticket description. With streams, it would take 13min. With just a simple 
for loop, it would take 8min. Strange and I would love to debug it and find out 
why it behaved that way. Anyway, in the present execution path both the ways 
seem to be almost equal. For loop actually takes a couple of seconds more at 50 
seconds. 

I agree with the root cause you mention. Recursing for every feature dependency 
was the major problem. 

Regarding the ranges in my PR, I made the mistake of parsing the range into a 
version value. As you said and is evident from the new build, we probably do 
not cache with verion->feature mapping. Thanks for fixing that. 

Thanks again your help with resolving this issue. I look forward to using this 
performance fix in our build process. Which version and when can we expect this 
in a release?

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134910#comment-16134910
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 5a8133dbbfe7ede7366bcb943e2bd04937d8eff5 in karaf's branch 
refs/heads/model_features from [~ch...@die-schneider.net]
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=5a8133d ]

[KARAF-5314] Only recurse into dependencies once per feature


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134909#comment-16134909
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 0ed141dafd5ffb321bc851886e88457b616c1c06 in karaf's branch 
refs/heads/model_features from [~ch...@die-schneider.net]
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=0ed141d ]

[KARAF-5314] Extract addFeatures into class FeatureSelector


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134908#comment-16134908
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 3cbd810ea37e837950f1f8427f793c878423ba7a in karaf's branch 
refs/heads/model_features from Vinay Shankar (e24113)
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=3cbd810 ]

KARAF-5314 Added a features cache that increases the performance of the 
filtering of the required features from the available features.

The performance of the filtering logic decresed when the java streams API was 
used in 4.1 compared to for loops in 4.0. I reverted the filtering logic to use 
for loops and also introduced a features cache. This increased the performance 
by a huge margin, almost 20X faster when there are ~900 available features and 
~300 required features from a highly complex and huge feature dependency tree.


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread Christian Schneider (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134877#comment-16134877
 ] 

Christian Schneider commented on KARAF-5314:


I have now merged the commit of Vinay and my branch. The build looks good. 
[~vinayshankar] I would be very interested how it performs compared to the 
original code and to your PR.


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134866#comment-16134866
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 0ed141dafd5ffb321bc851886e88457b616c1c06 in karaf's branch 
refs/heads/master from [~ch...@die-schneider.net]
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=0ed141d ]

[KARAF-5314] Extract addFeatures into class FeatureSelector


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134867#comment-16134867
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 5a8133dbbfe7ede7366bcb943e2bd04937d8eff5 in karaf's branch 
refs/heads/master from [~ch...@die-schneider.net]
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=5a8133d ]

[KARAF-5314] Only recurse into dependencies once per feature


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134865#comment-16134865
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 3cbd810ea37e837950f1f8427f793c878423ba7a in karaf's branch 
refs/heads/master from Vinay Shankar (e24113)
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=3cbd810 ]

KARAF-5314 Added a features cache that increases the performance of the 
filtering of the required features from the available features.

The performance of the filtering logic decresed when the java streams API was 
used in 4.1 compared to for loops in 4.0. I reverted the filtering logic to use 
for loops and also introduced a features cache. This increased the performance 
by a huge margin, almost 20X faster when there are ~900 available features and 
~300 required features from a highly complex and huge feature dependency tree.


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-20 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134437#comment-16134437
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 5a8133dbbfe7ede7366bcb943e2bd04937d8eff5 in karaf's branch 
refs/heads/KARAF-5314-2 from [~ch...@die-schneider.net]
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=5a8133d ]

[KARAF-5314] Only recurse into dependencies once per feature


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-20 Thread Christian Schneider (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134430#comment-16134430
 ] 

Christian Schneider commented on KARAF-5314:


I just found a possible change that should speed up the code quite a lot. 
Currently we always recurse into every feature dependency. I think we we can 
make this a lot faster by only recursing into the dependencies of a feature 
once. 

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-20 Thread Christian Schneider (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134429#comment-16134429
 ] 

Christian Schneider commented on KARAF-5314:


In this part of the code we are only going through the features and their 
feature dependencies. So we are traversing at max 900 features and recurse 
through their dependencies. With the caching on feature name we only need to 
traverse the different versions of a feature. So I think this should be way 
below the level where parallel execution would help. 

After my extraction of the FeatureSelector class it should be quite easy to do 
unit tests on the performance. So I will setup some test cases. Then we can 
easily try the performance impact of changes.


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-20 Thread Achim Nierbeck (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134406#comment-16134406
 ] 

Achim Nierbeck commented on KARAF-5314:
---

Do we have a "high load test" available? In that case I'd try if parallel 
streams might help to increase performance when doing the streaming.  

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-20 Thread Christian Schneider (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134368#comment-16134368
 ] 

Christian Schneider commented on KARAF-5314:


I now extracted the caching logic into its own class. Please see the branch 
KARAF-5314-2.
Compared to your code I removed the caching of the version -> feature for now. 
We also need to support ranges like spring/[3.2,4) and It think this is not 
covered by the caching you implemented. 

I also reintroduced the stream for selecting the matching versions.  I am not 
sure if this is slower than a for loop. Can you test this with a loop and with 
a stream. If the for loop is performing noticeably better then we will use a 
for loop.

I think the main performance gain you saw came from the caching. In my branch 
this will be a little slower as I only cache by name. The question is how big 
the difference really is. We can create a caching using name, range if needed 
but I am not sure if it makes a big difference. I expect this to only matter 
when there is a high number of versions for a feature name.
So if you think this should make a difference in your case we can do this.

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-20 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134366#comment-16134366
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 0ed141dafd5ffb321bc851886e88457b616c1c06 in karaf's branch 
refs/heads/KARAF-5314-2 from [~ch...@die-schneider.net]
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=0ed141d ]

[KARAF-5314] Extract addFeatures into class FeatureSelector


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-20 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134365#comment-16134365
 ] 

ASF subversion and git services commented on KARAF-5314:


Commit 3cbd810ea37e837950f1f8427f793c878423ba7a in karaf's branch 
refs/heads/KARAF-5314-2 from Vinay Shankar (e24113)
[ https://git-wip-us.apache.org/repos/asf?p=karaf.git;h=3cbd810 ]

KARAF-5314 Added a features cache that increases the performance of the 
filtering of the required features from the available features.

The performance of the filtering logic decresed when the java streams API was 
used in 4.1 compared to for loops in 4.0. I reverted the filtering logic to use 
for loops and also introduced a features cache. This increased the performance 
by a huge margin, almost 20X faster when there are ~900 available features and 
~300 required features from a highly complex and huge feature dependency tree.


> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-19 Thread Christian Schneider (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134003#comment-16134003
 ] 

Christian Schneider commented on KARAF-5314:


I am not sure if the streams API is the root of the problem. Instead I think 
the problem is that we repeatedly go through the features to find a match for a 
given feature/version.
So a cache is a good idea to speed this up.

Currently there are two cases for a version. The version can be open if only 
the name is given or the version is 0.0.0. In this case all versions would 
match. 
The other case is that a version is given. Then only this version would match.

The current code filters using a range which would return add all matching 
features.
The code in the PR uses a cache that holds only one feature. So I am not sure 
this will always do the same thing.
 

> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KARAF-5314) The performance of profile builder used by karaf maven plugin has reduced significantly in 4.1 compared to 4.0

2017-08-18 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KARAF-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133490#comment-16133490
 ] 

ASF GitHub Bot commented on KARAF-5314:
---

GitHub user vinayshankar opened a pull request:

https://github.com/apache/karaf/pull/337

KARAF-5314 Added a features cache that increases the performance of the 
filtering of the required features from the available features.

The performance of the filtering logic decresed when the java streams API 
was used in 4.1 compared to for loops in 4.0. I reverted the filtering logic to 
use for loops and also introduced a features cache. This increased the 
performance by a huge margin, almost 20X faster when there are ~900 available 
features and ~300 required features in a highly complex and huge feature 
dependency tree.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vinayshankar/karaf KARAF-5314

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/karaf/pull/337.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #337


commit 06366f5db7db26b7b4bd56bcec8d193ff4a975dd
Author: Vinay Shankar (e24113) 
Date:   2017-08-18T18:55:58Z

KARAF-5314 Added a features cache that increases the performance of the 
filtering of the required features from the available features.

The performance of the filtering logic decresed when the java streams API 
was used in 4.1 compared to for loops in 4.0. I reverted the filtering logic to 
use for loops and also introduced a features cache. This increased the 
performance by a huge margin, almost 20X faster when there are ~900 available 
features and ~300 required features from a highly complex and huge feature 
dependency tree.




> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0
> --
>
> Key: KARAF-5314
> URL: https://issues.apache.org/jira/browse/KARAF-5314
> Project: Karaf
>  Issue Type: Bug
>Reporter: Vinay Shankar
> Fix For: 4.1.3
>
>
> The performance of profile builder used by karaf maven plugin has reduced 
> significantly in 4.1 compared to 4.0. 
> The java streams API is being used in 4.1 to filter our the required features 
> from the set of all features present in the repositories that are part of the 
> profile. This is done in the "addFeatures" method in the "Builder.java" class 
> in the "org.apache.karaf.profile.core" bundle. This change (from 4.0) has 
> drastically reduced the performance. For a profile with ~900 features in all 
> the repositories in the profile and ~300 required features from a highly 
> complex feature dependency tree, this function is taking around 13min to 
> complete. The same execution took around 3-5min in 4.0 (where simple for 
> loops were being used). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)