[jira] [Updated] (MDEPLOY-297) [REGRESSION] deploy-file no longer honor 'packaging' parameter

[Dan Tran (Jira)]

 [ 
https://issues.apache.org/jira/browse/MDEPLOY-297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated MDEPLOY-297:
-
Summary: [REGRESSION] deploy-file no longer honor 'packaging' parameter  
(was: [REGRESSION] deploy-file not longer honor 'packaging' parameter)

> [REGRESSION] deploy-file no longer honor 'packaging' parameter
> --
>
> Key: MDEPLOY-297
> URL: https://issues.apache.org/jira/browse/MDEPLOY-297
> Project: Maven Deploy Plugin
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Dan Tran
>Priority: Major
>
> prior to version 3.0.0,  we can use deploy:deploy-file to upload local file 
> x/y/z.jar but change the file type to something else with   packaging=xyx



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (MDEPLOY-297) [REGRESSION] deploy-file not longer honor 'packaging' parameter

[Dan Tran (Jira)]

 [ 
https://issues.apache.org/jira/browse/MDEPLOY-297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated MDEPLOY-297:
-
Summary: [REGRESSION] deploy-file not longer honor 'packaging' parameter  
(was: [REGRESSION] deploy-file not long honor 'packaging' parameter)

> [REGRESSION] deploy-file not longer honor 'packaging' parameter
> ---
>
> Key: MDEPLOY-297
> URL: https://issues.apache.org/jira/browse/MDEPLOY-297
> Project: Maven Deploy Plugin
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Dan Tran
>Priority: Major
>
> prior to version 3.0.0,  we can use deploy:deploy-file to upload local file 
> x/y/z.jar but change the file type to something else with   packaging=xyx



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (MDEPLOY-297) [REGRESSION] deploy-file not long honor 'packaging' parameter

[Dan Tran (Jira)]

Dan Tran created MDEPLOY-297:


 Summary: [REGRESSION] deploy-file not long honor 'packaging' 
parameter
 Key: MDEPLOY-297
 URL: https://issues.apache.org/jira/browse/MDEPLOY-297
 Project: Maven Deploy Plugin
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Dan Tran


prior to version 3.0.0,  we can use deploy:deploy-file to upload local file 
x/y/z.jar but change the file type to something else with   packaging=xyx





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (MWRAPPER-68) MVNW_REPOURL improperly formed distributionUrl

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MWRAPPER-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553503#comment-17553503
 ] 

Dan Tran commented on MWRAPPER-68:
--

I am facing same issue after upgrading to 3.1.1

> MVNW_REPOURL improperly formed distributionUrl
> --
>
> Key: MWRAPPER-68
> URL: https://issues.apache.org/jira/browse/MWRAPPER-68
> Project: Maven Wrapper
>  Issue Type: Bug
>  Components: Maven Wrapper Jar
>Affects Versions: 3.1.1
>Reporter: HumanFund
>Priority: Major
>
> In Maven Wrapper v3.1.1, Installer::createDist(), file 
> maven-wrapper/src/main/java/org/apache/maven/wrapper/Installer.java, was 
> updated on line 74 to be:
> distributionUrl = new URI( mvnwRepoUrl ).resolve( "/" ).resolve( mvnPath );
> The above update is causing the distributionUrl to be improperly formed based 
> on the MVNW_REPOURL environment variable and the mvnPath which is extracted 
> from the distributionUrl in maven-wrapper.properties, specifically the 
> substring starting with "org/apache/maven".
> The update was introduced in the following commit:
> [https://github.com/apache/maven-wrapper/commit/22a3268def96e5e648aa97a49d9e146e529b7c87#diff-193f3775e6efb0b6ed01219b21272f9eb3861965ce3af3586a0ce8eb153359c0]
> An example of the results are shown below.  Note the "Downloading" URI does 
> not include the entire repo url, only the scheme, host, and port, then the 
> maven path is appended.
> The repo url is getting truncated by the call to resolve( "/" ) on line 74.  
> I do not currently see a purpose for having this call in place.  I made the 
> following update to line 74 and it works fine:
> distributionUrl = new URI( mvnwRepoUrl ).resolve( mvnPath );
> Note that in Maven Wrapper v3.1.0, the distributionUrl was formed simply by 
> appending the maven path to the MVNW_REPOURL:
> distributionUrl = new URI( mvnwRepoUrl + "/" + mvnPath );
> Example output demonstrating issue:
> [exec] [INFO] Apache Maven Wrapper 3.1.1
> [exec] [INFO] Detected MVNW_REPOURL environment variable 
> [http://localhost:8081/repository/repo-maven-apache-org-maven2/]
> [exec] [INFO] Installing Maven distribution 
> /home/myexamplehome/maven/wrapper/dists/apache-maven-3.6.3-bin/cf3cf814
> [exec] [INFO] Downloading 
> [http://localhost:8081/org/apache/maven/apache-maven/3.6.3/apache-maven-3.6.3-bin.zip]
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Comment Edited] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523970#comment-17523970
 ] 

Dan Tran edited comment on MNG-7433 at 4/19/22 1:16 AM:


Actually, I firmly believe it has something to do with one of my reactor 
modules spinning another maven instance/JVM, after that, all other aggregating 
mojos running at other threads are blocked

and again, so far I have no luck reproducing this scenario in a simple way 

and yes, it would be helpful  to print a warning showing why an mojo-execution 
is blocked




was (Author: dantran):
Actually, I firmly believe it has something to do with one of my reactor 
modules spinning another maven instance/JVM, after that, all other aggregating 
mojos running at other threads are blocked

and again, so far I have no luck reproducing this scenario in a simple way 

and yes, it would be helpful  to print a warning showing why it is blocked



> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523970#comment-17523970
 ] 

Dan Tran edited comment on MNG-7433 at 4/19/22 1:15 AM:


Actually, I firmly believe it has something to do with one of my reactor 
modules spinning another maven instance/JVM, after that, all other aggregating 
mojos running at other threads are blocked

and again, so far I have no luck reproducing this scenario in a simple way 

and yes, it would be helpful  to print a warning showing why it is blocked




was (Author: dantran):
Actually, I firmly believe it has something to do with one of my reactor 
modules spinning another maven instance/JVM, after that, all other aggregating 
mojos running at other threads are blocked

and again, so far I have no luck reproducing this scenario in a simple way 

and yes, it would be helpful  to print a warning



> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523970#comment-17523970
 ] 

Dan Tran commented on MNG-7433:
---

Actually, I firmly believe it has something to do with one of my reactor 
modules spinning another maven instance/JVM, after that, all other aggregating 
mojos running at other threads are blocked

and again, so far I have no luck reproducing this scenario in a simple way 

and yes, it would be helpful  to print a warning



> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523616#comment-17523616
 ] 

Dan Tran commented on MNG-7433:
---

basically,  all running mojo executions in a reactor multi-thread build must be 
non-aggregator mojos

> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523538#comment-17523538
 ] 

Dan Tran commented on MNG-7433:
---

I will proceed to update all our internal mojos to non-aggregate, and aggregate 
when it really does so.  Also need to update buildnumber:create-metadata to the 
same.

We can close this ticket i think 

> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521340#comment-17521340
 ] 

Dan Tran commented on MNG-7433:
---

buildnumber:create-metadate is used since it has a feature we need to dump more 
properties into its classifier

On another note:  at work, we purposely set most of our internal mojos to use   
_aggregator=false_ for the following reasons

   * We can run it at command line plugin:goal without it triggering reactor 
build ( execute once) - main use case
   * we can configure at any parent pom without explicitly setting inherit=false

I guess my understanding of aggregating mojo is completely wrong?  



> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520123#comment-17520123
 ] 

Dan Tran edited comment on MNG-7433 at 4/10/22 8:58 AM:


per my finding,  i updated my mdev:sonar  ( the one that spins another mvn, 
invoker style) to nonagregating mojo, and can see the build runs more steps 
further at the other module running in another thread until it steps on one of 
its aggregating mojo (such as buildnumber-m-p) and stuck.  And just follow the 
same pattern to push more steps.  This means I need to ensure no aggregating 
mojos ( my own internal company mojo, and the ones from outside company) 


was (Author: dantran):
per my finding,  i updated my mdev:sonar  ( the one that spins another mvn, 
invoker style) to nonagregating mojo, and can see the build runs more steps 
further at the other module running in another thread until it steps one of its 
 aggregating mojo (such as buildnumber-m-p) and stuck.  And just follow the 
same pattern to push more steps.  This means I need to ensure no aggregating 
mojos ( my own internal company mojo, and the ones from outside company) 

> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520123#comment-17520123
 ] 

Dan Tran commented on MNG-7433:
---

per my finding,  i updated my mdev:sonar  ( the one that spins another mvn, 
invoker style) to nonagregating mojo, and can see the build runs more steps 
further at the other module running in another thread until it steps one of its 
 aggregating mojo (such as buildnumber-m-p) and stuck.  And just follow the 
same pattern to push more steps.  This means I need to ensure no aggregating 
mojos ( my own internal company mojo, and the ones from outside company) 

> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520061#comment-17520061
 ] 

Dan Tran commented on MNG-7433:
---

[~cstamas]  from your explanation, can I assume that all  'aggregating' mojo 
executions are serialized in a multi-threaded reactor build? and it has nothing 
to do with another Maven instant.

so far I am not able to reproduce it by using 'aggregating' sleep mojo in a 
multi modules settings




> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ https://issues.apache.org/jira/browse/MNG-7433 ]


Dan Tran deleted comment on MNG-7433:
---

was (Author: dantran):
further testing and debugging show that  all mojo, with  'aggregator = true' 
set at @Mojo,  executions are serialized in multi-thread reactor build

and it has nothing to do with  _'another mvn instant'_.  




> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520058#comment-17520058
 ] 

Dan Tran commented on MNG-7433:
---

further testing and debugging show that  all mojo, with  'aggregator = true' 
set at @Mojo,  executions are serialized in multi-thread reactor build

and it has nothing to do with  _'another mvn instant'_.  




> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519781#comment-17519781
 ] 

Dan Tran edited comment on MNG-7433 at 4/8/22 9:39 PM:
---

Thanks for the suggestion regarding 'aggregating mojo',  I found the consistent 
pattern where

* after sonar-analysis module execution invoked mdev:sonar ( aggregating 
mojo which spins a maven instance to run sonar:sonar), other thread is blocked 
when it runs any aggregating mojos. 

we have a number of internal mojos with 'aggregator = true' set at @Mojo 
annotation used during the build process. 
We set it this way so that it is not running in reactor mode in default

My guess here is I need to update all internal mojos with aggregator=false.  
btw buildnumber-m-p from MojoHaus is also in the same category

is it something the Maven team can fix in 3.9?





was (Author: dantran):
Thanks for the suggestion regarding 'aggregating mojo',  I found the consistent 
pattern where

* after sonar-analysis module execution invoked mdev:sonar ( aggregating 
mojo which spins a maven instance to run sonar:sonar), other thread is blocked 
when it runs any aggregating mojos. Attempt to make mdev:sonar as non 
'aggregating mojo' don't seem to help

we have a number of internal mojos with 'aggregator = true' set at @Mojo 
annotation used during the build process. 
We set it this way so that it is not running in reactor mode in default

My guess here is I need to update all internal mojos with aggregator=false.  
btw buildnumber-m-p also in same category

is it something Maven team can fix in 3.9?




> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519781#comment-17519781
 ] 

Dan Tran edited comment on MNG-7433 at 4/8/22 8:21 PM:
---

Thanks for the suggestion regarding 'aggregating mojo',  I found the consistent 
pattern where

* after sonar-analysis module execution invoked mdev:sonar ( aggregating 
mojo which spins a maven instance to run sonar:sonar), other thread is blocked 
when it runs any aggregating mojos. Attempt to make mdev:sonar as non 
'aggregating mojo' don't seem to help

we have a number of internal mojos with 'aggregator = true' set at @Mojo 
annotation used during the build process. 
We set it this way so that it is not running in reactor mode in default

My guess here is I need to update all internal mojos with aggregator=false.  
btw buildnumber-m-p also in same category

is it something Maven team can fix in 3.9?





was (Author: dantran):
Thanks for the suggestion regarding 'aggregating mojo',  I found the consistent 
pattern where

* after sonar-analysis module execution invoked  mdev:sonar ( aggregating 
mojo which spins a maven instance to run sonar:sonar), other thread is blocked 
when it runs any aggregating mojos

we have a number of internal mojos with 'aggregator = true' set at @Mojo 
annotation used during the build process.
We set it this way so that it is not running in reactor mode in default

My guess here is I need to update all internal mojos with aggregator=false.  
btw buildnumber-m-p also in same category

is it something Maven team can fix in 3.9?




> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519781#comment-17519781
 ] 

Dan Tran commented on MNG-7433:
---

Thanks for the suggestion regarding 'aggregating mojo',  I found the consistent 
pattern where

* after sonar-analysis module execution invoked  mdev:sonar ( aggregating 
mojo which spins a maven instance to run sonar:sonar), other thread is blocked 
when it runs any aggregating mojos

we have a number of internal mojos with 'aggregator = true' set at @Mojo 
annotation used during the build process.
We set it this way so that it is not running in reactor mode in default

My guess here is I need to update all internal mojos with aggregator=false.  
btw buildnumber-m-p also in same category

is it something Maven team can fix in 3.9?




> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519165#comment-17519165
 ] 

Dan Tran edited comment on MNG-7433 at 4/8/22 3:18 AM:
---

here is a brief just to visualize the issue and its steps

{code:java}
10:59:16  [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
xxx-mvnrepo ---
10:59:16  [INFO] 
10:59:16  [INFO] --< com.xxx.xxx.xxx:sonar-analysis 
>--
10:59:16  [INFO] Building sonar-analysis 19.11.0-1-SNAPSHOT 
[538/540]
10:59:16  [INFO] [ pom 
]-
10:59:16  [INFO] 
10:59:16  [INFO] ---< com.xxx.xxx.xxx.installer:xxx-puppet-config 
>
10:59:16  [INFO] Building xxx-puppet-config 19.11.0-1-SNAPSHOT  
[539/540]
10:59:16  [INFO] [ rpm 
]-
10:59:16  [INFO] 
10:59:16  [INFO] --- maven-enforcer-plugin:1.4.1:enforce (default) @ 
sonar-analysis ---
10:59:16  [INFO] 
10:59:16  [INFO] --- jacoco-maven-plugin:0.8.7:prepare-agent (jacoco-all) @ 
sonar-analysis ---
10:59:16  [INFO] argLine set to 
-javaagent:/space/jenkins/workspace/xxx_xxx_xxx_PR-4236/.repository/org/jacoco/org.jacoco.agent/0.8.7/org.jacoco.agent-0.8.7-runtime.jar=destfile=/space/jenkins/workspace/xxx_xxx_xxx_PR-4236/sonar/target/jacoco.exec
10:59:16  [INFO] 
10:59:16  [INFO] --- flatten-maven-plugin:1.2.7:flatten (flatten) @ 
sonar-analysis ---
10:59:16  [INFO] Generating flattened POM of project 
com.xxx.xxx.xxx:sonar-analysis:pom:19.11.0-1-SNAPSHOT...
10:59:16  [INFO] 
10:59:16  [INFO] --- jacoco-maven-plugin:0.8.7:report (jacoco-all) @ 
sonar-analysis ---
10:59:16  [INFO] Skipping JaCoCo execution due to missing execution data file.
10:59:20  [INFO] 
10:59:20  [INFO] --- maven-enforcer-plugin:1.4.1:enforce (default) @ 
xxx-puppet-config ---
10:59:20  [INFO] 
10:59:20  [INFO] --- mdev-maven-plugin:1.6.3:sonar 
(quiet-sonar-scan-by-default) @ sonar-analysis ---
10:59:20  [INFO] Executing: /bin/sh -c cd 
'/space/jenkins/workspace/xxx_xxx_xxx_PR-4236' && 
'/xxx/build/.m2/wrapper/dists/apache-maven-3.8.5-bin/5sf9ufumnhp6odg6f4svjl54q1/apache-maven-3.8.5/bin/mvn'
 '-ntp' 'sonar:sonar' '-Dsonar.pullrequest.key=4236' 
'-Dsonar.pullrequest.branch=personal/trand8/maven-3.8.5-2' 
'-Dsonar.pullrequest.base=master' '--quiet' '-P!sonar' '-s' 
'/tmp/settings43566726mvn'. Working directory: 
/space/jenkins/workspace/xxx_xxx_xxx_PR-4236
{code}

it shows - after xxx-mvnrepos is done,  2 modules are then run in parallel: 
sonar-analysis and xxx-puppet-config.  The xxx-puppet-config stalls after 
sonar-analisys spins another maven instant to run sonar:sonar

Note I can replace sonar:sonar with other goal such as 'compile' and see same 
blocking issue





was (Author: dantran):
here is a brief just to visualize the issue and its steps

{code:java}
10:59:16  [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
xxx-mvnrepo ---
10:59:16  [INFO] 
10:59:16  [INFO] --< com.xxx.xxx.xxx:sonar-analysis 
>--
10:59:16  [INFO] Building sonar-analysis 19.11.0-1-SNAPSHOT 
[538/540]
10:59:16  [INFO] [ pom 
]-
10:59:16  [INFO] 
10:59:16  [INFO] ---< com.xxx.xxx.xxx.installer:xxx-puppet-config 
>
10:59:16  [INFO] Building xxx-puppet-config 19.11.0-1-SNAPSHOT  
[539/540]
10:59:16  [INFO] [ rpm 
]-
10:59:16  [INFO] 
10:59:16  [INFO] --- maven-enforcer-plugin:1.4.1:enforce (default) @ 
sonar-analysis ---
10:59:16  [INFO] 
10:59:16  [INFO] --- jacoco-maven-plugin:0.8.7:prepare-agent (jacoco-all) @ 
sonar-analysis ---
10:59:16  [INFO] argLine set to 
-javaagent:/space/jenkins/workspace/xxx_xxx_xxx_PR-4236/.repository/org/jacoco/org.jacoco.agent/0.8.7/org.jacoco.agent-0.8.7-runtime.jar=destfile=/space/jenkins/workspace/xxx_xxx_xxx_PR-4236/sonar/target/jacoco.exec
10:59:16  [INFO] 
10:59:16  [INFO] --- flatten-maven-plugin:1.2.7:flatten (flatten) @ 
sonar-analysis ---
10:59:16  [INFO] Generating flattened POM of project 
com.xxx.xxx.xxx:sonar-analysis:pom:19.11.0-1-SNAPSHOT...
10:59:16  [INFO] 
10:59:16  [INFO] --- jacoco-maven-plugin:0.8.7:report (jacoco-all) @ 
sonar-analysis ---
10:59:16  [INFO] Skipping JaCoCo execution due to missing execution data file.
10:59:20  [INFO] 
10:59:20  [INFO] --- maven-enforcer-plugin:1.4.1:enforce (default) @ 
xxx-puppet-config ---
10:59:20  [INFO] 
10:59:20  [INFO] --- mdev-maven-plugin:1.6.3:sonar 
(quiet-sonar-scan-by-default) @ sonar-analysis ---
10:59:20  [INFO] Executing: /bin/sh -c cd 
'/space/jenkins/workspace/xxx_xxx_xxx_PR-4236' && 
'/xxx/build/.m2/wrapper/dists/apache-maven-3.8.5-bin/5sf9ufumnhp6odg6f4svjl54q1/apache-maven-3.8.5/bin/mvn'
 '-ntp' 

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519165#comment-17519165
 ] 

Dan Tran commented on MNG-7433:
---

here is a brief just to visualize the issue and its steps

{code:java}
10:59:16  [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
xxx-mvnrepo ---
10:59:16  [INFO] 
10:59:16  [INFO] --< com.xxx.xxx.xxx:sonar-analysis 
>--
10:59:16  [INFO] Building sonar-analysis 19.11.0-1-SNAPSHOT 
[538/540]
10:59:16  [INFO] [ pom 
]-
10:59:16  [INFO] 
10:59:16  [INFO] ---< com.xxx.xxx.xxx.installer:xxx-puppet-config 
>
10:59:16  [INFO] Building xxx-puppet-config 19.11.0-1-SNAPSHOT  
[539/540]
10:59:16  [INFO] [ rpm 
]-
10:59:16  [INFO] 
10:59:16  [INFO] --- maven-enforcer-plugin:1.4.1:enforce (default) @ 
sonar-analysis ---
10:59:16  [INFO] 
10:59:16  [INFO] --- jacoco-maven-plugin:0.8.7:prepare-agent (jacoco-all) @ 
sonar-analysis ---
10:59:16  [INFO] argLine set to 
-javaagent:/space/jenkins/workspace/xxx_xxx_xxx_PR-4236/.repository/org/jacoco/org.jacoco.agent/0.8.7/org.jacoco.agent-0.8.7-runtime.jar=destfile=/space/jenkins/workspace/xxx_xxx_xxx_PR-4236/sonar/target/jacoco.exec
10:59:16  [INFO] 
10:59:16  [INFO] --- flatten-maven-plugin:1.2.7:flatten (flatten) @ 
sonar-analysis ---
10:59:16  [INFO] Generating flattened POM of project 
com.xxx.xxx.xxx:sonar-analysis:pom:19.11.0-1-SNAPSHOT...
10:59:16  [INFO] 
10:59:16  [INFO] --- jacoco-maven-plugin:0.8.7:report (jacoco-all) @ 
sonar-analysis ---
10:59:16  [INFO] Skipping JaCoCo execution due to missing execution data file.
10:59:20  [INFO] 
10:59:20  [INFO] --- maven-enforcer-plugin:1.4.1:enforce (default) @ 
xxx-puppet-config ---
10:59:20  [INFO] 
10:59:20  [INFO] --- mdev-maven-plugin:1.6.3:sonar 
(quiet-sonar-scan-by-default) @ sonar-analysis ---
10:59:20  [INFO] Executing: /bin/sh -c cd 
'/space/jenkins/workspace/xxx_xxx_xxx_PR-4236' && 
'/xxx/build/.m2/wrapper/dists/apache-maven-3.8.5-bin/5sf9ufumnhp6odg6f4svjl54q1/apache-maven-3.8.5/bin/mvn'
 '-ntp' 'sonar:sonar' '-Dsonar.pullrequest.key=4236' 
'-Dsonar.pullrequest.branch=personal/trand8/maven-3.8.5-2' 
'-Dsonar.pullrequest.base=master' '--quiet' '-P!sonar' '-s' 
'/tmp/settings43566726mvn'. Working directory: 
/space/jenkins/workspace/xxx_xxx_xxx_PR-4236
{code}

it shows - after xxx-mvnrepos is done,  2 module then run in parallel: 
sonar-analysis and xxx-puppet-config.  The xxx-puppet-config stalls after 
sonar-analisys spins another maven instant to run sonar:sonar




> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519031#comment-17519031
 ] 

Dan Tran commented on MNG-7433:
---

my .mvn/extensions.xml does have

{code:java}
 
io.takari.maven
takari-smart-builder
0.6.1
 
{code}

but it is not activated during maven build,  and getting rid of it also does 
not fix the blocking issue

So far i don't have any luck creating a simple producer.   still digging


> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) [REGRESSION] Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518656#comment-17518656
 ] 

Dan Tran commented on MNG-7433:
---

[~cstamas] thanks for the advice.  In my use case  i have 2 MVN each of course 
has its JVM.  SuSE Linux +java11  OpenJDK from Amazon.  This is definitely a 
very weird lockup issue

> [REGRESSION] Multiple maven instances working on same source tree can lock 
> each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508424#comment-17508424
 ] 

Dan Tran edited comment on MNG-7433 at 3/20/22, 7:49 PM:
-

I also tried:

* switch apache maven-invoker to run sonar:sonar ( instead of internal 
custom invoker)
* replicate a copy of local repo so that maven-invoker has its own maven 
local repo
* invoke a different goal such as compile, or just a simple sleep  instead 
of sonar:sonar
* latest  3.9.0 from source


A workaround is to disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock


was (Author: dantran):
I also tried:

* switch apache maven-invoker to run sonar:sonar ( instead of internal 
custom invoker)
* replicate a copy of local repo so that maven-invoker has its own maven 
local repo
* invoke a different goal ( compile  instead of sonar:sonar)
* latest  3.9.0 from source

Possible that the lock is on stdout/stderr stream?

A workaround is to disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock

> Multiple maven instances working on same source tree can lock each other
> 
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508563#comment-17508563
 ] 

Dan Tran edited comment on MNG-7433 at 3/20/22, 7:49 AM:
-

The below comment block in interesting


{code:java}
/**
 * Aggregating mojo executions (possibly) modify all MavenProjects, 
including those that are currently in use
 * by concurrently running mojo executions. To prevent race conditions, an 
aggregating execution will block
 * all other executions until finished.
 * We also lock on a given project to forbid a forked lifecycle to be 
executed concurrently with the project.
 * TODO: ideally, the builder should take care of the ordering in a smarter 
way
 * TODO: and concurrency issues fixed with MNG-7157
 */
private static class ProjectLock implements AutoCloseable
{code}
 
Wonder  if running invoke-plugin consider as a 'aggregated project' and its 
locks other modules at reactor from running


was (Author: dantran):
The below comment block in interesting


{code:java}
/**
 * Aggregating mojo executions (possibly) modify all MavenProjects, 
including those that are currently in use
 * by concurrently running mojo executions. To prevent race conditions, an 
aggregating execution will block
 * all other executions until finished.
 * We also lock on a given project to forbid a forked lifecycle to be 
executed concurrently with the project.
 * TODO: ideally, the builder should take care of the ordering in a smarter 
way
 * TODO: and concurrency issues fixed with MNG-7157
 */
private static class ProjectLock implements AutoCloseable
{code}


> Multiple maven instances working on same source tree can lock each other
> 
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509393#comment-17509393
 ] 

Dan Tran commented on MNG-7433:
---

[~gnodet] please advice

> Multiple maven instances working on same source tree can lock each other
> 
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (MNG-7433) Multiple maven instances working on same source tree can lock each other

[Dan Tran (Jira)]

 [ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated MNG-7433:
--
Summary: Multiple maven instances working on same source tree can lock each 
other  (was: multiple maven instances working same source tree can lock each 
other)

> Multiple maven instances working on same source tree can lock each other
> 
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508424#comment-17508424
 ] 

Dan Tran edited comment on MNG-7433 at 3/20/22, 3:35 AM:
-

I also tried:

* switch apache maven-invoker to run sonar:sonar ( instead of internal 
custom invoker)
* replicate a copy of local repo so that maven-invoker has its own maven 
local repo
* invoke a different goal ( compile  instead of sonar:sonar)
* latest  3.9.0 from source

Possible that the lock is on stdout/stderr stream?

A workaround is to disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock


was (Author: dantran):
I also tried:

* switch apache maven-invoker to run sonar ( instead of internal custom 
invoker)
* replicate a copy of local repo so that maven-invoker has its own maven 
local repo
* latest  3.9.0 from source

A workaround is to disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock

> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508424#comment-17508424
 ] 

Dan Tran edited comment on MNG-7433 at 3/20/22, 2:27 AM:
-

I also tried:

* switch apache maven-invoker to run sonar ( instead of internal custom 
invoker)
* replicate a copy of local repo so that maven-invoker has its own maven 
local repo
* latest  3.9.0 from source

A workaround is to disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock


was (Author: dantran):
I also tried:

* switch apache maven-invoker to invoke sonar ( instead of internal custom 
invoker)
* replicate a copy of local repo so that maven-invoker has its own maven 
local repo
* latest  3.9.0 from source

A workaround is to disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock

> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508424#comment-17508424
 ] 

Dan Tran edited comment on MNG-7433 at 3/20/22, 2:26 AM:
-

I also tried:

* switch apache maven-invoker to invoke sonar ( instead of internal custom 
invoker)
* replicate a copy of local repo so that maven-invoker has its own maven 
local repo
* latest  3.9.0 from source

A workaround is to disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock


was (Author: dantran):
I also tried:

* replicate local repo to run with  sonar:sonar maven instance
* latest  3.9.0 from source

The same locking found

disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock

> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508563#comment-17508563
 ] 

Dan Tran commented on MNG-7433:
---

The below comment block in interesting


{code:java}
/**
 * Aggregating mojo executions (possibly) modify all MavenProjects, 
including those that are currently in use
 * by concurrently running mojo executions. To prevent race conditions, an 
aggregating execution will block
 * all other executions until finished.
 * We also lock on a given project to forbid a forked lifecycle to be 
executed concurrently with the project.
 * TODO: ideally, the builder should take care of the ordering in a smarter 
way
 * TODO: and concurrency issues fixed with MNG-7157
 */
private static class ProjectLock implements AutoCloseable
{code}


> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508424#comment-17508424
 ] 

Dan Tran edited comment on MNG-7433 at 3/18/22, 3:21 AM:
-

I also tried:

* replicate local repo to run with  sonar:sonar maven instance
* latest  3.9.0 from source

The same locking found

disable this line 
https://github.com/apache/maven/blob/master/maven-core/src/main/java/org/apache/maven/lifecycle/internal/MojoExecutor.java#L226
  will remove the lock


was (Author: dantran):
I also tried:

* replicate local repo to run with  sonar:sonar maven instance
* latest  3.9.0 from source

The same locking found

> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508424#comment-17508424
 ] 

Dan Tran edited comment on MNG-7433 at 3/17/22, 8:50 PM:
-

I also tried:

* replicate local repo to run with  sonar:sonar maven instance
* latest  3.9.0 from source

The same locking found


was (Author: dantran):

I also tried:

* replicate local repo to dedicate for  sonar:sonar maven instance
* latest  3.9.0 from source

Same locking found

> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508424#comment-17508424
 ] 

Dan Tran commented on MNG-7433:
---


I also tried:

* replicate local repo to dedicate for  sonar:sonar maven instance
* latest  3.9.0 from source

Same locking found

> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

 [ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated MNG-7433:
--
Description: 
I have a large multi modules java maven build where:

  * phase 1 - basic build + unit tests + jacoco  - 40 min
  * phase 2 - sonar:sonar 20 min
  * phase 3 - final packaging and basic smoke-test - 20 min

To take advantage of Maven multi-threaded build, during the reactor build, one 
of our maven module spins another instance of Maven to run sonar:sonar goal 
right after the basic build is done. 

This means  our phase 2 and phase 3 run in parallel sharing the same source 
tree, same local maven repo (where sonar:sonar should have all needed 
dependencies at the share local maven repo to run its task)

With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
until phase 2 is done. 

I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
locking started the happen

How does the lock mechanic work?  there must be a local file where both Maven 
instances are watching each other.  Is there an option to disable this lock?


  was:
I have a large multi modules java maven build where:

  * phase 1 - basic build + unit tests  - 40 min
  * phase 2 - sonar:sonar 20 min
  * phase 3 - final packaging and basic smoke-test - 20 min

To take advantage of Maven multi-threaded build, during the reactor build, one 
of our maven module spins another instance of Maven to run sonar:sonar goal 
right after the basic build is done. 

This means  our phase 2 and phase 3 are running parallel sharing the same 
source tree, same local maven repo (where sonar:sonar should have all needed 
dependencies at the share local maven repo to run its task)

With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
until phase 2 is done. 

I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
locking started the happen

How does the lock mechanic work?  there must be a local file where both Maven 
instances are watching each other.  Is there an option to disable this lock?



> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests + jacoco  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 run in parallel sharing the same source 
> tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

 [ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated MNG-7433:
--
Description: 
I have a large multi modules java maven build where:

  * phase 1 - basic build + unit tests  - 40 min
  * phase 2 - sonar:sonar 20 min
  * phase 3 - final packaging and basic smoke-test - 20 min

To take advantage of Maven multi-threaded build, during the reactor build, one 
of our maven module spins another instance of Maven to run sonar:sonar goal 
right after the basic build is done. 

This means  our phase 2 and phase 3 are running parallel sharing the same 
source tree, same local maven repo (where sonar:sonar should have all needed 
dependencies at the share local maven repo to run its task)

With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
until phase 2 is done. 

I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
locking started the happen

How does the lock mechanic work?  there must be a local file where both Maven 
instances are watching each other.  Is there an option to disable this lock?


  was:
I have a large multi modules java maven build where:

  * phase 1 - basic build + unit tests  - 40 min
  * phase 2 - sonar:sonar 20 min
  * phase 3 - final packaging and basic smoke-test - 20 min

To take advantage of Maven multi-threaded build, during the reactor build, one 
of our maven module spins another instance of Maven to run sonar:sonar goal 
right after the basic build is done. 

This means  our phase 2 and phase 3 are running parallel sharing the same 
source tree, same local maven repo (where sonar:sonar should have all needed 
dependencies to run its task)

With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
until phase 2 is done. 

I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
locking started the happen

How does the lock mechanic work?  there must be a local file where both Maven 
instances are watching each other.  Is there an option to disable this lock?



> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 are running parallel sharing the same 
> source tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies at the share local maven repo to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

 [ 
https://issues.apache.org/jira/browse/MNG-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated MNG-7433:
--
Description: 
I have a large multi modules java maven build where:

  * phase 1 - basic build + unit tests  - 40 min
  * phase 2 - sonar:sonar 20 min
  * phase 3 - final packaging and basic smoke-test - 20 min

To take advantage of Maven multi-threaded build, during the reactor build, one 
of our maven module spins another instance of Maven to run sonar:sonar goal 
right after the basic build is done. 

This means  our phase 2 and phase 3 are running parallel sharing the same 
source tree, same local maven repo (where sonar:sonar should have all needed 
dependencies to run its task)

With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
until phase 2 is done. 

I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
locking started the happen

How does the lock mechanic work?  there must be a local file where both Maven 
instances are watching each other.  Is there an option to disable this lock?


  was:
I have a large multi modules java maven build where:

  * phase 1 - basic build + unit tests  - 40 min
  * phase 2 - sonar:sonar 20 min
  * phase 3 - final packaging and basic smoke-test - 20 min

To take advantage of Maven multi-threaded build, during the reactor build, one 
of our maven module spins another instance of Maven to run sonar:sonar goal 
right after the basic build is done. 

This means  our phase 1 and phase 2 are running parallel sharing the same 
source tree, same local maven repo (where sonar:sonar should have all needed 
dependencies to run its task)

With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
until phase 2 is done. 

I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
locking started the happen

How does the lock mechanic work?  there must be a local file where both Maven 
instances are watching each other.  Is there an option to disable this lock?



> multiple maven instances working same source tree can lock each other
> -
>
> Key: MNG-7433
> URL: https://issues.apache.org/jira/browse/MNG-7433
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.8.5
>Reporter: Dan Tran
>Priority: Major
>
> I have a large multi modules java maven build where:
>   * phase 1 - basic build + unit tests  - 40 min
>   * phase 2 - sonar:sonar 20 min
>   * phase 3 - final packaging and basic smoke-test - 20 min
> To take advantage of Maven multi-threaded build, during the reactor build, 
> one of our maven module spins another instance of Maven to run sonar:sonar 
> goal right after the basic build is done. 
> This means  our phase 2 and phase 3 are running parallel sharing the same 
> source tree, same local maven repo (where sonar:sonar should have all needed 
> dependencies to run its task)
> With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
> until phase 2 is done. 
> I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
> locking started the happen
> How does the lock mechanic work?  there must be a local file where both Maven 
> instances are watching each other.  Is there an option to disable this lock?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (MNG-7433) multiple maven instances working same source tree can lock each other

[Dan Tran (Jira)]

Dan Tran created MNG-7433:
-

 Summary: multiple maven instances working same source tree can 
lock each other
 Key: MNG-7433
 URL: https://issues.apache.org/jira/browse/MNG-7433
 Project: Maven
  Issue Type: Bug
Affects Versions: 3.8.5
Reporter: Dan Tran


I have a large multi modules java maven build where:

  * phase 1 - basic build + unit tests  - 40 min
  * phase 2 - sonar:sonar 20 min
  * phase 3 - final packaging and basic smoke-test - 20 min

To take advantage of Maven multi-threaded build, during the reactor build, one 
of our maven module spins another instance of Maven to run sonar:sonar goal 
right after the basic build is done. 

This means  our phase 1 and phase 2 are running parallel sharing the same 
source tree, same local maven repo (where sonar:sonar should have all needed 
dependencies to run its task)

With maven-3.8.5, parallelization is no longer possible, phase 3 is blocked 
until phase 2 is done. 

I am able to trace it to  https://github.com/apache/maven/pull/628 where the 
locking started the happen

How does the lock mechanic work?  there must be a local file where both Maven 
instances are watching each other.  Is there an option to disable this lock?




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] (SUREFIRE-2033) Regression: 3.0.0-M5 misidentifies JUnit 5 configuration as JUnit 4

[Dan Tran (Jira)]

[ https://issues.apache.org/jira/browse/SUREFIRE-2033 ]


Dan Tran deleted comment on SUREFIRE-2033:


was (Author: dantran):
I do agree with [~roxspring] this is a regression compared with M4.  with M6, i 
now have to manually add provider ( in my case testing, junit4, junit5) to 
surefire plugin's dependencies

In M4 it works out of the box [1]

https://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html

> Regression: 3.0.0-M5 misidentifies JUnit 5 configuration as JUnit 4
> ---
>
> Key: SUREFIRE-2033
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2033
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 3.0.0-M5
>Reporter: Robert James Oxspring
>Priority: Major
> Attachments: example.zip
>
>
> 3.0.0-M5 misidentifies a clearly JUnit 5 configuration and attempts to run 
> using JUnit 4.
> In particular the attached project only has JUnit 5 dependencies:
> {code:java}
> 
> 
> org.junit.jupiter
> junit-jupiter-engine
> 5.6.2
> test
> 
> 
> org.junit.platform
> junit-platform-runner
> 1.6.2
> test
> 
> {code}
> and a JUnit 5 test:
> {code:java}
> package pkg;
> import org.junit.jupiter.api.Test;
> class JUnit5Test {
> @Test
> public void test() {
> }
> }{code}
> When the project is run with with older version (as far back as 2.22.0) the 
> test is found and run, but 3.0.0-M5 fails to run any test:
> {code:java}
> % mvn test -Dsurefire.version=2.22.0 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.337 s
> [INFO] Finished at: 2022-03-03T09:42:27Z
> [INFO] 
>  
> {code}
>  
> {code:java}
> % mvn test -Dsurefire.version=3.0.0-M1 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.708 s
> [INFO] Finished at: 2022-03-03T09:34:58Z
> [INFO] 
>  
> % mvn test -Dsurefire.version=3.0.0-M2 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.701 s
> [INFO] Finished at: 2022-03-03T09:36:26Z
> [INFO] 
> 
> % mvn test -Dsurefire.version=3.0.0-M3 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.612 s
> [INFO] Finished at: 2022-03-03T09:37:02Z
> [INFO] 
> 
> % mvn test -Dsurefire.version=3.0.0-M4 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.619 s
> [INFO] Finished at: 2022-03-03T09:37:37Z
> [INFO] 
> 
> % mvn test -Dsurefire.version=3.0.0-M5 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.344 s
> [INFO] Finished at: 2022-03-03T09:38:01Z
> [INFO] 
> {code}
> Note the final run above using 3.0.0-M5 and logging: {{Tests run: 0}}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (SUREFIRE-2033) Regression: 3.0.0-M5 misidentifies JUnit 5 configuration as JUnit 4

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17505302#comment-17505302
 ] 

Dan Tran commented on SUREFIRE-2033:


I do agree with [~roxspring] this is a regression compared with M4.  with M6, i 
now have to manually add provider ( in my case testing, junit4, junit5) to 
surefire plugin's dependencies

In M4 it works out of the box [1]

https://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html

> Regression: 3.0.0-M5 misidentifies JUnit 5 configuration as JUnit 4
> ---
>
> Key: SUREFIRE-2033
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2033
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 3.0.0-M5
>Reporter: Robert James Oxspring
>Priority: Major
> Attachments: example.zip
>
>
> 3.0.0-M5 misidentifies a clearly JUnit 5 configuration and attempts to run 
> using JUnit 4.
> In particular the attached project only has JUnit 5 dependencies:
> {code:java}
> 
> 
> org.junit.jupiter
> junit-jupiter-engine
> 5.6.2
> test
> 
> 
> org.junit.platform
> junit-platform-runner
> 1.6.2
> test
> 
> {code}
> and a JUnit 5 test:
> {code:java}
> package pkg;
> import org.junit.jupiter.api.Test;
> class JUnit5Test {
> @Test
> public void test() {
> }
> }{code}
> When the project is run with with older version (as far back as 2.22.0) the 
> test is found and run, but 3.0.0-M5 fails to run any test:
> {code:java}
> % mvn test -Dsurefire.version=2.22.0 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.337 s
> [INFO] Finished at: 2022-03-03T09:42:27Z
> [INFO] 
>  
> {code}
>  
> {code:java}
> % mvn test -Dsurefire.version=3.0.0-M1 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.708 s
> [INFO] Finished at: 2022-03-03T09:34:58Z
> [INFO] 
>  
> % mvn test -Dsurefire.version=3.0.0-M2 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.701 s
> [INFO] Finished at: 2022-03-03T09:36:26Z
> [INFO] 
> 
> % mvn test -Dsurefire.version=3.0.0-M3 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.612 s
> [INFO] Finished at: 2022-03-03T09:37:02Z
> [INFO] 
> 
> % mvn test -Dsurefire.version=3.0.0-M4 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.619 s
> [INFO] Finished at: 2022-03-03T09:37:37Z
> [INFO] 
> 
> % mvn test -Dsurefire.version=3.0.0-M5 | tail
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO] 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time:  1.344 s
> [INFO] Finished at: 2022-03-03T09:38:01Z
> [INFO] 
> {code}
> Note the final run above using 3.0.0-M5 and logging: {{Tests run: 0}}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (MRESOLVER-123) Provide a global locking sync context by default

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174068#comment-17174068
 ] 

Dan Tran edited comment on MRESOLVER-123 at 8/10/20, 4:21 AM:
--

some thing like below 
{code:java}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) 
on project xxx: Compilation failure: Compilation failure: 
[ERROR] /path/to/a/java/file.java:[16,35] package x.y.z does not exist
{code}

The package paths include both internal and external packages

and it is not possible to see if they can occur without lock ( too slow)




was (Author: dantran):
some thing like below 
{code:java}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) 
on project xxx: Compilation failure: Compilation failure: 
[ERROR] /path/to/a/java/file.java:[16,35] package x.y.z does not exist
{code}

The package paths include both internal and external packages

and it is not possible to see if they can occur without lock ( to slow)



> Provide a global locking sync context by default
> 
>
> Key: MRESOLVER-123
> URL: https://issues.apache.org/jira/browse/MRESOLVER-123
> Project: Maven Resolver
>  Issue Type: New Feature
>  Components: resolver
>Affects Versions: 1.4.2
>Reporter: Michael Osipov
>Assignee: Michael Osipov
>Priority: Critical
> Fix For: 1.5.0
>
> Attachments: checksum-error-debug.log, mvn-debug-1.4.3-123.txt.gz
>
>
> This is an umbrella ticket for a long standing issue with Maven Resolver: Our 
> concurrency support is mediocre in a way that if two or more threads try to 
> download the same file and fail to queue those write actions nicely. The 
> problem is that The {{SyncContext}} and the its factory provided by Maven 
> Resolver does not employ any locking at all. As layed out in detail in 
> MRESOLVER-114 we need striped read write locks on artifacts and its metadata. 
> This issue shall track progress on it. Even Takari Concurrent Repository 
> extension does not help because it is only intended to synchronize concurrent 
> access by multple JVMs and not threads.
> This improvement will provide solely a global locking sync context which will 
> work in *single* JVM. It is a non-goal to make it work with mulitple JVMs. A 
> downside of this solution is that is coarse, possible degregation of 
> performance for the sake of stability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (MRESOLVER-123) Provide a global locking sync context by default

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174068#comment-17174068
 ] 

Dan Tran commented on MRESOLVER-123:


some thing like below 
{code:java}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) 
on project xxx: Compilation failure: Compilation failure: 
[ERROR] /path/to/a/java/file.java:[16,35] package x.y.z does not exist
{code}

The package paths include both internal and external packages

and it is not possible to see if they can occur without lock ( to slow)



> Provide a global locking sync context by default
> 
>
> Key: MRESOLVER-123
> URL: https://issues.apache.org/jira/browse/MRESOLVER-123
> Project: Maven Resolver
>  Issue Type: New Feature
>  Components: resolver
>Affects Versions: 1.4.2
>Reporter: Michael Osipov
>Assignee: Michael Osipov
>Priority: Critical
> Fix For: 1.5.0
>
> Attachments: checksum-error-debug.log, mvn-debug-1.4.3-123.txt.gz
>
>
> This is an umbrella ticket for a long standing issue with Maven Resolver: Our 
> concurrency support is mediocre in a way that if two or more threads try to 
> download the same file and fail to queue those write actions nicely. The 
> problem is that The {{SyncContext}} and the its factory provided by Maven 
> Resolver does not employ any locking at all. As layed out in detail in 
> MRESOLVER-114 we need striped read write locks on artifacts and its metadata. 
> This issue shall track progress on it. Even Takari Concurrent Repository 
> extension does not help because it is only intended to synchronize concurrent 
> access by multple JVMs and not threads.
> This improvement will provide solely a global locking sync context which will 
> work in *single* JVM. It is a non-goal to make it work with mulitple JVMs. A 
> downside of this solution is that is coarse, possible degregation of 
> performance for the sake of stability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (MRESOLVER-123) Provide a global locking sync context by default

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173401#comment-17173401
 ] 

Dan Tran commented on MRESOLVER-123:


I do have random compilation failures.  The error sounds like my dependencies 
not available at my local repo.  this leads me this issue and want to try it out

so definitely global lock has a big impact on large CI build where we have to 
build from the clean local repo 

> Provide a global locking sync context by default
> 
>
> Key: MRESOLVER-123
> URL: https://issues.apache.org/jira/browse/MRESOLVER-123
> Project: Maven Resolver
>  Issue Type: New Feature
>  Components: resolver
>Affects Versions: 1.4.2
>Reporter: Michael Osipov
>Assignee: Michael Osipov
>Priority: Critical
> Fix For: 1.5.0
>
> Attachments: checksum-error-debug.log, mvn-debug-1.4.3-123.txt.gz
>
>
> This is an umbrella ticket for a long standing issue with Maven Resolver: Our 
> concurrency support is mediocre in a way that if two or more threads try to 
> download the same file and fail to queue those write actions nicely. The 
> problem is that The {{SyncContext}} and the its factory provided by Maven 
> Resolver does not employ any locking at all. As layed out in detail in 
> MRESOLVER-114 we need striped read write locks on artifacts and its metadata. 
> This issue shall track progress on it. Even Takari Concurrent Repository 
> extension does not help because it is only intended to synchronize concurrent 
> access by multple JVMs and not threads.
> This improvement will provide solely a global locking sync context which will 
> work in *single* JVM. It is a non-goal to make it work with mulitple JVMs. A 
> downside of this solution is that is coarse, possible degregation of 
> performance for the sake of stability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (MRESOLVER-123) Provide a global locking sync context by default

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173329#comment-17173329
 ] 

Dan Tran commented on MRESOLVER-123:


w/o smart build,  no issue with Resolver 1.4.2  but 15.1-SNAPSHOT takes forever 
( sorry i cant give you a number)

> Provide a global locking sync context by default
> 
>
> Key: MRESOLVER-123
> URL: https://issues.apache.org/jira/browse/MRESOLVER-123
> Project: Maven Resolver
>  Issue Type: New Feature
>  Components: resolver
>Affects Versions: 1.4.2
>Reporter: Michael Osipov
>Assignee: Michael Osipov
>Priority: Critical
> Fix For: 1.5.0
>
> Attachments: checksum-error-debug.log, mvn-debug-1.4.3-123.txt.gz
>
>
> This is an umbrella ticket for a long standing issue with Maven Resolver: Our 
> concurrency support is mediocre in a way that if two or more threads try to 
> download the same file and fail to queue those write actions nicely. The 
> problem is that The {{SyncContext}} and the its factory provided by Maven 
> Resolver does not employ any locking at all. As layed out in detail in 
> MRESOLVER-114 we need striped read write locks on artifacts and its metadata. 
> This issue shall track progress on it. Even Takari Concurrent Repository 
> extension does not help because it is only intended to synchronize concurrent 
> access by multple JVMs and not threads.
> This improvement will provide solely a global locking sync context which will 
> work in *single* JVM. It is a non-goal to make it work with mulitple JVMs. A 
> downside of this solution is that is coarse, possible degregation of 
> performance for the sake of stability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (MRESOLVER-123) Provide a global locking sync context by default

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173017#comment-17173017
 ] 

Dan Tran commented on MRESOLVER-123:


my 350 modules build without tests + Takari 4 thread smart builder  takes 
around 5min 

with resolver 1.5.1-SNAPSHOT,  after 15min, I still don't see the build rolling 
to first module

without Takari smart builder, it is a little better but still unbearable



> Provide a global locking sync context by default
> 
>
> Key: MRESOLVER-123
> URL: https://issues.apache.org/jira/browse/MRESOLVER-123
> Project: Maven Resolver
>  Issue Type: New Feature
>  Components: resolver
>Affects Versions: 1.4.2
>Reporter: Michael Osipov
>Assignee: Michael Osipov
>Priority: Critical
> Fix For: 1.5.0
>
> Attachments: checksum-error-debug.log, mvn-debug-1.4.3-123.txt.gz
>
>
> This is an umbrella ticket for a long standing issue with Maven Resolver: Our 
> concurrency support is mediocre in a way that if two or more threads try to 
> download the same file and fail to queue those write actions nicely. The 
> problem is that The {{SyncContext}} and the its factory provided by Maven 
> Resolver does not employ any locking at all. As layed out in detail in 
> MRESOLVER-114 we need striped read write locks on artifacts and its metadata. 
> This issue shall track progress on it. Even Takari Concurrent Repository 
> extension does not help because it is only intended to synchronize concurrent 
> access by multple JVMs and not threads.
> This improvement will provide solely a global locking sync context which will 
> work in *single* JVM. It is a non-goal to make it work with mulitple JVMs. A 
> downside of this solution is that is coarse, possible degregation of 
> performance for the sake of stability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (MRESOLVER-123) Provide a global locking sync context by default

[Dan Tran (Jira)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172965#comment-17172965
 ] 

Dan Tran commented on MRESOLVER-123:


I rebuilt maven 3.6.3 with apache-resolver-1.5.1-SNAPSHOT and notice for clean 
local repo, download dependencies are serialized, and therefore can notice the 
big slowing down.   

the original 3.6.3 with older resolver download dependencies in parallel 

> Provide a global locking sync context by default
> 
>
> Key: MRESOLVER-123
> URL: https://issues.apache.org/jira/browse/MRESOLVER-123
> Project: Maven Resolver
>  Issue Type: New Feature
>  Components: resolver
>Affects Versions: 1.4.2
>Reporter: Michael Osipov
>Assignee: Michael Osipov
>Priority: Critical
> Fix For: 1.5.0
>
> Attachments: checksum-error-debug.log, mvn-debug-1.4.3-123.txt.gz
>
>
> This is an umbrella ticket for a long standing issue with Maven Resolver: Our 
> concurrency support is mediocre in a way that if two or more threads try to 
> download the same file and fail to queue those write actions nicely. The 
> problem is that The {{SyncContext}} and the its factory provided by Maven 
> Resolver does not employ any locking at all. As layed out in detail in 
> MRESOLVER-114 we need striped read write locks on artifacts and its metadata. 
> This issue shall track progress on it. Even Takari Concurrent Repository 
> extension does not help because it is only intended to synchronize concurrent 
> access by multple JVMs and not threads.
> This improvement will provide solely a global locking sync context which will 
> work in *single* JVM. It is a non-goal to make it work with mulitple JVMs. A 
> downside of this solution is that is coarse, possible degregation of 
> performance for the sake of stability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (MNG-6604) Intermittent failures while downloading GAVs from Nexus

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MNG-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814886#comment-16814886
 ] 

Dan Tran edited comment on MNG-6604 at 4/10/19 9:41 PM:


This is very strange,  I am very sure there are a lot of us successfully 
building maven projects with hundreds of modules using smart builder. So far I 
have not seen this issue even with the broken takari local-repo.  I am using 
artifactory thou

Could this be one of those build ordering issue at user side? This is  fixable 
via maven dependency 


was (Author: dantran):
This is very strange,  I am very sure there are a lot of us successfully 
building maven projects with hundreds of modules using smart builder. So far I 
have not seen this issue even with the broken takari local-repo.  I am using 
artifactory thou

> Intermittent failures while downloading GAVs from Nexus
> ---
>
> Key: MNG-6604
> URL: https://issues.apache.org/jira/browse/MNG-6604
> Project: Maven
>  Issue Type: Bug
>  Components: Command Line, Toolchains
>Affects Versions: 3.6.0
> Environment: Nexus OSS 3.15.2-01
> Docker 18.09.2 on Ubuntu 18.04.2 LTS
> Gitlab runner 11.8.0
>Reporter: Ivan Rizzante
>Priority: Major
> Attachments: docker-env.txt, log.txt
>
>
> Hello
> we're running maven 3.6.0 builds in a docker container and we use Nexus OSS 
> configured as proxy for Maven Central.
> While running our builds using Gitlab CI, we're experiencing intermittent 
> build failures because Maven cannot find artifacts in Nexus which we verified 
> they are actually available.
> Error example below:
> {noformat}
> 20744 [main] [ERROR] Failed to execute goal on project EcotransitWSClient: 
> Could not resolve dependencies for project 
> it.sdb.ecotransit:EcotransitWSClient:jar:7.0.1-SNAPSHOT: Could not transfer 
> artifact commons-beanutils:commons-beanutils:jar:1.9.3 from/to nexus 
> (http://maven-repo.sdb.it:8081/repository/maven-public/): 
> /builds/sdb/webportal/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar.part
>  (No such file or directory) -> [Help 1]
> {noformat}
> I attached the full maven build log and our Docker env settings.
> We tried disabling the keep alive and also disabling the connection pooling 
> but nothing seems to fix the issue.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MNG-6604) Intermittent failures while downloading GAVs from Nexus

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MNG-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814886#comment-16814886
 ] 

Dan Tran commented on MNG-6604:
---

This is very strange,  I am very sure there are a lot of us successfully 
building maven projects with hundreds of modules using smart builder. So far I 
have not seen this issue even with the broken takari local-repo.  I am using 
artifactory thou

> Intermittent failures while downloading GAVs from Nexus
> ---
>
> Key: MNG-6604
> URL: https://issues.apache.org/jira/browse/MNG-6604
> Project: Maven
>  Issue Type: Bug
>  Components: Command Line, Toolchains
>Affects Versions: 3.6.0
> Environment: Nexus OSS 3.15.2-01
> Docker 18.09.2 on Ubuntu 18.04.2 LTS
> Gitlab runner 11.8.0
>Reporter: Ivan Rizzante
>Priority: Major
> Attachments: docker-env.txt, log.txt
>
>
> Hello
> we're running maven 3.6.0 builds in a docker container and we use Nexus OSS 
> configured as proxy for Maven Central.
> While running our builds using Gitlab CI, we're experiencing intermittent 
> build failures because Maven cannot find artifacts in Nexus which we verified 
> they are actually available.
> Error example below:
> {noformat}
> 20744 [main] [ERROR] Failed to execute goal on project EcotransitWSClient: 
> Could not resolve dependencies for project 
> it.sdb.ecotransit:EcotransitWSClient:jar:7.0.1-SNAPSHOT: Could not transfer 
> artifact commons-beanutils:commons-beanutils:jar:1.9.3 from/to nexus 
> (http://maven-repo.sdb.it:8081/repository/maven-public/): 
> /builds/sdb/webportal/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar.part
>  (No such file or directory) -> [Help 1]
> {noformat}
> I attached the full maven build log and our Docker env settings.
> We tried disabling the keep alive and also disabling the connection pooling 
> but nothing seems to fix the issue.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1585) Align JUnit Platform version at runtime

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782257#comment-16782257
 ] 

Dan Tran commented on SUREFIRE-1585:


[~tibor17] could you deploy a new snaphost for 3.0.0-M4-SNAPSHOT?

> Align JUnit Platform version at runtime
> ---
>
> Key: SUREFIRE-1585
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1585
> Project: Maven Surefire
>  Issue Type: Improvement
>  Components: JUnit 5.x support
>Affects Versions: 2.22.1
>Reporter: Christian Stein
>Assignee: Christian Stein
>Priority: Minor
>  Labels: features
> Fix For: 3.0.0-M4
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The internal *JUnitPlatformProviderInfo* should use the JUnit Platform 
> Launcher artifact version matching the one (highest?) the user's classpath 
> hints to.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763744#comment-16763744
 ] 

Dan Tran edited comment on SUREFIRE-1546 at 2/8/19 4:35 PM:


Yes, we can consume the snapshot directly. Hope we don't use the snapshot for 
too long.  Thanks


was (Author: dantran):
Yes, we can consume the snapshot directly. Since this is production build, we 
also wonder when we can be out of snapshot mode 

> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763744#comment-16763744
 ] 

Dan Tran commented on SUREFIRE-1546:


Yes, we can consume the snapshot directly. Since this is production build, we 
also wonder when we can be out of snapshot mode 

> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763134#comment-16763134
 ] 

Dan Tran edited comment on SUREFIRE-1546 at 2/7/19 10:29 PM:
-

"By default, there would be current behavior."  this is good news for us be 
able to replace testcase name for the display name.  

Can we have M4 release rolling out? 



was (Author: dantran):
"By default, there would be current behavior."  this is good news for us be 
able to replace testcase name for the display name.  

Can we have M4 release rolling out?



> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763134#comment-16763134
 ] 

Dan Tran commented on SUREFIRE-1546:


"By default, there would be current behavior."  this is good news for us be 
able to replace testcase name for the display name.  

Can we have M4 release rolling out?



> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758909#comment-16758909
 ] 

Dan Tran edited comment on SUREFIRE-1546 at 2/2/19 6:53 AM:


what is the final verdict? 

what is the format of adding display name to std-out:   displayName= ? and 
as the first line of each test case ( test method) stdout?




was (Author: dantran):
what is the final verdict? 

what is the format of adding display name to std-out:   displayName= ?



> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758909#comment-16758909
 ] 

Dan Tran commented on SUREFIRE-1546:


what is the final verdict? 

what is the format of adding display name to std-out:   displayName= ?



> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758478#comment-16758478
 ] 

Dan Tran commented on SUREFIRE-1546:


correction, we are not parsing the content of display name but for reporting 
purpose only. And also agree, we must not break the compatibility

so using an additional surefire config should be perfect for us

> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758382#comment-16758382
 ] 

Dan Tran edited comment on SUREFIRE-1546 at 2/1/19 2:59 PM:


For reporting purpose.   We parse TEST-*.xml under target/surefire-report to 
render the report at Micro Focus quality center.  

So having displayName to replace the actual testcase's name and classname are 
so crucial for us.

{code:java}

{code}

We are also using 
https://github.com/BogdanLivadariu/bootstraped-multi-test-results-report to 
post report to jenkins as well 

please please keep the implementation, or provide an option to activate it



was (Author: dantran):
For reporting purpose.   We parse TEST-*.xml under target/surefire-report to 
render the report at Micro Focus quality center.  

So having displayName to replace the actual testcase's name and classname are 
so crucial for us.

{code:java}

{code}

It works


> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758382#comment-16758382
 ] 

Dan Tran commented on SUREFIRE-1546:


For reporting purpose.   We parse TEST-*.xml under target/surefire-report to 
render the report at Micro Focus quality center.  

So having displayName to replace the actual testcase's name and classname are 
so crucial for us.

{code:java}

{code}

It works


> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755320#comment-16755320
 ] 

Dan Tran commented on SUREFIRE-1546:


tested 3.0.0-M4-SNAPSHOT, looking forward to M4 release. Many thanks

> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16753392#comment-16753392
 ] 

Dan Tran commented on SUREFIRE-1546:


@Tibor please deploy new snapshot at 
https://repository.apache.org/content/groups/snapshots/org/apache/maven/plugins/maven-surefire-plugin


> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Tibor Digana
>Priority: Major
>  Labels: junit5
> Fix For: 3.0.0-M4
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-544) Work around JSch issue #122

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732805#comment-16732805
 ] 

Dan Tran commented on WAGON-544:


the branch looks good, I integrated WAGON-544 branch with maven 3.6.1-SNAPSHOT. 
 the wagon-maven-plugin ssh-it test passes

> Work around JSch issue #122
> ---
>
> Key: WAGON-544
> URL: https://issues.apache.org/jira/browse/WAGON-544
> Project: Maven Wagon
>  Issue Type: Task
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Michael Osipov
>Assignee: Michael Osipov
>Priority: Major
> Fix For: 3.3.1
>
>
> JSch's {{Channels$MyPipedInputStream#read()}} does not return -1 when EOF is 
> reached. This causes the reading thread to block and never to resume. This 
> happens only when more bytes are requested to be read than available from the 
> stream. It does not read at most bytes. So one has to know the last 
> less-than-the-buffer-length chunk and recalculate remaining bytes. This 
> worked conincidentially because it worked that way.
> This is brittle, as long as the upstream bug is not fixed we will restore the 
> previous {{AbstractWagon#transfer()}} implementation in {{AbstractJschWagon}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732297#comment-16732297
 ] 

Dan Tran commented on WAGON-543:


Let's go with that option

> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Assignee: Michael Osipov
>Priority: Major
> Attachments: debugger screenshot.png
>
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer
> Reverting WAGON-537 fixes this issue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730905#comment-16730905
 ] 

Dan Tran commented on WAGON-543:


Looks like new changes at AbstractWagon's transfer method requires NIO behavior 
and jcsh cant provide?

> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Assignee: Michael Osipov
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer
> Reverting WAGON-537 fixes this issue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730860#comment-16730860
 ] 

Dan Tran edited comment on WAGON-543 at 12/30/18 1:04 AM:
--

To help with debugging this issue, I cooked up a sshshell wrapper of wagon-ssh 
with a demo unit test at

https://github.com/dantran/wagon-ssh-demo

Just load up this maven module to your IDE, adjust test credential, and run it


was (Author: dantran):
To help with debugging this issue, I cooked up a sshshell wrapper of wagon-ssh 
with a demo unit test at

https://github.com/dantran/wagon-ssh-demo

> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Assignee: Michael Osipov
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer
> Reverting WAGON-537 fixes this issue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730860#comment-16730860
 ] 

Dan Tran commented on WAGON-543:


To help with debugging this issue, I cooked up a sshshell wrapper of wagon-ssh 
with a demo unit test at

https://github.com/dantran/wagon-ssh-demo

> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Assignee: Michael Osipov
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer
> Reverting WAGON-537 fixes this issue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730845#comment-16730845
 ] 

Dan Tran edited comment on WAGON-543 at 12/29/18 10:55 PM:
---

it passes b/c you have wagon-provider-api-3.2.0.jar under your maven's lib 
directory

did you build maven-3.6.0-SNAPSHOT with wagon 3.3.1-SNAPSHOT

you should see


{code:java}
ls wagon*
wagon-file-3.3.1-SNAPSHOT.jar  wagon-http-3.3.1-SNAPSHOT-shaded.jar  
wagon-provider-api-3.3.1-SNAPSHOT.jar
{code}







was (Author: dantran):
it passes b/c you have wagon-provider-api-3.2.0.jar under your maven's lib 
directory

did you build maven-3.6.0-SNAPSHOT with wagon 3.3.1-SNAPSHOT

you should see

ls wagon*
wagon-file-3.3.1-SNAPSHOT.jar  wagon-http-3.3.1-SNAPSHOT-shaded.jar  
wagon-provider-api-3.3.1-SNAPSHOT.jar




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Assignee: Michael Osipov
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer
> Reverting WAGON-537 fixes this issue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730845#comment-16730845
 ] 

Dan Tran commented on WAGON-543:


it passes b/c you have wagon-provider-api-3.2.0.jar under your maven's lib 
directory

did you build maven-3.6.0-SNAPSHOT with wagon 3.3.1-SNAPSHOT

you should see

ls wagon*
wagon-file-3.3.1-SNAPSHOT.jar  wagon-http-3.3.1-SNAPSHOT-shaded.jar  
wagon-provider-api-3.3.1-SNAPSHOT.jar




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Assignee: Michael Osipov
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer
> Reverting WAGON-537 fixes this issue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-537) Maven transfer speed of large artifacts is slow due to unsuitable buffer strategy

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730818#comment-16730818
 ] 

Dan Tran commented on WAGON-537:


To test wagon 3.3.0 download speed, I build maven-3.6.1-snapshot with the 
latest wagon, and run mvn dependency:get to download a single 7G+ file.  

Also tested with another artifactory's mirror over a wan link,   the download 
speed for mvn 3.6, 3.6.1, and curl are the same.  ~60M/s

Note: the network traffic during the holiday is very quiet ( ie no build/test)


> Maven transfer speed of large artifacts is slow due to unsuitable buffer 
> strategy
> -
>
> Key: WAGON-537
> URL: https://issues.apache.org/jira/browse/WAGON-537
> Project: Maven Wagon
>  Issue Type: Improvement
>  Components: wagon-http, wagon-provider-api
>Affects Versions: 3.2.0
> Environment: Windows 10, JDK 1.8, Nexus  Artifact store > 100MB/s 
> network connection.
>Reporter: Olaf Otto
>Assignee: Michael Osipov
>Priority: Major
>  Labels: perfomance
> Fix For: 3.3.0
>
> Attachments: wagon-issue.png
>
>
> We are using maven for build process automation with docker. This sometimes 
> involves uploading and downloading artifacts with a few gigabytes in size. 
> Here, maven's transfer speed is consistently and reproducibly slow. For 
> instance, an artifact with 7,5 GB in size took almost two hours to transfer 
> in spite of a 100 MB/s connection with respective reproducible download speed 
> from the remote nexus artifact repository when using a browser to download. 
> The same is true when uploding such an artifact.
> I have investigated the issue using JProfiler. The result shows an issue in 
> AbstractWagon's transfer( Resource resource, InputStream input, OutputStream 
> output, int requestType, long maxSize ) method used for remote artifacts and 
> the same issue in AbstractHttpClientWagon#writeTo(OutputStream).
> Here, the input stream is read in a loop using a 4 Kb buffer. Whenever data 
> is received, the received data is pushed to downstream listeners via 
> fireTransferProgress. These listeners (or rather consumers) perform expensive 
> tasks.
> Now, the underlying InputStream implementation used in transfer will return 
> calls to read(buffer, offset, length) as soon as *some* data is available. 
> That is, fireTransferProgress may well be invoked with an average number of 
> bytes less than half the buffer capacity (this varies with the underlying 
> network and hardware architecture). Consequently, fireTransferProgress is 
> invoked *millions of times* for large files. As this is a blocking operation, 
> the time spent in fireTransferProgress dominates and drastically slows down 
> the transfers by at least one order of magnitude. 
> !wagon-issue.png! 
> In our case, we found download speed reduced from a theoretical optimum of 
> ~80 seconds to to more than 3200 seconds.
> From an architectural perspective, I would not want to make the consumers / 
> listeners invoked via fireTransferProgress aware of their potential impact on 
> download speed, but rather refactor the transfer method such that it uses a 
> buffer strategy reducing the the number of fireTransferProgress invocations. 
> This should be done with regard to the expected file size of the transfer, 
> such that fireTransferProgress is invoked often enough but not to frequent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated WAGON-543:
---
Description: 
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

other notes:

  * No problem with maven 3.6.0
  * I also have internal ssh library wrapper of wagon-ssh to perform 
sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
  * The root cause likely is from WAGON-537  with changes at AbstractWagon 
which may have side effect on internal JSCH 32K buffer

Reverting WAGON-537 fixes this issue



  was:
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

other notes:

  * No problem with maven 3.6.0
  * I also have internal ssh library wrapper of wagon-ssh to perform 
sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
  * The root cause likely is from WAGON-537  with changes at AbstractWagon 
which may have side effect on internal JSCH 32K buffer




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer
> Reverting WAGON-537 fixes this issue



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-537) Maven transfer speed of large artifacts is slow due to unsuitable buffer strategy

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16730513#comment-16730513
 ] 

Dan Tran commented on WAGON-537:


I see no gain in my download. Basic single download test with curl 

curl -O 
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
100 6779M  100 6779M0 0   301M  0  0:00:22  0:00:22 --:--:--  343M

maven 3.6.1 with wagon 3.3 is around 115M/s

maven 3.6.0 with wagon 3.2 is around 115M/s

No improvement between 3.6.0/3.6.1.  Did check to make sure wagon-3.3 does 
landed at my local maven-3.6.1 build

> Maven transfer speed of large artifacts is slow due to unsuitable buffer 
> strategy
> -
>
> Key: WAGON-537
> URL: https://issues.apache.org/jira/browse/WAGON-537
> Project: Maven Wagon
>  Issue Type: Improvement
>  Components: wagon-http, wagon-provider-api
>Affects Versions: 3.2.0
> Environment: Windows 10, JDK 1.8, Nexus  Artifact store > 100MB/s 
> network connection.
>Reporter: Olaf Otto
>Assignee: Michael Osipov
>Priority: Major
>  Labels: perfomance
> Fix For: 3.3.0
>
> Attachments: wagon-issue.png
>
>
> We are using maven for build process automation with docker. This sometimes 
> involves uploading and downloading artifacts with a few gigabytes in size. 
> Here, maven's transfer speed is consistently and reproducibly slow. For 
> instance, an artifact with 7,5 GB in size took almost two hours to transfer 
> in spite of a 100 MB/s connection with respective reproducible download speed 
> from the remote nexus artifact repository when using a browser to download. 
> The same is true when uploding such an artifact.
> I have investigated the issue using JProfiler. The result shows an issue in 
> AbstractWagon's transfer( Resource resource, InputStream input, OutputStream 
> output, int requestType, long maxSize ) method used for remote artifacts and 
> the same issue in AbstractHttpClientWagon#writeTo(OutputStream).
> Here, the input stream is read in a loop using a 4 Kb buffer. Whenever data 
> is received, the received data is pushed to downstream listeners via 
> fireTransferProgress. These listeners (or rather consumers) perform expensive 
> tasks.
> Now, the underlying InputStream implementation used in transfer will return 
> calls to read(buffer, offset, length) as soon as *some* data is available. 
> That is, fireTransferProgress may well be invoked with an average number of 
> bytes less than half the buffer capacity (this varies with the underlying 
> network and hardware architecture). Consequently, fireTransferProgress is 
> invoked *millions of times* for large files. As this is a blocking operation, 
> the time spent in fireTransferProgress dominates and drastically slows down 
> the transfers by at least one order of magnitude. 
> !wagon-issue.png! 
> In our case, we found download speed reduced from a theoretical optimum of 
> ~80 seconds to to more than 3200 seconds.
> From an architectural perspective, I would not want to make the consumers / 
> listeners invoked via fireTransferProgress aware of their potential impact on 
> download speed, but rather refactor the transfer method such that it uses a 
> buffer strategy reducing the the number of fireTransferProgress invocations. 
> This should be done with regard to the expected file size of the transfer, 
> such that fireTransferProgress is invoked often enough but not to frequent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated WAGON-543:
---
Description: 
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

other notes:

  * No problem with maven 3.6.0
  * I also have internal ssh library wrapper of wagon-ssh to perform 
sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
  * The root cause likely is from WAGON-537  with changes at AbstractWagon 
which may have side effect on internal JSCH 32K buffer



  was:
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

other notes:

  * No problem with maven 3.6.0
  * I also have internal ssh library wrapper of wagon-ssh to perform 
sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
3.3.0
  * The root cause likely is from WAGON-537  with changes at AbstractWagon 
which may have side effect on internal JSCH 32K buffer




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue with  wagon 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated WAGON-543:
---
Description: 
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

other notes:

  * No problem with maven 3.6.0
  * I also have internal ssh library wrapper of wagon-ssh to perform 
sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
3.3.0
  * The root cause likely is from WAGON-537  with changes at AbstractWagon 
which may have side effect on internal JSCH 32K buffer



  was:
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

other notes:

  * No problem with maven 3.6.0
  * I also have internal ssh library wrapper of wagon-ssh to perform 
sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
3.3.0
  * The root cause likely is from WAGON-537  with changes at AbstractWagon 
which may have side effect on internal jcsh 32K buffer




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
> 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal JSCH 32K buffer



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated WAGON-543:
---
Description: 
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

other notes:

  * No problem with maven 3.6.0
  * I also have internal ssh library wrapper of wagon-ssh to perform 
sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
3.3.0
  * The root cause likely is from WAGON-537  with changes at AbstractWagon 
which may have side effect on internal jcsh 32K buffer



  was:
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

No problem with maven 3.6.0

I also have internal ssh library wrapper of wagon-ssh to perform  
sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
3.3.0

The root cause likely is from WAGON-537  with changes at AbstractWagon




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> other notes:
>   * No problem with maven 3.6.0
>   * I also have internal ssh library wrapper of wagon-ssh to perform 
> sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
> 3.3.0
>   * The root cause likely is from WAGON-537  with changes at AbstractWagon 
> which may have side effect on internal jcsh 32K buffer



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated WAGON-543:
---
Description: 
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

No problem with maven 3.6.0

I also have internal ssh library wrapper of wagon-ssh to performce  
sshexe/upload/download.  Seeing same issue when pickup wagon 3.3.0

The root cause likely is from WAGON-537  with changes at AbstractWagon



  was:
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

No problem with maven 3.6.0

The root cause likely is from WAGON-537  with changes at AbstractWagon




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> No problem with maven 3.6.0
> I also have internal ssh library wrapper of wagon-ssh to performce  
> sshexe/upload/download.  Seeing same issue when pickup wagon 3.3.0
> The root cause likely is from WAGON-537  with changes at AbstractWagon



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated WAGON-543:
---
Description: 
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

No problem with maven 3.6.0

I also have internal ssh library wrapper of wagon-ssh to perform  
sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
3.3.0

The root cause likely is from WAGON-537  with changes at AbstractWagon



  was:
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

No problem with maven 3.6.0

I also have internal ssh library wrapper of wagon-ssh to performce  
sshexe/upload/download.  Seeing same issue when pickup wagon 3.3.0

The root cause likely is from WAGON-537  with changes at AbstractWagon




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> No problem with maven 3.6.0
> I also have internal ssh library wrapper of wagon-ssh to perform  
> sshexe/upload/download activities.  Seeing the same issue when pickup wagon 
> 3.3.0
> The root cause likely is from WAGON-537  with changes at AbstractWagon



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated WAGON-543:
---
Description: 
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

No problem with maven 3.6.0

The root cause likely is from WAGON-537  with changes at AbstractWagon



  was:
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

No problem with maven 3.6.0




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> No problem with maven 3.6.0
> The root cause likely is from WAGON-537  with changes at AbstractWagon



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/WAGON-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated WAGON-543:
---
Description: 
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key

No problem with maven 3.6.0



  was:
To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key




> wagon-ssh download hangs
> 
>
> Key: WAGON-543
> URL: https://issues.apache.org/jira/browse/WAGON-543
> Project: Maven Wagon
>  Issue Type: Bug
>  Components: wagon-ssh
>Affects Versions: 3.3.0
>Reporter: Dan Tran
>Priority: Major
>
> To reproduce this issue
>   1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
>   2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
>   3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0
> Assume your local account can ssh to same host with ssh key
> No problem with maven 3.6.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (WAGON-543) wagon-ssh download hangs

[Dan Tran (JIRA)]

Dan Tran created WAGON-543:
--

 Summary: wagon-ssh download hangs
 Key: WAGON-543
 URL: https://issues.apache.org/jira/browse/WAGON-543
 Project: Maven Wagon
  Issue Type: Bug
  Components: wagon-ssh
Affects Versions: 3.3.0
Reporter: Dan Tran


To reproduce this issue

  1. checkout https://github.com/mojohaus/wagon-maven-plugin.git
  2. Locally build maven 3.6.1-SNAPSHOT with wagon 3.3.0 and run it with
  3. cd src/test/project/ssh-it; mvn -Dit-plugin.version=2.0.0

Assume your local account can ssh to same host with ssh key





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-537) Maven transfer speed of large artifacts is slow due to unsuitable buffer strategy

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729902#comment-16729902
 ] 

Dan Tran commented on WAGON-537:


Tested with latest maven 3.6.1-SNAPSHOT with both latest wagon snapshot and 
wagon-2.3.0 currently at staging repo.  Dont see upload improvement. Here one 
of my 7.1 GB .ova upload  (7.1 GB at 35 MB/s) . Am I missing anything?

Maybe due to my Artifactory and jenkins are 1 network hop away?


> Maven transfer speed of large artifacts is slow due to unsuitable buffer 
> strategy
> -
>
> Key: WAGON-537
> URL: https://issues.apache.org/jira/browse/WAGON-537
> Project: Maven Wagon
>  Issue Type: Improvement
>  Components: wagon-http, wagon-provider-api
>Affects Versions: 3.2.0
> Environment: Windows 10, JDK 1.8, Nexus  Artifact store > 100MB/s 
> network connection.
>Reporter: Olaf Otto
>Assignee: Michael Osipov
>Priority: Major
>  Labels: perfomance
> Fix For: 3.3.0
>
> Attachments: wagon-issue.png
>
>
> We are using maven for build process automation with docker. This sometimes 
> involves uploading and downloading artifacts with a few gigabytes in size. 
> Here, maven's transfer speed is consistently and reproducibly slow. For 
> instance, an artifact with 7,5 GB in size took almost two hours to transfer 
> in spite of a 100 MB/s connection with respective reproducible download speed 
> from the remote nexus artifact repository when using a browser to download. 
> The same is true when uploding such an artifact.
> I have investigated the issue using JProfiler. The result shows an issue in 
> AbstractWagon's transfer( Resource resource, InputStream input, OutputStream 
> output, int requestType, long maxSize ) method used for remote artifacts and 
> the same issue in AbstractHttpClientWagon#writeTo(OutputStream).
> Here, the input stream is read in a loop using a 4 Kb buffer. Whenever data 
> is received, the received data is pushed to downstream listeners via 
> fireTransferProgress. These listeners (or rather consumers) perform expensive 
> tasks.
> Now, the underlying InputStream implementation used in transfer will return 
> calls to read(buffer, offset, length) as soon as *some* data is available. 
> That is, fireTransferProgress may well be invoked with an average number of 
> bytes less than half the buffer capacity (this varies with the underlying 
> network and hardware architecture). Consequently, fireTransferProgress is 
> invoked *millions of times* for large files. As this is a blocking operation, 
> the time spent in fireTransferProgress dominates and drastically slows down 
> the transfers by at least one order of magnitude. 
> !wagon-issue.png! 
> In our case, we found download speed reduced from a theoretical optimum of 
> ~80 seconds to to more than 3200 seconds.
> From an architectural perspective, I would not want to make the consumers / 
> listeners invoked via fireTransferProgress aware of their potential impact on 
> download speed, but rather refactor the transfer method such that it uses a 
> buffer strategy reducing the the number of fireTransferProgress invocations. 
> This should be done with regard to the expected file size of the transfer, 
> such that fireTransferProgress is invoked often enough but not to frequent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-537) Maven transfer speed of large artifacts is slow due to unsuitable buffer strategy

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729287#comment-16729287
 ] 

Dan Tran commented on WAGON-537:


sorry about the noise, I am able to cook up a multi SCM freeslyle job to build 
both wagon and maven

> Maven transfer speed of large artifacts is slow due to unsuitable buffer 
> strategy
> -
>
> Key: WAGON-537
> URL: https://issues.apache.org/jira/browse/WAGON-537
> Project: Maven Wagon
>  Issue Type: Improvement
>  Components: wagon-http, wagon-provider-api
>Affects Versions: 3.2.0
> Environment: Windows 10, JDK 1.8, Nexus  Artifact store > 100MB/s 
> network connection.
>Reporter: Olaf Otto
>Assignee: Michael Osipov
>Priority: Major
>  Labels: perfomance
> Fix For: 3.3.0
>
> Attachments: wagon-issue.png
>
>
> We are using maven for build process automation with docker. This sometimes 
> involves uploading and downloading artifacts with a few gigabytes in size. 
> Here, maven's transfer speed is consistently and reproducibly slow. For 
> instance, an artifact with 7,5 GB in size took almost two hours to transfer 
> in spite of a 100 MB/s connection with respective reproducible download speed 
> from the remote nexus artifact repository when using a browser to download. 
> The same is true when uploding such an artifact.
> I have investigated the issue using JProfiler. The result shows an issue in 
> AbstractWagon's transfer( Resource resource, InputStream input, OutputStream 
> output, int requestType, long maxSize ) method used for remote artifacts and 
> the same issue in AbstractHttpClientWagon#writeTo(OutputStream).
> Here, the input stream is read in a loop using a 4 Kb buffer. Whenever data 
> is received, the received data is pushed to downstream listeners via 
> fireTransferProgress. These listeners (or rather consumers) perform expensive 
> tasks.
> Now, the underlying InputStream implementation used in transfer will return 
> calls to read(buffer, offset, length) as soon as *some* data is available. 
> That is, fireTransferProgress may well be invoked with an average number of 
> bytes less than half the buffer capacity (this varies with the underlying 
> network and hardware architecture). Consequently, fireTransferProgress is 
> invoked *millions of times* for large files. As this is a blocking operation, 
> the time spent in fireTransferProgress dominates and drastically slows down 
> the transfers by at least one order of magnitude. 
> !wagon-issue.png! 
> In our case, we found download speed reduced from a theoretical optimum of 
> ~80 seconds to to more than 3200 seconds.
> From an architectural perspective, I would not want to make the consumers / 
> listeners invoked via fireTransferProgress aware of their potential impact on 
> download speed, but rather refactor the transfer method such that it uses a 
> buffer strategy reducing the the number of fireTransferProgress invocations. 
> This should be done with regard to the expected file size of the transfer, 
> such that fireTransferProgress is invoked often enough but not to frequent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-537) Maven transfer speed of large artifacts is slow due to unsuitable buffer strategy

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16729276#comment-16729276
 ] 

Dan Tran commented on WAGON-537:


Could you deploy latest wagon snapshot to repository.apache.org?  This is much 
easier for me to build apache-maven at our Jenkins with  
-DwagonVersion=3.3.0-SNAPSHOT.



> Maven transfer speed of large artifacts is slow due to unsuitable buffer 
> strategy
> -
>
> Key: WAGON-537
> URL: https://issues.apache.org/jira/browse/WAGON-537
> Project: Maven Wagon
>  Issue Type: Improvement
>  Components: wagon-http, wagon-provider-api
>Affects Versions: 3.2.0
> Environment: Windows 10, JDK 1.8, Nexus  Artifact store > 100MB/s 
> network connection.
>Reporter: Olaf Otto
>Assignee: Michael Osipov
>Priority: Major
>  Labels: perfomance
> Fix For: 3.3.0
>
> Attachments: wagon-issue.png
>
>
> We are using maven for build process automation with docker. This sometimes 
> involves uploading and downloading artifacts with a few gigabytes in size. 
> Here, maven's transfer speed is consistently and reproducibly slow. For 
> instance, an artifact with 7,5 GB in size took almost two hours to transfer 
> in spite of a 100 MB/s connection with respective reproducible download speed 
> from the remote nexus artifact repository when using a browser to download. 
> The same is true when uploding such an artifact.
> I have investigated the issue using JProfiler. The result shows an issue in 
> AbstractWagon's transfer( Resource resource, InputStream input, OutputStream 
> output, int requestType, long maxSize ) method used for remote artifacts and 
> the same issue in AbstractHttpClientWagon#writeTo(OutputStream).
> Here, the input stream is read in a loop using a 4 Kb buffer. Whenever data 
> is received, the received data is pushed to downstream listeners via 
> fireTransferProgress. These listeners (or rather consumers) perform expensive 
> tasks.
> Now, the underlying InputStream implementation used in transfer will return 
> calls to read(buffer, offset, length) as soon as *some* data is available. 
> That is, fireTransferProgress may well be invoked with an average number of 
> bytes less than half the buffer capacity (this varies with the underlying 
> network and hardware architecture). Consequently, fireTransferProgress is 
> invoked *millions of times* for large files. As this is a blocking operation, 
> the time spent in fireTransferProgress dominates and drastically slows down 
> the transfers by at least one order of magnitude. 
> !wagon-issue.png! 
> In our case, we found download speed reduced from a theoretical optimum of 
> ~80 seconds to to more than 3200 seconds.
> From an architectural perspective, I would not want to make the consumers / 
> listeners invoked via fireTransferProgress aware of their potential impact on 
> download speed, but rather refactor the transfer method such that it uses a 
> buffer strategy reducing the the number of fireTransferProgress invocations. 
> This should be done with regard to the expected file size of the transfer, 
> such that fireTransferProgress is invoked often enough but not to frequent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-537) Maven transfer speed of large artifacts is slow due to unsuitable buffer strategy

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728872#comment-16728872
 ] 

Dan Tran commented on WAGON-537:


I think not able to use dependency as snapshot is a setback.  but i am looking 
forward to test maven 3.6.1-SNAPSHOT with wagon 3.3.0

> Maven transfer speed of large artifacts is slow due to unsuitable buffer 
> strategy
> -
>
> Key: WAGON-537
> URL: https://issues.apache.org/jira/browse/WAGON-537
> Project: Maven Wagon
>  Issue Type: Improvement
>  Components: wagon-http, wagon-provider-api
>Affects Versions: 3.2.0
> Environment: Windows 10, JDK 1.8, Nexus  Artifact store > 100MB/s 
> network connection.
>Reporter: Olaf Otto
>Assignee: Michael Osipov
>Priority: Major
>  Labels: perfomance
> Fix For: 3.3.0
>
> Attachments: wagon-issue.png
>
>
> We are using maven for build process automation with docker. This sometimes 
> involves uploading and downloading artifacts with a few gigabytes in size. 
> Here, maven's transfer speed is consistently and reproducibly slow. For 
> instance, an artifact with 7,5 GB in size took almost two hours to transfer 
> in spite of a 100 MB/s connection with respective reproducible download speed 
> from the remote nexus artifact repository when using a browser to download. 
> The same is true when uploding such an artifact.
> I have investigated the issue using JProfiler. The result shows an issue in 
> AbstractWagon's transfer( Resource resource, InputStream input, OutputStream 
> output, int requestType, long maxSize ) method used for remote artifacts and 
> the same issue in AbstractHttpClientWagon#writeTo(OutputStream).
> Here, the input stream is read in a loop using a 4 Kb buffer. Whenever data 
> is received, the received data is pushed to downstream listeners via 
> fireTransferProgress. These listeners (or rather consumers) perform expensive 
> tasks.
> Now, the underlying InputStream implementation used in transfer will return 
> calls to read(buffer, offset, length) as soon as *some* data is available. 
> That is, fireTransferProgress may well be invoked with an average number of 
> bytes less than half the buffer capacity (this varies with the underlying 
> network and hardware architecture). Consequently, fireTransferProgress is 
> invoked *millions of times* for large files. As this is a blocking operation, 
> the time spent in fireTransferProgress dominates and drastically slows down 
> the transfers by at least one order of magnitude. 
> !wagon-issue.png! 
> In our case, we found download speed reduced from a theoretical optimum of 
> ~80 seconds to to more than 3200 seconds.
> From an architectural perspective, I would not want to make the consumers / 
> listeners invoked via fireTransferProgress aware of their potential impact on 
> download speed, but rather refactor the transfer method such that it uses a 
> buffer strategy reducing the the number of fireTransferProgress invocations. 
> This should be done with regard to the expected file size of the transfer, 
> such that fireTransferProgress is invoked often enough but not to frequent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-537) Maven transfer speed of large artifacts is slow due to unsuitable buffer strategy

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728837#comment-16728837
 ] 

Dan Tran commented on WAGON-537:


[~michael-o] could you hook up latest wagon snapshot with maven 3.6.1-SNAPSHOT, 
love to test this out early

> Maven transfer speed of large artifacts is slow due to unsuitable buffer 
> strategy
> -
>
> Key: WAGON-537
> URL: https://issues.apache.org/jira/browse/WAGON-537
> Project: Maven Wagon
>  Issue Type: Improvement
>  Components: wagon-http, wagon-provider-api
>Affects Versions: 3.2.0
> Environment: Windows 10, JDK 1.8, Nexus  Artifact store > 100MB/s 
> network connection.
>Reporter: Olaf Otto
>Assignee: Michael Osipov
>Priority: Major
>  Labels: perfomance
> Fix For: 3.3.0
>
> Attachments: wagon-issue.png
>
>
> We are using maven for build process automation with docker. This sometimes 
> involves uploading and downloading artifacts with a few gigabytes in size. 
> Here, maven's transfer speed is consistently and reproducibly slow. For 
> instance, an artifact with 7,5 GB in size took almost two hours to transfer 
> in spite of a 100 MB/s connection with respective reproducible download speed 
> from the remote nexus artifact repository when using a browser to download. 
> The same is true when uploding such an artifact.
> I have investigated the issue using JProfiler. The result shows an issue in 
> AbstractWagon's transfer( Resource resource, InputStream input, OutputStream 
> output, int requestType, long maxSize ) method used for remote artifacts and 
> the same issue in AbstractHttpClientWagon#writeTo(OutputStream).
> Here, the input stream is read in a loop using a 4 Kb buffer. Whenever data 
> is received, the received data is pushed to downstream listeners via 
> fireTransferProgress. These listeners (or rather consumers) perform expensive 
> tasks.
> Now, the underlying InputStream implementation used in transfer will return 
> calls to read(buffer, offset, length) as soon as *some* data is available. 
> That is, fireTransferProgress may well be invoked with an average number of 
> bytes less than half the buffer capacity (this varies with the underlying 
> network and hardware architecture). Consequently, fireTransferProgress is 
> invoked *millions of times* for large files. As this is a blocking operation, 
> the time spent in fireTransferProgress dominates and drastically slows down 
> the transfers by at least one order of magnitude. 
> !wagon-issue.png! 
> In our case, we found download speed reduced from a theoretical optimum of 
> ~80 seconds to to more than 3200 seconds.
> From an architectural perspective, I would not want to make the consumers / 
> listeners invoked via fireTransferProgress aware of their potential impact on 
> download speed, but rather refactor the transfer method such that it uses a 
> buffer strategy reducing the the number of fireTransferProgress invocations. 
> This should be done with regard to the expected file size of the transfer, 
> such that fireTransferProgress is invoked often enough but not to frequent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MASSEMBLY-766) OOM with 1CPU

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MASSEMBLY-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728087#comment-16728087
 ] 

Dan Tran commented on MASSEMBLY-766:


Let's close this ticket since it is no longer practical

> OOM with 1CPU
> -
>
> Key: MASSEMBLY-766
> URL: https://issues.apache.org/jira/browse/MASSEMBLY-766
> Project: Maven Assembly Plugin
>  Issue Type: Bug
>  Components: maven-archiver
>Affects Versions: 2.5.4
> Environment: jdk 8, sles 11.3, 1G heap build large assemblies 100M +
>Reporter: Dan Tran
>Priority: Major
>
> Just happen to run on 1CPU VM, and VM crash on OOM
> sample errors
> #
> # There is insufficient memory for the Java Runtime Environment to continue.
> # Native memory allocation (malloc) failed to allocate 82576 bytes for 
> Chunk::new
> # An error report file with more information is saved as:
> # /var/lib/jenkins/jobs/-all-bugId-template/workspace/hs_err_pid26261.log
> #
> # Compiler replay data is saved as:
> # /var/lib/jenkins/jobs/xxx-all-bugId-template/workspace/replay_pid26261.log
> Build step 'Invoke top-level Maven targets' marked build as failure
> another one
> [INFO] --- maven-assembly-plugin:2.5.4:single (uber-jar) @ xxx-configmgr ---
> mmap failed for CEN and END part of zip file
> java.lang.OutOfMemoryError
>   at java.util.zip.ZipFile.open(Native Method)
> 
> org.codehaus.plexus.archiver.zip.PlexusIoZipFileResourceCollection$1.getResource(PlexusIoZipFileResourceCollection.java:69)
>   at 
> org.codehaus.plexus.archiver.zip.PlexusIoZipFileResourceCollection$ZipFileResourceIterator$1.getURL(PlexusIoZipFileResourceCollection.java:125)
>   at 
> org.codehaus.plexus.components.io.resources.PlexusIoURLResource.getContents(PlexusIoURLResource.java:37)
>   at 
> org.codehaus.plexus.archiver.ArchiveEntry.getInputStream(ArchiveEntry.java:126)
>   at 
> org.codehaus.plexus.archiver.zip.AbstractZipArchiver.zipFile(AbstractZipArchiver.java:514)
>   at 
> org.codehaus.plexus.archiver.zip.AbstractZipArchiver.addResources(AbstractZipArchiver.java:370)
>   at 
> org.codehaus.plexus.archiver.zip.AbstractZipArchiver.createArchiveMain(AbstractZipArchiver.java:326)
>   at 
> org.codehaus.plexus.archiver.zip.AbstractZipArchiver.execute(AbstractZipArchiver.java:227)
>   at 
> org.codehaus.plexus.archiver.AbstractArchiver.createArchive(AbstractArchiver.java:990)
>   at 
> org.apache.maven.plugin.assembly.archive.archiver.AssemblyProxyArchiver.createArchive(AssemblyProxyArchiver.java:437)
>   at 
> org.apache.maven.plugin.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:181)
>   at 
> org.apache.maven.plugin.assembly.mojos.AbstractAssemblyMojo.execute(AbstractAssemblyMojo.java:484)
>   at 
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MRESOLVER-64) restore setLoggerFactory(...) API deleted from version 1.3.0

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718504#comment-16718504
 ] 

Dan Tran commented on MRESOLVER-64:
---

Are we all set? 

> restore setLoggerFactory(...) API deleted from version 1.3.0
> 
>
> Key: MRESOLVER-64
> URL: https://issues.apache.org/jira/browse/MRESOLVER-64
> Project: Maven Resolver
>  Issue Type: Wish
>  Components: resolver
>Affects Versions: 1.3.0, 1.3.1
>Reporter: Hervé Boutemy
>Assignee: Hervé Boutemy
>Priority: Major
> Fix For: 1.3.2
>
>
> during work on MRESOLVER-36 to switch from LoggerFactory to direct slf4j, 
> {{setLoggerFactory(...)}} methods were removed since they are not strictly 
> required now
> 2 cases of removal (in {{DefaultArtifactResolver}} and 
> {{DefaultRepositorySystem}}) broke Intellij IDEA Maven integration
> adding back the method with empty implementation in the 2 classes, marking 
> them deprecated, would restore strict API compatibility for older IDEA 
> versions without much complexity



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1546) JUnit 5 runner does not honor JUnit 5 display names

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716519#comment-16716519
 ] 

Dan Tran commented on SUREFIRE-1546:


is this scheduled for 3.0 surefire?

> JUnit 5 runner does not honor JUnit 5 display names
> ---
>
> Key: SUREFIRE-1546
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1546
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: JUnit 5.x support
>Affects Versions: 2.22.0
>Reporter: Romain Manni-Bucau
>Assignee: Christian Stein
>Priority: Major
>  Labels: junit5
>
> JUnit 5 runner should respect the test @DisplayName instead of displaying the 
> classname if any is defined. Seems last release doesn't support that feature 
> of JUnit 5 making the console output and reports not the expected ones.
>  
> Origin: https://github.com/junit-team/junit5/issues/990



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (MRESOLVER-64) restore setLoggerFactory(...) API deleted from version 1.3.0

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714222#comment-16714222
 ] 

Dan Tran edited comment on MRESOLVER-64 at 12/10/18 4:03 AM:
-

I am able to reproduce this issue with instructions from Rostislav Krasny and 
[~slachiewicz]

The issue is fixed then hook it up with my local maven 3.6.1-SNAPShOT + new 
version of  resolver


was (Author: dantran):
I am able to reproduce this issue with instructions from Rostislav Krasny and 
[~slachiewicz], then hook it up with my local maven 3.6.1-SNAPShOT + new 
resolver

> restore setLoggerFactory(...) API deleted from version 1.3.0
> 
>
> Key: MRESOLVER-64
> URL: https://issues.apache.org/jira/browse/MRESOLVER-64
> Project: Maven Resolver
>  Issue Type: Wish
>  Components: resolver
>Affects Versions: 1.3.0, 1.3.1
>Reporter: Hervé Boutemy
>Assignee: Hervé Boutemy
>Priority: Major
> Fix For: 1.3.2
>
>
> during work on MRESOLVER-36 to switch from LoggerFactory to direct slf4j, 
> {{setLoggerFactory(...)}} methods were removed since they are not strictly 
> required now
> 2 cases of removal (in {{DefaultArtifactResolver}} and 
> {{DefaultRepositorySystem}}) broke Intellij IDEA Maven integration
> adding back the method with empty implementation in the 2 classes, marking 
> them deprecated, would restore strict API compatibility for older IDEA 
> versions without much complexity



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MRESOLVER-64) restore setLoggerFactory(...) API deleted from version 1.3.0

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714222#comment-16714222
 ] 

Dan Tran commented on MRESOLVER-64:
---

I am able to reproduce this issue with instructions from Rostislav Krasny and 
[~slachiewicz], then hook it up with my local maven 3.6.1-SNAPShOT + new 
resolver

> restore setLoggerFactory(...) API deleted from version 1.3.0
> 
>
> Key: MRESOLVER-64
> URL: https://issues.apache.org/jira/browse/MRESOLVER-64
> Project: Maven Resolver
>  Issue Type: Wish
>  Components: resolver
>Affects Versions: 1.3.0, 1.3.1
>Reporter: Hervé Boutemy
>Assignee: Hervé Boutemy
>Priority: Major
> Fix For: 1.3.2
>
>
> during work on MRESOLVER-36 to switch from LoggerFactory to direct slf4j, 
> {{setLoggerFactory(...)}} methods were removed since they are not strictly 
> required now
> 2 cases of removal (in {{DefaultArtifactResolver}} and 
> {{DefaultRepositorySystem}}) broke Intellij IDEA Maven integration
> adding back the method with empty implementation in the 2 classes, marking 
> them deprecated, would restore strict API compatibility for older IDEA 
> versions without much complexity



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MRESOLVER-64) restore setLoggerFactory(...) API deleted from version 1.3.0

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714161#comment-16714161
 ] 

Dan Tran commented on MRESOLVER-64:
---

[~slachiewicz]  Could you help to give it a try.  I totally new to IntelliJ, so 
far no luck in terms of reproducing the issue :(

> restore setLoggerFactory(...) API deleted from version 1.3.0
> 
>
> Key: MRESOLVER-64
> URL: https://issues.apache.org/jira/browse/MRESOLVER-64
> Project: Maven Resolver
>  Issue Type: Wish
>  Components: resolver
>Affects Versions: 1.3.0, 1.3.1
>Reporter: Hervé Boutemy
>Assignee: Hervé Boutemy
>Priority: Major
> Fix For: 1.3.2
>
>
> during work on MRESOLVER-36 to switch from LoggerFactory to direct slf4j, 
> {{setLoggerFactory(...)}} methods were removed since they are not strictly 
> required now
> 2 cases of removal (in {{DefaultArtifactResolver}} and 
> {{DefaultRepositorySystem}}) broke Intellij IDEA Maven integration
> adding back the method with empty implementation in the 2 classes, marking 
> them deprecated, would restore strict API compatibility for older IDEA 
> versions without much complexity



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MRESOLVER-64) restore setLoggerFactory(...) API deleted from version 1.3.0

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MRESOLVER-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713887#comment-16713887
 ] 

Dan Tran commented on MRESOLVER-64:
---

I think we should hook this up with Maven 3.6.1-SNAPSHOT and test out at 
IntelliJ.  If it works out, let's release 3.6.1. 

I am blocked on maven 3.6 upgrade in order to pickup the CI/CD fix



> restore setLoggerFactory(...) API deleted from version 1.3.0
> 
>
> Key: MRESOLVER-64
> URL: https://issues.apache.org/jira/browse/MRESOLVER-64
> Project: Maven Resolver
>  Issue Type: Wish
>  Components: resolver
>Affects Versions: 1.3.0, 1.3.1
>Reporter: Hervé Boutemy
>Priority: Major
>
> during work on MRESOLVER-36 to switch from LoggerFactory to direct slf4j, 2 
> setLoggerFactory(...) methods were removed since they are not strictly 
> required now
> this broke Intellij IDEA Maven integration
> adding back these methods with empty implementation, marking them deprecated, 
> would restore strict API compatibility without much complexity



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (WAGON-537) Maven transfer speed of large artifacts is slow due to unsuitable buffer strategy

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/WAGON-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682745#comment-16682745
 ] 

Dan Tran commented on WAGON-537:


This is Xmas present ... early. Many thanks 

> Maven transfer speed of large artifacts is slow due to unsuitable buffer 
> strategy
> -
>
> Key: WAGON-537
> URL: https://issues.apache.org/jira/browse/WAGON-537
> Project: Maven Wagon
>  Issue Type: Improvement
>  Components: wagon-http, wagon-provider-api
>Affects Versions: 3.2.0
> Environment: Windows 10, JDK 1.8, Nexus  Artifact store > 100MB/s 
> network connection.
>Reporter: Olaf Otto
>Assignee: Michael Osipov
>Priority: Major
>  Labels: perfomance
> Fix For: 3.2.1
>
> Attachments: wagon-issue.png
>
>
> We are using maven for build process automation with docker. This sometimes 
> involves uploading and downloading artifacts with a few gigabytes in size. 
> Here, maven's transfer speed is consistently and reproducibly slow. For 
> instance, an artifact with 7,5 GB in size took almost two hours to transfer 
> in spite of a 100 MB/s connection with respective reproducible download speed 
> from the remote nexus artifact repository when using a browser to download. 
> The same is true when uploding such an artifact.
> I have investigated the issue using JProfiler. The result shows an issue in 
> AbstractWagon's transfer( Resource resource, InputStream input, OutputStream 
> output, int requestType, long maxSize ) method used for remote artifacts and 
> the same issue in AbstractHttpClientWagon#writeTo(OutputStream).
> Here, the input stream is read in a loop using a 4 Kb buffer. Whenever data 
> is received, the received data is pushed to downstream listeners via 
> fireTransferProgress. These listeners (or rather consumers) perform expensive 
> tasks.
> Now, the underlying InputStream implementation used in transfer will return 
> calls to read(buffer, offset, length) as soon as *some* data is available. 
> That is, fireTransferProgress may well be invoked with an average number of 
> bytes less than half the buffer capacity (this varies with the underlying 
> network and hardware architecture). Consequently, fireTransferProgress is 
> invoked *millions of times* for large files. As this is a blocking operation, 
> the time spent in fireTransferProgress dominates and drastically slows down 
> the transfers by at least one order of magnitude. 
> !wagon-issue.png! 
> In our case, we found download speed reduced from a theoretical optimum of 
> ~80 seconds to to more than 3200 seconds.
> From an architectural perspective, I would not want to make the consumers / 
> listeners invoked via fireTransferProgress aware of their potential impact on 
> download speed, but rather refactor the transfer method such that it uses a 
> buffer strategy reducing the the number of fireTransferProgress invocations. 
> This should be done with regard to the expected file size of the transfer, 
> such that fireTransferProgress is invoked often enough but not to frequent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1359) Corrupted stdin stream in forked JVM 1. See the dump file

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674259#comment-16674259
 ] 

Dan Tran commented on SUREFIRE-1359:


[~MattNelson] I am experiencing the same issue.  maven 3.6 java 1.8.0-171.  How 
did you implement the workaround?

> Corrupted stdin stream in forked JVM 1. See the dump file
> -
>
> Key: SUREFIRE-1359
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1359
> Project: Maven Surefire
>  Issue Type: Bug
>  Components: Maven Surefire Plugin
>Affects Versions: 2.20
> Environment: JDK 1.8.0_121
> maven version 3.5.0
>Reporter: Brian Oxley
>Assignee: Tibor Digana
>Priority: Major
> Attachments: 2017-04-13T10-56-35_635-jvmRun1.dumpstream, 
> 2017-04-13T10-56-36_592.dumpstream, TEST-hm.binkley.labs.GitInfoIT.xml, 
> TEST-hm.binkley.labs.GreetingControllerIT.xml, failsafe-summary.xml, 
> hm.binkley.labs.GitInfoIT.txt, hm.binkley.labs.GreetingControllerIT.txt
>
>
> Surefire 2.20 complains "Corrupted stdin stream in forked JVM 1. See the dump 
> file".  Version 2.19.1 does not, and testing completes successfully.
> Project is here: https://github.com/binkley/sproingk.
> Run with "mvn clean verify".
> Same behavior on Windows 10 (Oracle JDK), MacOS and the Linux subsystem on 
> Windows.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MNG-6412) Exceeding project discovery time when using CI friendly versions

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/MNG-6412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1555#comment-1555
 ] 

Dan Tran commented on MNG-6412:
---

oh mine!! this is fixed on my 300+ maven modules project using maven 3.6.  
Thank you!!!

> Exceeding project discovery time when using CI friendly versions
> 
>
> Key: MNG-6412
> URL: https://issues.apache.org/jira/browse/MNG-6412
> Project: Maven
>  Issue Type: Bug
>Affects Versions: 3.5.3
>Reporter: Christoph Amshoff
>Assignee: Sylwester Lachiewicz
>Priority: Critical
> Fix For: 3.6.0
>
> Attachments: ci-version-validate.log, fix-version-validate.log, 
> test-ci-project-scanning.zip
>
>
> We are switching a larger project to CI friendly versions 
> ([https://maven.apache.org/maven-ci-friendly.html]), using properties 
> _${revision}_ (fix) and _${changelist}_ (maps to the CI build number) to 
> create the actual version. However, doing so, the project discovery time 
> (right after Maven says “Scanning for projects...”) increases dramatically 
> from few seconds to more than 3 minutes, and we had to double the maximum 
> heap in order to avoid “GC overhead limit exceeded” exception.
> The project root POM looks like this:
> {noformat}
>   test
>   parent
>   ${revision}-${changelist}
>   pom
>   ...
>   
> 1.0.0
>     UNDEFINED
>    {noformat}
> and is built by passing “-Dchangelist=123” to the Maven process for CI build 
> 123, for instance.
> In debug output, we can see that for each POM of the reactor there are 
> messages like these, repeatedly for every single parent POM up the complete 
> hierarchy:
> {noformat}
> [DEBUG] Extension realms for project test:grandchild1-1:jar:1.0.0-123: (none)
> [DEBUG] Looking up lifecycle mappings for packaging jar from 
> ClassRealm[plexus.core, parent: null]
> [DEBUG] Extension realms for project test:child1:pom:1.0.0-123: (none)
> [DEBUG] Looking up lifecycle mappings for packaging pom from 
> ClassRealm[plexus.core, parent: null]
> [DEBUG] Extension realms for project test:parent:pom:1.0.0-123: (none)
> [DEBUG] Looking up lifecycle mappings for packaging pom from 
> ClassRealm[plexus.core, parent: null]
> {noformat}
> When using fix version string (release or SNAPSHOT), these messages are only 
> present for the reactor POMs, but not their parents, grand-parents, etc. up 
> to the root POM.
> Looking up the lifecycle mapping for the same parent POMs over and over again 
> seems wrong to me, and is probably the cause of high memory consumption. 
> Altogether, there are more than 2,800 messages "Looking up lifecycle 
> mappings..." in the logs for about 350 subprojects built in the reactor!
> See attached test projects and logs for fix version vs. CI friendly versions. 
> The test project structure is like this:
> {noformat}
> parent
>   child1
>     grandchild1-1
>     grandchild1-2
>   child2
>     grandchild2-1
>     grandchild2-2{noformat}
> If you compare both log files, you'll see the difference in project resolving.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (SUREFIRE-1567) Surefire Plugin should evaluate Junit 5 @DisplayName

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SUREFIRE-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran closed SUREFIRE-1567.
--
Resolution: Duplicate

> Surefire Plugin should evaluate Junit 5 @DisplayName
> 
>
> Key: SUREFIRE-1567
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1567
> Project: Maven Surefire
>  Issue Type: Improvement
>Affects Versions: 2.22.0
>Reporter: Dan Tran
>Priority: Major
>
> Details discussed at https://github.com/junit-team/junit5/issues/990
> Per junit5 team, surefire is the right place to add this enhancement



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (SUREFIRE-1567) Surefire Plugin should evaluate Junit 5 @DisplayName

[Dan Tran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SUREFIRE-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Tran updated SUREFIRE-1567:
---
Description: 
Details discussed at https://github.com/junit-team/junit5/issues/990

Per junit5 team, surefire is the right place to add this enhancement

  was:
Details discussed at https://github.com/junit-team/junit5/issues/990

Per junit5 team, surefire should fix this


> Surefire Plugin should evaluate Junit 5 @DisplayName
> 
>
> Key: SUREFIRE-1567
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1567
> Project: Maven Surefire
>  Issue Type: Improvement
>Affects Versions: 2.22.0
>Reporter: Dan Tran
>Priority: Major
>
> Details discussed at https://github.com/junit-team/junit5/issues/990
> Per junit5 team, surefire is the right place to add this enhancement



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (SUREFIRE-1567) Surefire Plugin should evaluate Junit 5 @DisplayName

[Dan Tran (JIRA)]

Dan Tran created SUREFIRE-1567:
--

 Summary: Surefire Plugin should evaluate Junit 5 @DisplayName
 Key: SUREFIRE-1567
 URL: https://issues.apache.org/jira/browse/SUREFIRE-1567
 Project: Maven Surefire
  Issue Type: Improvement
Affects Versions: 2.22.0
Reporter: Dan Tran


Details discussed at https://github.com/junit-team/junit5/issues/990

Per junit5 team, surefire should fix this



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (SUREFIRE-1527) Surefire does not discover junit5 test case when testng is in the classpath

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529343#comment-16529343
 ] 

Dan Tran commented on SUREFIRE-1527:


Ouch, my simple naive solution breaks any TestNG test cases without testng.xml 
file. It basically skips those tests

> Surefire does not discover junit5 test case when testng is in the classpath
> ---
>
> Key: SUREFIRE-1527
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1527
> Project: Maven Surefire
>  Issue Type: Improvement
>  Components: JUnit 5.x support, TestNG support
>Affects Versions: 2.22.0
>Reporter: Dan Tran
>Priority: Major
>
> My maven module uses junit5 but also has testng in its test classpath and my 
> surefire does not specify any testng.xml files
> Looks like surefire only look for testng test cases



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (SUREFIRE-1527) Surefire does not discover junit5 test case when testng is in the classpath

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529019#comment-16529019
 ] 

Dan Tran edited comment on SUREFIRE-1527 at 7/1/18 4:51 PM:


Here is proposed fix at TestNgProviderInfo
{code:java}
  @Override
  public boolean isApplicable()
  {
      return testNgArtifact != null && hasSuiteXmlFiles();
  }{code}

Another workaround at surefire configuration to force surefire not able to 
detect the present of testng
{code:java}

org.testng:testng-ignore 
{code}


was (Author: dantran):
Here is proposed fix at TestNgProviderInfo
{code:java}
  @Override
  public boolean isApplicable()
  {
      return testNgArtifact != null && hasSuiteXmlFiles();
  }{code}

Another workaround to force surefire not able to detect the present of testng
{code:java}

org.testng:testng-ignore 
{code}

> Surefire does not discover junit5 test case when testng is in the classpath
> ---
>
> Key: SUREFIRE-1527
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1527
> Project: Maven Surefire
>  Issue Type: Improvement
>  Components: JUnit 5.x support, TestNG support
>Affects Versions: 2.22.0
>Reporter: Dan Tran
>Priority: Major
>
> My maven module uses junit5 but also has testng in its test classpath and my 
> surefire does not specify any testng.xml files
> Looks like surefire only look for testng test cases



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (SUREFIRE-1527) Surefire does not discover junit5 test case when testng is in the classpath

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529019#comment-16529019
 ] 

Dan Tran edited comment on SUREFIRE-1527 at 7/1/18 9:05 AM:


Here is proposed fix at TestNgProviderInfo
{code:java}
  @Override
  public boolean isApplicable()
  {
      return testNgArtifact != null && hasSuiteXmlFiles();
  }{code}

Another workaround to force surefire not able to detect the present of testng
{code:java}

org.testng:testng-ignore 
{code}


was (Author: dantran):
Here is proposed fix at TestNgProviderInfo
{code:java}
    @Override
    public boolean isApplicable()
    {
        return testNgArtifact != null && hasSuiteXmlFiles();
    }{code}

Another workaround to force surefire not able to detect the present of testng
{code:java}

org.testng:testng-ignore 
{code}

> Surefire does not discover junit5 test case when testng is in the classpath
> ---
>
> Key: SUREFIRE-1527
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1527
> Project: Maven Surefire
>  Issue Type: Improvement
>  Components: JUnit 5.x support, TestNG support
>Affects Versions: 2.22.0
>Reporter: Dan Tran
>Priority: Major
>
> My maven module uses junit5 but also has testng in its test classpath and my 
> surefire does not specify any testng.xml files
> Looks like surefire only look for testng test cases



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (SUREFIRE-1527) Surefire does not discover junit5 test case when testng is in the classpath

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529019#comment-16529019
 ] 

Dan Tran edited comment on SUREFIRE-1527 at 7/1/18 9:04 AM:


Here is proposed fix at TestNgProviderInfo
{code:java}
    @Override
    public boolean isApplicable()
    {
        return testNgArtifact != null && hasSuiteXmlFiles();
    }{code}

Another workaround to force surefire not able to detect the present of testng
{code:java}

org.testng:testng-ignore 
{code}


was (Author: dantran):
Here is proposed fix at TestNgProviderInfo
{code:java}
    @Override
    public boolean isApplicable()
    {
        return testNgArtifact != null && hasSuiteXmlFiles();
    }{code}

> Surefire does not discover junit5 test case when testng is in the classpath
> ---
>
> Key: SUREFIRE-1527
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1527
> Project: Maven Surefire
>  Issue Type: Improvement
>  Components: JUnit 5.x support, TestNG support
>Affects Versions: 2.22.0
>Reporter: Dan Tran
>Priority: Major
>
> My maven module uses junit5 but also has testng in its test classpath and my 
> surefire does not specify any testng.xml files
> Looks like surefire only look for testng test cases



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (SUREFIRE-1527) Surefire does not discover junit5 test case when testng is in the classpath

[Dan Tran (JIRA)]

[ 
https://issues.apache.org/jira/browse/SUREFIRE-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529019#comment-16529019
 ] 

Dan Tran edited comment on SUREFIRE-1527 at 7/1/18 9:04 AM:


Here is proposed fix at TestNgProviderInfo
{code:java}
    @Override
    public boolean isApplicable()
    {
        return testNgArtifact != null && hasSuiteXmlFiles();
    }{code}


was (Author: dantran):
Here is proposed fix at TestNgProviderInfo

{code}

    @Override
    public boolean isApplicable()
    {
        return testNgArtifact != null && hasSuiteXmlFiles();
    }

{code>

 

Another workaround to force surefire not able to detect the present of testng
{code:java}

org.testng:testng-ignore 
{code}

> Surefire does not discover junit5 test case when testng is in the classpath
> ---
>
> Key: SUREFIRE-1527
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1527
> Project: Maven Surefire
>  Issue Type: Improvement
>  Components: JUnit 5.x support, TestNG support
>Affects Versions: 2.22.0
>Reporter: Dan Tran
>Priority: Major
>
> My maven module uses junit5 but also has testng in its test classpath and my 
> surefire does not specify any testng.xml files
> Looks like surefire only look for testng test cases



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)