[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902940#comment-16902940 ] Zakharov Denis commented on MNG-2802: - I'm not 100% sure but i have put Takari Extension into lib/ext folder and continue to get errors like this yet: 15:11:13.552 [INFO] [ war ]- [|http://fobo-build01.corp.mvideo.ru:8082/jenkins/blue/organizations/jenkins/Build%2FBuildWincash/detail/release%2F20.0.0/72/pipeline/#step-424-log-62]15:11:13.617 [WARNING] The POM for **::jar:20.0.0 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details It means that file was changed from another maven process. Mb something changed for years but it doesn't work for me with maven 3.6.1 It will be very appreciate if you add concurrent-safe access to local repository in maven core functionality > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. > *NOTE:* > http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is > a Takari Extension for Apache Maven that introduces a concurrent-safe > local-repository. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707488#comment-16707488 ] Robert Scholte commented on MNG-2802: - [~zakharovdi] Have you read this? bq. http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is a Takari Extension for Apache Maven that introduces a concurrent-safe local-repository. > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. > *NOTE:* > http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is > a Takari Extension for Apache Maven that introduces a concurrent-safe > local-repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706808#comment-16706808 ] Zakharov Denis commented on MNG-2802: - Can someone answer on @erikhakansson questions or merge PR? We have a lot of build failed due to this issue > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. > *NOTE:* > http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is > a Takari Extension for Apache Maven that introduces a concurrent-safe > local-repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656638#comment-16656638 ] ASF GitHub Bot commented on MNG-2802: - erikhakansson edited a comment on issue #49: [MNG-2802] Concurrent-safe access to local Maven repository URL: https://github.com/apache/maven-wagon/pull/49#issuecomment-431325340 Can someone have a new look at this after my comments regarding System.exit? To reiterate, the System.exit isn't run in anything manaded by Surefire and thus shouldn't affect/be affected by surefire behaviour. LazyLockableFileOutputStreamTestDriver is never run by Surefire The test (LazyLockableFileOutputStreamIntegrationTest) launches two separate JVMs to be able to properly isolate file locking behaviour and not having different processes in the same JVM affect each other. I.e, simulating real world behaviour with different instances of Maven accessing the same file. @olamy This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. > *NOTE:* > http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is > a Takari Extension for Apache Maven that introduces a concurrent-safe > local-repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656635#comment-16656635 ] ASF GitHub Bot commented on MNG-2802: - erikhakansson commented on issue #49: [MNG-2802] Concurrent-safe access to local Maven repository URL: https://github.com/apache/maven-wagon/pull/49#issuecomment-431325340 Can someone have a new look at this after my comments regarding System.exit? To reiterate, the System.exit isn't run in anything manaded by Surefire and thus shouldn't affect/be affected by surefire behaviour. LazyLockableFileOutputStreamTestDriver is never run by Surefire The test (LazyLockableFileOutputStreamIntegrationTest) launches two separate JVMs to be able to properly isolate file locking behaviour and not having different processes in the same JVM affect each other. I.e, simulating real world behaviour with different instances of Maven accessing the same file. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. > *NOTE:* > http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is > a Takari Extension for Apache Maven that introduces a concurrent-safe > local-repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[
https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625479#comment-16625479
]
ASF GitHub Bot commented on MNG-2802:
-
erikhakansson commented on a change in pull request #49: [MNG-2802]
Concurrent-safe access to local Maven repository
URL: https://github.com/apache/maven-wagon/pull/49#discussion_r219747705
##
File path:
wagon-providers/wagon-file-test/src/main/java/org/apache/maven/wagon/providers/file/LazyLockableFileOutputStreamTestDriver.java
##
@@ -0,0 +1,75 @@
+package org.apache.maven.wagon.providers.file;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.File;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Test driver for LazyLockableFileOutputStreamIntegrationTest
+ */
+public class LazyLockableFileOutputStreamTestDriver
+{
+public static void main( String[] args ) throws Exception
+{
+File file = new File( args[0] );
+long timeout = Long.parseLong( args[1] );
+long startTime = System.currentTimeMillis();
+OutputStream outputStream = new LazyLockableFileOutputStream( file,
timeout, TimeUnit.SECONDS );
+try
+{
+outputStream.write( 1 );
+}
+catch ( IOException e )
+{
+if ( e.getMessage().equals( "Can't write file, lock " +
file.getAbsolutePath() + ".lck exists" ) )
+{
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 126 );
+// CHECKSTYLE_ON: MagicNumber
+}
+else if ( e.getMessage().equals( "Failed to create lockfile " +
file.getAbsolutePath()
++ ".lck after waiting " + timeout + " seconds. File
already exists." ) )
+{
+long diff = System.currentTimeMillis() - startTime;
+if ( diff < TimeUnit.SECONDS.toMillis( timeout ) )
+{
+throw new Exception( "We were supposed to wait for " +
timeout
++ " seconds, but Exception came early at " + diff
+ " milliseconds." );
+}
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 127 );
Review comment:
@olamy sorry for nagging, but have you had time to think about this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Concurrent-safe access to local Maven repository
>
>
> Key: MNG-2802
> URL: https://issues.apache.org/jira/browse/MNG-2802
> Project: Maven
> Issue Type: Improvement
> Components: Artifacts and Repositories
>Reporter: Stepan Roh
>Assignee: John Casey
>Priority: Major
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is
> multiple Mavens running in parallel may damage contents of local Maven
> repository. It would be a nice improvement, because sharing of local
> repository will lower the need for contacting any other repository. I know
> that Maven proxy can be used, but that adds another layer which may
> unnecessarily stress the machine it runs on.
> *NOTE:*
> http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is
> a Takari Extension for Apache Maven that introduces a concurrent-safe
> local-repository.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[
https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591349#comment-16591349
]
ASF GitHub Bot commented on MNG-2802:
-
erikhakansson commented on a change in pull request #49: [MNG-2802]
Concurrent-safe access to local Maven repository
URL: https://github.com/apache/maven-wagon/pull/49#discussion_r212562988
##
File path:
wagon-providers/wagon-file-test/src/main/java/org/apache/maven/wagon/providers/file/LazyLockableFileOutputStreamTestDriver.java
##
@@ -0,0 +1,75 @@
+package org.apache.maven.wagon.providers.file;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.File;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Test driver for LazyLockableFileOutputStreamIntegrationTest
+ */
+public class LazyLockableFileOutputStreamTestDriver
+{
+public static void main( String[] args ) throws Exception
+{
+File file = new File( args[0] );
+long timeout = Long.parseLong( args[1] );
+long startTime = System.currentTimeMillis();
+OutputStream outputStream = new LazyLockableFileOutputStream( file,
timeout, TimeUnit.SECONDS );
+try
+{
+outputStream.write( 1 );
+}
+catch ( IOException e )
+{
+if ( e.getMessage().equals( "Can't write file, lock " +
file.getAbsolutePath() + ".lck exists" ) )
+{
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 126 );
+// CHECKSTYLE_ON: MagicNumber
+}
+else if ( e.getMessage().equals( "Failed to create lockfile " +
file.getAbsolutePath()
++ ".lck after waiting " + timeout + " seconds. File
already exists." ) )
+{
+long diff = System.currentTimeMillis() - startTime;
+if ( diff < TimeUnit.SECONDS.toMillis( timeout ) )
+{
+throw new Exception( "We were supposed to wait for " +
timeout
++ " seconds, but Exception came early at " + diff
+ " milliseconds." );
+}
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 127 );
Review comment:
ping @olamy
what do you think?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Concurrent-safe access to local Maven repository
>
>
> Key: MNG-2802
> URL: https://issues.apache.org/jira/browse/MNG-2802
> Project: Maven
> Issue Type: Improvement
> Components: Artifacts and Repositories
>Reporter: Stepan Roh
>Assignee: John Casey
>Priority: Major
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is
> multiple Mavens running in parallel may damage contents of local Maven
> repository. It would be a nice improvement, because sharing of local
> repository will lower the need for contacting any other repository. I know
> that Maven proxy can be used, but that adds another layer which may
> unnecessarily stress the machine it runs on.
> *NOTE:*
> http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is
> a Takari Extension for Apache Maven that introduces a concurrent-safe
> local-repository.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[
https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584715#comment-16584715
]
ASF GitHub Bot commented on MNG-2802:
-
erikhakansson commented on a change in pull request #49: [MNG-2802]
Concurrent-safe access to local Maven repository
URL: https://github.com/apache/maven-wagon/pull/49#discussion_r211069493
##
File path:
wagon-providers/wagon-file-test/src/main/java/org/apache/maven/wagon/providers/file/LazyLockableFileOutputStreamTestDriver.java
##
@@ -0,0 +1,75 @@
+package org.apache.maven.wagon.providers.file;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.File;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Test driver for LazyLockableFileOutputStreamIntegrationTest
+ */
+public class LazyLockableFileOutputStreamTestDriver
+{
+public static void main( String[] args ) throws Exception
+{
+File file = new File( args[0] );
+long timeout = Long.parseLong( args[1] );
+long startTime = System.currentTimeMillis();
+OutputStream outputStream = new LazyLockableFileOutputStream( file,
timeout, TimeUnit.SECONDS );
+try
+{
+outputStream.write( 1 );
+}
+catch ( IOException e )
+{
+if ( e.getMessage().equals( "Can't write file, lock " +
file.getAbsolutePath() + ".lck exists" ) )
+{
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 126 );
+// CHECKSTYLE_ON: MagicNumber
+}
+else if ( e.getMessage().equals( "Failed to create lockfile " +
file.getAbsolutePath()
++ ".lck after waiting " + timeout + " seconds. File
already exists." ) )
+{
+long diff = System.currentTimeMillis() - startTime;
+if ( diff < TimeUnit.SECONDS.toMillis( timeout ) )
+{
+throw new Exception( "We were supposed to wait for " +
timeout
++ " seconds, but Exception came early at " + diff
+ " milliseconds." );
+}
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 127 );
Review comment:
Ah, I was not aware of that! But since the
LazyLockableFileOutputStreamTestDriver isn't directly managed by surefire, but
instead launched in a separate JVM, shouldn't it be quite safe anyway?
It's a very convenient way of reporting results out to the actual test
without having to read the output stream. Which would be the alternative
solution.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Concurrent-safe access to local Maven repository
>
>
> Key: MNG-2802
> URL: https://issues.apache.org/jira/browse/MNG-2802
> Project: Maven
> Issue Type: Improvement
> Components: Artifacts and Repositories
>Reporter: Stepan Roh
>Assignee: John Casey
>Priority: Major
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is
> multiple Mavens running in parallel may damage contents of local Maven
> repository. It would be a nice improvement, because sharing of local
> repository will lower the need for contacting any other repository. I know
> that Maven proxy can be used, but that adds another layer which may
> unnecessarily stress the machine it runs on.
> *NOTE:*
> http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is
> a Takari Extension for Apache Maven that introduces a concurrent-safe
> local-repository.
--
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[
https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584525#comment-16584525
]
ASF GitHub Bot commented on MNG-2802:
-
olamy commented on a change in pull request #49: [MNG-2802] Concurrent-safe
access to local Maven repository
URL: https://github.com/apache/maven-wagon/pull/49#discussion_r211057067
##
File path:
wagon-providers/wagon-file-test/src/main/java/org/apache/maven/wagon/providers/file/LazyLockableFileOutputStreamTestDriver.java
##
@@ -0,0 +1,75 @@
+package org.apache.maven.wagon.providers.file;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.File;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Test driver for LazyLockableFileOutputStreamIntegrationTest
+ */
+public class LazyLockableFileOutputStreamTestDriver
+{
+public static void main( String[] args ) throws Exception
+{
+File file = new File( args[0] );
+long timeout = Long.parseLong( args[1] );
+long startTime = System.currentTimeMillis();
+OutputStream outputStream = new LazyLockableFileOutputStream( file,
timeout, TimeUnit.SECONDS );
+try
+{
+outputStream.write( 1 );
+}
+catch ( IOException e )
+{
+if ( e.getMessage().equals( "Can't write file, lock " +
file.getAbsolutePath() + ".lck exists" ) )
+{
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 126 );
Review comment:
please avoid System.exit this can fail to correctly report failures with
surefire
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Concurrent-safe access to local Maven repository
>
>
> Key: MNG-2802
> URL: https://issues.apache.org/jira/browse/MNG-2802
> Project: Maven
> Issue Type: Improvement
> Components: Artifacts and Repositories
>Reporter: Stepan Roh
>Assignee: John Casey
>Priority: Major
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is
> multiple Mavens running in parallel may damage contents of local Maven
> repository. It would be a nice improvement, because sharing of local
> repository will lower the need for contacting any other repository. I know
> that Maven proxy can be used, but that adds another layer which may
> unnecessarily stress the machine it runs on.
> *NOTE:*
> http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is
> a Takari Extension for Apache Maven that introduces a concurrent-safe
> local-repository.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[
https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584521#comment-16584521
]
ASF GitHub Bot commented on MNG-2802:
-
olamy commented on a change in pull request #49: [MNG-2802] Concurrent-safe
access to local Maven repository
URL: https://github.com/apache/maven-wagon/pull/49#discussion_r211057038
##
File path:
wagon-providers/wagon-file-test/src/main/java/org/apache/maven/wagon/providers/file/LazyLockableFileOutputStreamTestDriver.java
##
@@ -0,0 +1,75 @@
+package org.apache.maven.wagon.providers.file;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.io.File;
+import java.io.IOException;
+import java.io.OutputStream;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Test driver for LazyLockableFileOutputStreamIntegrationTest
+ */
+public class LazyLockableFileOutputStreamTestDriver
+{
+public static void main( String[] args ) throws Exception
+{
+File file = new File( args[0] );
+long timeout = Long.parseLong( args[1] );
+long startTime = System.currentTimeMillis();
+OutputStream outputStream = new LazyLockableFileOutputStream( file,
timeout, TimeUnit.SECONDS );
+try
+{
+outputStream.write( 1 );
+}
+catch ( IOException e )
+{
+if ( e.getMessage().equals( "Can't write file, lock " +
file.getAbsolutePath() + ".lck exists" ) )
+{
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 126 );
+// CHECKSTYLE_ON: MagicNumber
+}
+else if ( e.getMessage().equals( "Failed to create lockfile " +
file.getAbsolutePath()
++ ".lck after waiting " + timeout + " seconds. File
already exists." ) )
+{
+long diff = System.currentTimeMillis() - startTime;
+if ( diff < TimeUnit.SECONDS.toMillis( timeout ) )
+{
+throw new Exception( "We were supposed to wait for " +
timeout
++ " seconds, but Exception came early at " + diff
+ " milliseconds." );
+}
+System.out.println( "ready" );
+// CHECKSTYLE_OFF: MagicNumber
+System.exit( 127 );
Review comment:
please avoid System.exit this can fail to correctly report failures with
surefire
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Concurrent-safe access to local Maven repository
>
>
> Key: MNG-2802
> URL: https://issues.apache.org/jira/browse/MNG-2802
> Project: Maven
> Issue Type: Improvement
> Components: Artifacts and Repositories
>Reporter: Stepan Roh
>Assignee: John Casey
>Priority: Major
> Fix For: Issues to be reviewed for 3.x
>
>
> It seems that access to local Maven repository is not concurrent-safe that is
> multiple Mavens running in parallel may damage contents of local Maven
> repository. It would be a nice improvement, because sharing of local
> repository will lower the need for contacting any other repository. I know
> that Maven proxy can be used, but that adds another layer which may
> unnecessarily stress the machine it runs on.
> *NOTE:*
> http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is
> a Takari Extension for Apache Maven that introduces a concurrent-safe
> local-repository.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584420#comment-16584420 ] Erik Håkansson commented on MNG-2802: - The PR mentioned above is for the File wagon in maven-wagon, and is only tested isolated in there. Since it's pretty much impossible to reliably reproduce this with a real maven, I had to be satisfied with that. > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. > *NOTE:* > http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is > a Takari Extension for Apache Maven that introduces a concurrent-safe > local-repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16584418#comment-16584418 ] ASF GitHub Bot commented on MNG-2802: - erikhakansson opened a new pull request #49: [MNG-2802] Concurrent-safe access to local Maven repository URL: https://github.com/apache/maven-wagon/pull/49 Made file wagon optionally lock files using an adapted version of LockableFileWriter from commons-io. Integration test is made by forking two separate java processes that attempts to lock the same file. Only the first will succeed. At first I tried using Java's FileLock, but that proved impossible on Windows Subsystem for Linux as that doesn't reliably support file locks, and after some googling I realized the native FileLocks are generally unreliable from Java. Instead I opted to use lockfiles using the commons-io approach. Use system property maven.wagon.file.fileLock=true to enable file lock. Default is off, default timeout is 300 seconds, but configurable with maven.wagon.file.fileLock.timeout This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. > *NOTE:* > http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is > a Takari Extension for Apache Maven that introduces a concurrent-safe > local-repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496303#comment-16496303 ] Mike K commented on MNG-2802: - +1 P.S takari is page is down. > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. > *NOTE:* > http://takari.io/book/30-team-maven.html#concurrent-safe-local-repository is > a Takari Extension for Apache Maven that introduces a concurrent-safe > local-repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400127#comment-16400127 ] Erik Håkansson commented on MNG-2802: - This issue plagues a lot of our builds. A lock file with a PID seems like it should solve the issue. > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey >Priority: Major > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205923#comment-16205923 ] Kevin Cross commented on MNG-2802: -- Are there any plans to merge the Takari fix for this issue into the standard Maven release? I believe this issue may be incorrectly classified as an improvement when it should be marked as a bug. When the issue was created it may have been an improvement as CI was less of a common practice but who would be using Maven now in an enterprise environment and not doing CI? There are work arounds for CI systems but if you don't know about this issue you will not put them in place which will lead to broken builds due to surprising behaviour by the build tool. > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MNG-2802) Concurrent-safe access to local Maven repository
[ https://issues.apache.org/jira/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199930#comment-16199930 ] Michal Domagala commented on MNG-2802: -- [~J-cztery] Local repository may be corrupted by Maven embedded with Eclipse etc. Takari extension is excellent but it is workaround only > Concurrent-safe access to local Maven repository > > > Key: MNG-2802 > URL: https://issues.apache.org/jira/browse/MNG-2802 > Project: Maven > Issue Type: Improvement > Components: Artifacts and Repositories >Reporter: Stepan Roh >Assignee: John Casey > Fix For: Issues to be reviewed for 3.x > > > It seems that access to local Maven repository is not concurrent-safe that is > multiple Mavens running in parallel may damage contents of local Maven > repository. It would be a nice improvement, because sharing of local > repository will lower the need for contacting any other repository. I know > that Maven proxy can be used, but that adds another layer which may > unnecessarily stress the machine it runs on. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven repository
[ https://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=278089#comment-278089 ] cforce commented on MNG-2802: - This issue hits us most on a Jenkins buidl server, where all Maven builds use same repo folder. Having many dupes of local repos will use a lot of local hard disk space. Maybe there is a Maven Jenkins Plugin for Jenkins which coordinates the Concurrent access? Concurrent-safe access to local Maven repository Key: MNG-2802 URL: https://jira.codehaus.org/browse/MNG-2802 Project: Maven 2 3 Issue Type: Improvement Components: Artifacts and Repositories Reporter: Stepan Roh Assignee: John Casey Fix For: Issues to be reviewed for 3.x It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven repository
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=238118#action_238118 ] John Lonergan commented on MNG-2802: This issue really impacts our ability to run concurrent continuous integration builds making out CI take several times longer to cycle than we'd like. We have compensated by having each CI task have it's own private local repo but this massively expands the disk requirements and has the affect of slowing everything down due to all the duplicate file downloading going on. But the really painful thing is that when we have separate local repo's the various threads cannot share the snapshots via the local repo. Please try to get a fix in place for this. Concurrent-safe access to local Maven repository Key: MNG-2802 URL: http://jira.codehaus.org/browse/MNG-2802 Project: Maven 2 3 Issue Type: Improvement Components: Artifacts and Repositories Reporter: Stepan Roh Assignee: John Casey Fix For: Issues to be reviewed for 3.x It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven repository
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=238175#action_238175 ] John Lonergan commented on MNG-2802: Or possibly someone could advise where the piece of code that maintains the local repo lies and I'll have look. Things that hurt seem mostly to be that the repo metadata gets corrupted. Concurrent-safe access to local Maven repository Key: MNG-2802 URL: http://jira.codehaus.org/browse/MNG-2802 Project: Maven 2 3 Issue Type: Improvement Components: Artifacts and Repositories Reporter: Stepan Roh Assignee: John Casey Fix For: Issues to be reviewed for 3.x It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven repository
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=215219#action_215219 ] Paul Benedict commented on MNG-2802: Giving Maven an exclusive lock on an artifact's local metadata should be a sure way of ensuring this. Concurrent-safe access to local Maven repository Key: MNG-2802 URL: http://jira.codehaus.org/browse/MNG-2802 Project: Maven 2 3 Issue Type: Improvement Components: Artifacts and Repositories Reporter: Stepan Roh Assignee: John Casey Fix For: 3.x (to be reviewed) It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven repository
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=147569#action_147569 ] Brett Porter commented on MNG-2802: --- This is also discussed in http://docs.codehaus.org/display/MAVEN/Local+repository+separation, along with MNG-3655 Concurrent-safe access to local Maven repository Key: MNG-2802 URL: http://jira.codehaus.org/browse/MNG-2802 Project: Maven 2 Issue Type: Improvement Components: Artifacts and Repositories Reporter: Stepan Roh Assignee: John Casey Fix For: 3.0 It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven repository
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=146184#action_146184 ] Christian Gruber commented on MNG-2802: --- Frankly, one could mitigate this substantially by downloading artifacts into temporary files, then once the artifact is down with all its pieces, moving the temporary files into the permanent location. There is still a race condition but it's down to the order of magnitude of milliseconds during the move operation, rather than the many seconds during downloading. Concurrent-safe access to local Maven repository Key: MNG-2802 URL: http://jira.codehaus.org/browse/MNG-2802 Project: Maven 2 Issue Type: Improvement Components: Artifacts and Repositories Reporter: Stepan Roh Assignee: John Casey Fix For: 3.0 It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MNG-2802) Concurrent-safe access to local Maven repository
[ http://jira.codehaus.org/browse/MNG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_126325 ] John Casey commented on MNG-2802: - To go any further than just ensuring the locking of resolver-status files (which are a sort of proto-metadata for the resolver to use that helps track unfound artifacts and adhere to an updateInterval for those), we'll need to modify the various Wagons, I think. Since these are file-oriented, they manage opening, writing to, and closing the files internally. One thing I might be able to do is try to manage a file lock on the main artifact file instead of the temp file that the wagon is using as its destination (which may not exist, so that could complicate things even further). Yet another approach might involve creating a lockfile, and managing a lock on that... I'll look into a couple of these strategies. I don't think it's optimal to bury the locking logic in the various wagons, as we're likely to run into concurrency problems again and again with each marginal wagon implementation. Concurrent-safe access to local Maven repository Key: MNG-2802 URL: http://jira.codehaus.org/browse/MNG-2802 Project: Maven 2 Issue Type: Improvement Components: Artifacts and Repositories Reporter: Stepan Roh Assignee: John Casey Fix For: 2.1 It seems that access to local Maven repository is not concurrent-safe that is multiple Mavens running in parallel may damage contents of local Maven repository. It would be a nice improvement, because sharing of local repository will lower the need for contacting any other repository. I know that Maven proxy can be used, but that adds another layer which may unnecessarily stress the machine it runs on. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
