without -Daether.checksums.algorithms=SHA1,MD5, very slow -D
On Sat, Sep 26, 2020 at 3:11 PM Michael Osipov <micha...@apache.org> wrote: > Awesome, what about with the fix, but with the default list of hashing > algos (SHA-512, SHA-256, SHA-1, MD5) in place? > > Am 2020-09-27 um 00:07 schrieb Dan Tran: > > it works using the provided commitID + > > -Daether.checksums.algorithms=SHA1,MD5 > > > > -D > > > > On Sat, Sep 26, 2020 at 2:25 PM Michael Osipov <micha...@apache.org> > wrote: > > > >> Gosh, I assumed so. Every SHA family is horrible in terms of > >> performance. This is going to be a huge problem for the community, I > >> would prefer Blake2 or Blake3 for this. > >> > >> Anyway, I think this is the root cause, actually the fix: > >> > >> > https://github.com/apache/maven-resolver/commit/a9e0ca8307e532fc30591a47d7d96e27fa647465 > >> > >> Can you please try with the property set > >> (-Daether.checksums.algorithms=SHA1,MD5)? > >> > >> Thrilled to see your results. If this works, please remove the property > >> and see whether the patch along with SHA-2 on URLs changes the > >> performance as well. > >> > >> Michael > >> > >> Am 2020-09-26 um 21:45 schrieb Dan Tran: > >>> I am able to pin down the problematic commit: > >>> > >> > https://github.com/apache/maven-resolver/commit/8e8a4f1944a0589397e36a5dd049ec765424e863 > >>> > >>> Adding -Daether.checksums.algorithms=SHA1,MD5 to maven command line > does > >>> not help > >>> > >>> Sorry about the delay > >>> > >>> -D > >>> > >>> On Wed, Sep 16, 2020 at 1:57 AM Michael Osipov <micha...@apache.org> > >> wrote: > >>> > >>>> Your logging config is sound, so appearently nothing is wrong. > >>>> You see no trace because DefaultSyncContextFactory is empty and there > is > >>>> nothing to trace. Something else is the cause. > >>>> > >>>> Can you git bisect 1.4.2 and 1.6.0 to find the offending commit? > >>>> Otherwise I have no idea and we are searching the needle in the > >> haystack. > >>>> > >>>> The only commits which come to my mind are: > >>>> * > >>>> > >>>> > >> > https://github.com/apache/maven-resolver/commit/8e8a4f1944a0589397e36a5dd049ec765424e863?w=1 > >>>> > >>>> You can disable/restore this one by using: > >>>> -Daether.checksums.algorithms=SHA1,MD5 (previous behavior) > >>>> > >>>> * > >>>> > >>>> > >> > https://github.com/apache/maven-resolver/commit/3c2a5141c9d393a45bdee873e6aa2c9c275b7b58?w=1 > >>>> > >>>> But still this wouldn't make any sense. > >>>> > >>>> Looking forward for some findings. > >>>> > >>>> Am 2020-09-15 um 22:45 schrieb Dan Tran: > >>>>> this is my log config > >>>>> > >>>>> org.slf4j.simpleLogger.defaultLogLevel=info > >>>>> org.slf4j.simpleLogger.showDateTime=true > >>>>> org.slf4j.simpleLogger.showThreadName=true > >>>>> org.slf4j.simpleLogger.showShortLogName=true > >>>>> org.slf4j.simpleLogger.showLogName=false > >>>>> org.slf4j.simpleLogger.logFile=System.out > >>>>> org.slf4j.simpleLogger.cacheOutputStream=true > >>>>> org.slf4j.simpleLogger.levelInBrackets=true > >>>>> org.slf4j.simpleLogger.log.Sisu=info > >>>>> org.slf4j.simpleLogger.warnLevelString=WARNING > >>>>> > >>>>> # MNG-6181: mvn -X also prints all debug logging from HttpClient > >>>>> # Be aware that the shaded packages are used > >>>>> # org.apache.http -> org.apache.maven.wagon.providers.http.httpclient > >>>>> > >>>> > >> > org.slf4j.simpleLogger.log.org.apache.maven.wagon.providers.http.httpclient=off > >>>>> > >>>> > >> > org.slf4j.simpleLogger.log.org.apache.maven.wagon.providers.http.httpclient.wire=off > >>>>> > >>>>> org.slf4j.simpleLogger.log.org.eclipse.aether.synccontext=trace > >>>>> > >>>>> > >>>>> no sign of [TRACE] > >>>>> > >>>>> > >>>>> > >>>>> On Tue, Sep 15, 2020 at 1:30 PM Dan Tran <dant...@gmail.com> wrote: > >>>>> > >>>>>> Maven 3.7.0 with Resolver 1.4.2 - no issue > >>>>>> > >>>>>> Maven 3.7.0 with Resolver 1.6.0 + trace. I cant get any indication > >>>>>> about > >>>>>> > >>>>>> 4626 [main] [TRACE] GlobalSyncContextFactory$GlobalSyncContext - > >>>> Acquiring global... > >>>>>> 35318 [main] [TRACE] GlobalSyncContextFactory$GlobalSyncContext - > >>>> Releasing global... > >>>>>> > >>>>>> > >>>>>> Very strange > >>>>>> > >>>>>> -D > >>>>>> > >>>>>> On Tue, Sep 15, 2020 at 4:37 AM Michael Osipov <micha...@apache.org > > > >>>>>> wrote: > >>>>>> > >>>>>>> There is no new feature active in 1.6.0 in this regard. > >>>>>>> Here is the diff: > >>>>>>> > >>>>>>> > >>>> > >> > https://github.com/apache/maven-resolver/compare/maven-resolver-1.4.2...maven-resolver-1.6.0?w=1 > >>>>>>> > >>>>>>> Since I do not have access to your project, can you please try the > >>>>>>> following: > >>>>>>> * Maven 3.7.0 with Resolver 1.4.2 > >>>>>>> * Maven 3.6.3 with Resolver 1.6.0 > >>>>>>> * If 3.7.0/1.4.2 does not affect the build, do a git bisect with > >> 1.4.2 > >>>>>>> and 1.6.0. > >>>>>>> > >>>>>>> Alternatively, provide the log output as configured here: > >>>>>>> > >>>>>>> > >>>> > >> > https://maven.apache.org/resolver-archives/resolver-LATEST/maven-resolver-synccontext-global/index.html#Debugging > >>>>>>> > >>>>>>> In 1.6.0 there is even less synchronization because the > >>>>>>> TrackingFileManager does not lock anything anymore. > >>>>>>> > >>>>>>> I will hold off vote completion until I get some analyzable data > from > >>>> you. > >>>>>>> > >>>>>>> Michael > >>>>>>> > >>>>>>> > >>>>>>> Am 2020-09-15 um 11:49 schrieb Dan Tran: > >>>>>>>> I build maven 3.7.0 + MResolve 1.6.0. The same slow build > observed > >> at > >>>>>>> my > >>>>>>>> 300+ modules ( 60min versus 5min) > >>>>>>>> > >>>>>>>> -1 ( non-binding) from me. there must and option to disable this > >> new > >>>>>>>> feature > >>>>>>>> > >>>>>>>> -D > >>>>>>>> > >>>>>>>> On Mon, Sep 14, 2020 at 4:33 AM Michael Osipov < > micha...@apache.org > >>> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Am 2020-09-14 um 10:57 schrieb Dan Tran: > >>>>>>>>>> sorry about the fast finger, my test shows a huge increase in > >> build > >>>>>>> time > >>>>>>>>>> for large builds. Possible to disable this feature? > >>>>>>>>> > >>>>>>>>> This cannot be. It has been superceded by > >>>>>>>>> https://issues.apache.org/jira/browse/MRESOLVER-130. The > >>>>>>>>> GlobalSyncContextFactory have been moved to a separate module, > the > >>>>>>>>> original behavior has been restored. I.e., this feature is not > >>>> enabled > >>>>>>>>> by default. > >>>>>>>>> > >>>>>>>>> Are you certain that you don't have leftovers in > >>>> ${maven.home}/lib/ext? > >>>>>>>>> > >>>>>>>>>> On Mon, Sep 14, 2020 at 1:49 AM Dan Tran <dant...@gmail.com> > >> wrote: > >>>>>>>>>> > >>>>>>>>>>> possible to disable this feature > >>>>>>>>>>> https://issues.apache.org/jira/browse/MRESOLVER-123 ? > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Sep 14, 2020 at 12:06 AM Romain Manni-Bucau < > >>>>>>>>> rmannibu...@gmail.com> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> +1, tested in java 8 with mvn 3.6.3. > >>>>>>>>>>>> > >>>>>>>>>>>> Romain Manni-Bucau > >>>>>>>>>>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog > >>>>>>>>>>>> <https://rmannibucau.metawerx.net/> | Old Blog > >>>>>>>>>>>> <http://rmannibucau.wordpress.com> | Github < > >>>>>>>>>>>> https://github.com/rmannibucau> | > >>>>>>>>>>>> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book > >>>>>>>>>>>> < > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>> > >> > https://www.packtpub.com/application-development/java-ee-8-high-performance > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Le lun. 14 sept. 2020 à 08:58, Enrico Olivelli < > >>>> eolive...@gmail.com> > >>>>>>> a > >>>>>>>>>>>> écrit : > >>>>>>>>>>>> > >>>>>>>>>>>>> +1 > >>>>>>>>>>>>> checked checksum and run tests on JDK14 + Mac > >>>>>>>>>>>>> > >>>>>>>>>>>>> mvn -v > >>>>>>>>>>>>> > >>>>>>>>>>>>> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f) > >>>>>>>>>>>>> > >>>>>>>>>>>>> Maven home: xxxxxx > >>>>>>>>>>>>> > >>>>>>>>>>>>> Java version: 14, vendor: AdoptOpenJDK, runtime: > >>>>>>>>>>>>> > >>>> /Library/Java/JavaVirtualMachines/adoptopenjdk-14.jdk/Contents/Home > >>>>>>>>>>>>> > >>>>>>>>>>>>> Default locale: it_IT, platform encoding: UTF-8 > >>>>>>>>>>>>> > >>>>>>>>>>>>> OS name: "mac os x", version: "10.15.4", arch: "x86_64", > >> family: > >>>>>>> "mac" > >>>>>>>>>>>>> > >>>>>>>>>>>>> Il giorno lun 14 set 2020 alle ore 08:55 Hervé BOUTEMY < > >>>>>>>>>>>>> herve.bout...@free.fr> ha scritto: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> +1 > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> sadly, the build is not reproducible: reference build done > >> with > >>>>>>> JDK 8 > >>>>>>>>>>>> on > >>>>>>>>>>>>>> Windows, but Felix Maven Bundle Plugin is causing issues in > >>>>>>> generated > >>>>>>>>>>>>> META- > >>>>>>>>>>>>>> INF/MANIFEST.MF (contains username, detailed JDK version, > and > >>>>>>> order > >>>>>>>>> or > >>>>>>>>>>>>>> Private > >>>>>>>>>>>>>> Package not reproducible): I'll have a look to check if > recent > >>>>>>>>>>>> updates to > >>>>>>>>>>>>>> this > >>>>>>>>>>>>>> plugin can fix the issue... > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Regards, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Hervé > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Le vendredi 11 septembre 2020, 20:19:56 CEST Michael Osipov > a > >>>>>>> écrit : > >>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> We solved 19 issues: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320628&ve > >>>>>>>>>>>>>>> rsion=12348666 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> There are still a couple of issues left in JIRA: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>> > >> > https://issues.apache.org/jira/projects/MRESOLVER/issues?filter=allopenissue > >>>>>>>>>>>>>>> s > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Staging repo: > >>>>>>>>>>>>>>> > >> https://repository.apache.org/content/repositories/maven-1605/ > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>> > >> > https://repository.apache.org/content/repositories/maven-1605/org/apache/mav > >>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>> > en/resolver/maven-resolver/1.6.0/maven-resolver-1.6.0-source-release.zip > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Source release checksum(s): > >>>>>>>>>>>>>>> maven-resolver-1.6.0-source-release.zip > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>> > >> > fd419a3f0bbe3c0ea0b3d7b881c4892fcfc73ec6cf812e7ddb724febb8bc010c754d5845498c > >>>>>>>>>>>>>>> 6c916a3bbdae2d90dd66787eb44fdbdd27310c745bdc1e8cc15b > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Staging site: > >>>>>>>>>>>>>>> > https://maven.apache.org/resolver-archives/resolver-LATEST/ > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Guide to testing staged releases: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>> > https://maven.apache.org/guides/development/guide-testing-releases.html > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Note: This Resolver version offers salvation from a 13.5 > >> years > >>>>>>> old > >>>>>>>>>>>>>>> feature request: MNG-2802 > >>>>>>>>>>>>>>> Yes, you can have now concurrent-safe access to your local > >>>> Maven > >>>>>>>>>>>>>>> repository synchronized with Redis. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Vote open for 72 hours. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> [ ] +1 > >>>>>>>>>>>>>>> [ ] +0 > >>>>>>>>>>>>>>> [ ] -1 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>> > --------------------------------------------------------------------- > >>>>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >>>>>>>>>>>>>>> For additional commands, e-mail: dev-h...@maven.apache.org > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>> > --------------------------------------------------------------------- > >>>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >>>>>>>>>>>>>> For additional commands, e-mail: dev-h...@maven.apache.org > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >> --------------------------------------------------------------------- > >>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >>>>>>>>> For additional commands, e-mail: dev-h...@maven.apache.org > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > --------------------------------------------------------------------- > >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >>>>>>> For additional commands, e-mail: dev-h...@maven.apache.org > >>>>>>> > >>>>>>> > >>>>> > >>>> > >>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >>>> For additional commands, e-mail: dev-h...@maven.apache.org > >>>> > >>>> > >>> > >> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >> For additional commands, e-mail: dev-h...@maven.apache.org > >> > >> > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
