Re: A Maven extension for dependency tracking

[Tamás Cservenák]

Howdy,

This is very cool, I was actually tinkering on very similar issues in
resolver coming from totally different angles.

And yes, the resolver collector is not quite "extension" friendly, but we
will make it right.
Just FYI, that in the latest resolver (1.8.0) there are actually two
implementations: depth-first (original) and depth-first.

By looking at your code: collection is most critical regarding performance
and memory in the resolver, so "hooking" into it (like sending events per
each step) might not be the best, but still, what kind of extension points
would you envision in the collector?

For example, to achieve what you want, it would be completely enough to
receive the final CollectResult (the full graph), no?
As -- from a resolver perspective -- that would be simplest, especially
that now we have two collector implementations...

Also, in case of multi threading, your shared stack would not cut, would it?

I personally was also looking into these, especially after some of the
latest additions to resolver in 1.8.0 and current master


Thanks
T


On Thu, Apr 28, 2022 at 12:45 PM Grzegorz Grzybek 
wrote:

> Hello
>
> TL;DR: https://github.com/grgrzybek/tracking-maven-extension
>
> I'd like to share some proof of concept I made. It all started with a
> question "why I'm getting log4j:log4j:1.2.12" in my local Maven repository
> when building trivial project with fresh local repo?
>
> I knew it's possible to `grep -r --include=*.pom 1.2.12` the poms that
> declare old log4j, but I needed something better.
>
> In short words - I managed to persist the information available in
>
> org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.Args#nodes
> stack.
> I wrote a Maven extension that can be put into $MAVEN_HOME/lib/ext or used
> with "-Dmaven.ext.class.path" which does two things:
>
>1. adds org.eclipse.aether.RepositoryListener component that writes some
>information when a dependency is FIRST downloaded from remote repository
>2. adds org.eclipse.aether.impl.DependencyCollector component (extension
>of org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector)
>that writes some information when a dependency is resolved against local
>repository when it's already there (where no download is needed)
>
> In the first case, I write something like this:
>
> ~~~
> Downloaded artifact log4j:log4j:pom::1.2.12 (repository: central (
> https://repo.maven.apache.org/maven2, default, releases))
>-> commons-logging:commons-logging:jar:1.1 (compile) (context: plugin)
>  -> commons-digester:commons-digester:jar:1.8 (compile) (context:
> plugin)
>-> org.apache.velocity:velocity-tools:jar:2.0 (compile) (context:
> plugin)
>  -> org.apache.maven.doxia:doxia-site-renderer:jar:1.11.1 (compile)
> (context: plugin)
>-> org.apache.maven.plugins:maven-site-plugin:jar:3.11.0 ()
> (context: plugin)
>   Reading descriptor for artifact log4j:log4j:jar::1.2.12 (context: plugin)
> (scope: ?) (repository: central (https://repo.maven.apache.org/maven2,
> default, releases))
> Transitive dependencies collection for
> org.apache.maven.plugins:maven-site-plugin:jar:3.11.0 ()
>   Resolution of plugin
> org.apache.maven.plugins:maven-site-plugin:3.11.0 (org.apache:apache:25)
> ~~~
> Downloaded artifact log4j:log4j:jar::1.2.12 (repository: central (
> https://repo.maven.apache.org/maven2, default, releases))
>   Resolution of plugin com.mycila:license-maven-plugin:3.0
> (org.apache.camel:camel-buildtools:3.17.0-SNAPSHOT)
>
> I simply write some information from available
> org.eclipse.aether.RepositoryEvent and event's
> org.eclipse.aether.RequestTrace.
>
> More interesting information is written in 2nd case. Because I wanted to
> track ALL attempts to resolve log4j:log4j:1.2.12 (and any other
> dependency), I needed some structure. And I decided this:
>
>- every dependency directory (where e.g., _remote.repositories is
>written along with the jar/pom/sha1/md5/...) gets ".tracking" directory
>- in ".tracking" directory I write files with names of this pattern:
>"groupId_artifactId_type_classifier_version.dep", e.g.,
>org.apache.maven.plugins_maven-dependency-plugin_jar_3.1.2.dep
>- each such file contains a _reverse dependency tree_ that shows my why
>given dependency was resolved.
>
> For example, in
>
> ~/.m2/repository/log4j/log4j/1.2.12/.tracking/org.apache.maven.plugins_maven-dependency-plugin_jar_3.1.2.dep
> (the path itself already contains information that
> org.apache.maven.plugins:maven-dependency-plugin:3.1.2 depends (directly or
> indirectly) in log4j:logj4:1.2.12.
> The content of this file is:
>
> log4j:log4j:pom:1.2.12
>  -> commons-logging:commons-logging:jar:1.1 (compile) (context: plugin)
>-> commons-digester:commons-digester:jar:1.8 (compile) (context: plugin)
>  -> org.apache.velocity:velocity-tools:jar:2.0 (compile) (context:
> plugin)
>-> 

Re: [VOTE] Release Maven Site Plugin version 4.0.0-M1

[Sylwester Lachiewicz]

+1

śr., 27 kwi 2022, 23:25 użytkownik Michael Osipov 
napisał:

> Hi,
>
> We solved 6 issues:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317923=12351657
>
> There are still a couple of issues left in JIRA:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20MSITE%20AND%20resolution%20%3D%20Unresolved
>
> Staging repo:
> https://repository.apache.org/content/repositories/maven-1748/
>
> https://repository.apache.org/content/repositories/maven-1748/org/apache/maven/plugins/maven-site-plugin/4.0.0-M1/maven-site-plugin-4.0.0-M1-source-release.zip
>
> Source release checksum(s):
> maven-site-plugin-4.0.0-M1-source-release.zip
> sha512:
>
> 170fcefd12099f4b527b3ad9bc94098b1ee0cab7788982dfebfa69097f24746f847a6e181dfaab3caa54f508f13994195fa44c64fe1deef66ddb723fe852
>
> Staging site:
> https://maven.apache.org/plugins-archives/maven-site-plugin-LATEST/
>
> Guide to testing staged releases:
> https://maven.apache.org/guides/development/guide-testing-releases.html
>
> Vote open for 72 hours.
>
> [ ] +1
> [ ] +0
> [ ] -1
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>

Re: [VOTE] Release Maven Site Plugin version 4.0.0-M1

[Michael Osipov]

Am 2022-04-27 um 23:25 schrieb Michael Osipov:

Hi,

We solved 6 issues:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317923=12351657 



There are still a couple of issues left in JIRA:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20MSITE%20AND%20resolution%20%3D%20Unresolved 



Staging repo:
https://repository.apache.org/content/repositories/maven-1748/
https://repository.apache.org/content/repositories/maven-1748/org/apache/maven/plugins/maven-site-plugin/4.0.0-M1/maven-site-plugin-4.0.0-M1-source-release.zip 



Source release checksum(s):
maven-site-plugin-4.0.0-M1-source-release.zip
sha512: 
170fcefd12099f4b527b3ad9bc94098b1ee0cab7788982dfebfa69097f24746f847a6e181dfaab3caa54f508f13994195fa44c64fe1deef66ddb723fe852 



Staging site:
https://maven.apache.org/plugins-archives/maven-site-plugin-LATEST/

Guide to testing staged releases:
https://maven.apache.org/guides/development/guide-testing-releases.html

Vote open for 72 hours.


+1


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: [VOTE] Release Maven Site Plugin version 4.0.0-M1

[Michael Osipov]

Am 2022-04-28 um 01:53 schrieb Olivier Lamy:

The issues are just some dependency updates.
So can we have some details? What are the impacts (if any) to end users?


No, those aren't just some dependency updates. It raises the baseline to 
a boldy reduced Doxia, removed thousands of lines of code. You should 
check the dependency releases what has actuall change.


For that reason I will review out two skins to make them properly run. 
Maven Site Plugin 3.x will remain production quality for quite some time.


M

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: mvnd plan? release soon?

[Romain Manni-Bucau]

What about moving it to @asf/maven project? Since it is coupled to it and
AFAIK will not be decoupled soon, it really belongs to it IMHO.

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le jeu. 28 avr. 2022 à 13:59, Olivier Lamy  a écrit :

> Hi
> I wonder what are the plans (or any work in progress) regarding mvnd?
> Such as moving to org.apache.maven namespace (distribution and java
> package)?
> any release with a more recent mvn core?  My understanding is mvnd is
> locked to a maven core version and it's not dynamic (such using M2_HOME).
>
> having some documentation such https://maven.apache.org/mvnd or not :)
>
> and etc.. :)
> It's a great tool and I like using it but I feel like nothing happened
> since it has been moved here?
> Do we need more hands on it?
>
> cheers
> Olivier
>

mvnd plan? release soon?

[Olivier Lamy]

Hi
I wonder what are the plans (or any work in progress) regarding mvnd?
Such as moving to org.apache.maven namespace (distribution and java
package)?
any release with a more recent mvn core?  My understanding is mvnd is
locked to a maven core version and it's not dynamic (such using M2_HOME).

having some documentation such https://maven.apache.org/mvnd or not :)

and etc.. :)
It's a great tool and I like using it but I feel like nothing happened
since it has been moved here?
Do we need more hands on it?

cheers
Olivier

A Maven extension for dependency tracking

[Grzegorz Grzybek]

Hello

TL;DR: https://github.com/grgrzybek/tracking-maven-extension

I'd like to share some proof of concept I made. It all started with a
question "why I'm getting log4j:log4j:1.2.12" in my local Maven repository
when building trivial project with fresh local repo?

I knew it's possible to `grep -r --include=*.pom 1.2.12` the poms that
declare old log4j, but I needed something better.

In short words - I managed to persist the information available in
org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.Args#nodes
stack.
I wrote a Maven extension that can be put into $MAVEN_HOME/lib/ext or used
with "-Dmaven.ext.class.path" which does two things:

   1. adds org.eclipse.aether.RepositoryListener component that writes some
   information when a dependency is FIRST downloaded from remote repository
   2. adds org.eclipse.aether.impl.DependencyCollector component (extension
   of org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector)
   that writes some information when a dependency is resolved against local
   repository when it's already there (where no download is needed)

In the first case, I write something like this:

~~~
Downloaded artifact log4j:log4j:pom::1.2.12 (repository: central (
https://repo.maven.apache.org/maven2, default, releases))
   -> commons-logging:commons-logging:jar:1.1 (compile) (context: plugin)
 -> commons-digester:commons-digester:jar:1.8 (compile) (context:
plugin)
   -> org.apache.velocity:velocity-tools:jar:2.0 (compile) (context:
plugin)
 -> org.apache.maven.doxia:doxia-site-renderer:jar:1.11.1 (compile)
(context: plugin)
   -> org.apache.maven.plugins:maven-site-plugin:jar:3.11.0 ()
(context: plugin)
  Reading descriptor for artifact log4j:log4j:jar::1.2.12 (context: plugin)
(scope: ?) (repository: central (https://repo.maven.apache.org/maven2,
default, releases))
Transitive dependencies collection for
org.apache.maven.plugins:maven-site-plugin:jar:3.11.0 ()
  Resolution of plugin
org.apache.maven.plugins:maven-site-plugin:3.11.0 (org.apache:apache:25)
~~~
Downloaded artifact log4j:log4j:jar::1.2.12 (repository: central (
https://repo.maven.apache.org/maven2, default, releases))
  Resolution of plugin com.mycila:license-maven-plugin:3.0
(org.apache.camel:camel-buildtools:3.17.0-SNAPSHOT)

I simply write some information from available
org.eclipse.aether.RepositoryEvent and event's
org.eclipse.aether.RequestTrace.

More interesting information is written in 2nd case. Because I wanted to
track ALL attempts to resolve log4j:log4j:1.2.12 (and any other
dependency), I needed some structure. And I decided this:

   - every dependency directory (where e.g., _remote.repositories is
   written along with the jar/pom/sha1/md5/...) gets ".tracking" directory
   - in ".tracking" directory I write files with names of this pattern:
   "groupId_artifactId_type_classifier_version.dep", e.g.,
   org.apache.maven.plugins_maven-dependency-plugin_jar_3.1.2.dep
   - each such file contains a _reverse dependency tree_ that shows my why
   given dependency was resolved.

For example, in
~/.m2/repository/log4j/log4j/1.2.12/.tracking/org.apache.maven.plugins_maven-dependency-plugin_jar_3.1.2.dep
(the path itself already contains information that
org.apache.maven.plugins:maven-dependency-plugin:3.1.2 depends (directly or
indirectly) in log4j:logj4:1.2.12.
The content of this file is:

log4j:log4j:pom:1.2.12
 -> commons-logging:commons-logging:jar:1.1 (compile) (context: plugin)
   -> commons-digester:commons-digester:jar:1.8 (compile) (context: plugin)
 -> org.apache.velocity:velocity-tools:jar:2.0 (compile) (context:
plugin)
   -> org.apache.maven.doxia:doxia-site-renderer:jar:1.7.4 (compile)
(context: plugin)
 -> org.apache.maven.reporting:maven-reporting-impl:jar:3.0.0
(compile) (context: plugin)
   -> org.apache.maven.plugins:maven-dependency-plugin:jar:3.1.2 ()
(context: plugin)

It's kind of obvious - dependency-plugin through maven-reporint-impl,
through doxia, velocity, commons-digester and commons-logging "depends" on
malicious log4j:1.2.12 library every security scanner screams about.

Since I wrote this extension, I keep it in my @MAVEN_HOME/lib/ext and build
everything in my work. Now I know why my
~/.m2/repository/org/codehaus/plexus/plexus-utils/ directory contains 57
different versions of plexus-utils for example. for example why 1.0.4 from
2005?

org.codehaus.plexus:plexus-utils:pom:1.0.4
 -> org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1
(compile) (context: plugin)
   -> org.codehaus.plexus:plexus-velocity:jar:1.2 (compile) (context:
plugin)
 -> org.apache.maven.doxia:doxia-site-renderer:jar:1.11.1 (compile)
(context: plugin)
   -> org.apache.maven.plugins:maven-javadoc-plugin:jar:3.3.2 ()
(context: plugin)

Why Guava 10.0.1?

com.google.guava:guava:pom:10.0.1
 -> org.eclipse.sisu:org.eclipse.sisu.plexus:jar:0.0.0.M5 (compile)
(context: plugin)
   ->