[
https://issues.apache.org/jira/browse/MRESOLVER-283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628108#comment-17628108
]
ASF GitHub Bot commented on MRESOLVER-283:
------------------------------------------
caiwei-ebay commented on code in PR #213:
URL: https://github.com/apache/maven-resolver/pull/213#discussion_r1012479745
##########
maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java:
##########
@@ -656,18 +622,4 @@ private void uploadChecksum( URI location, Object checksum
)
}
}
-
- private static class DirectExecutor
- implements Executor
- {
-
- static final Executor INSTANCE = new DirectExecutor();
-
- @Override
- public void execute( Runnable command )
- {
- command.run();
- }
-
Review Comment:
Given this being deleted, I think it may introduce additional cost for DF.
DF downloads pom one by one, originally it will be downloaded by main thread
directly and now it will be downloaded in a thread and the main thread has to
wait
##########
maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/collect/bf/BfDependencyCollector.java:
##########
@@ -475,23 +482,22 @@ else if ( descriptorResult == DataPool.NO_DESCRIPTOR )
static class ParallelDescriptorResolver
{
- final ExecutorService executorService;
+ final ResolverExecutor executor;
/**
* Artifact ID -> Future of DescriptorResolutionResult
*/
final Map<String, Future<DescriptorResolutionResult>> results = new
ConcurrentHashMap<>( 256 );
- final Logger logger = LoggerFactory.getLogger( getClass() );
- ParallelDescriptorResolver( RepositorySystemSession session )
+ ParallelDescriptorResolver( ResolverExecutor executor )
{
- this.executorService = getExecutorService( session );
+ this.executor = executor;
}
void resolveDescriptors( Artifact artifact,
Callable<DescriptorResolutionResult> callable )
{
results.computeIfAbsent( ArtifactIdUtils.toId( artifact ),
- key -> this.executorService.submit( callable ) );
+ key -> this.executor.submit( callable ) );
Review Comment:
In the illustration below:
https://camo.githubusercontent.com/71a9619274e478245df1c37423239b79f6b3036a3af72690d7ace08419ee21ac/68747470733a2f2f6973737565732e6170616368652e6f72672f6a6972612f7365637572652f6174746163686d656e742f31333034373931392f7265736f6c76655f646570732e706e67
As we submit a Callable<DescriptorResolutionResult> here, and this callable
finally resorts to RepositoryConnector which will then submit a
Collection<Artifact> (a collection with single artifact) to the executor, 2
different tasks running in one executor and besides the 2 tasks has
dependencies.
The original behavior is:
we submit a Callable<DescriptorResolutionResult> here, and this callable
finally resorts to RepositoryConnector which will then submit a
Collection<Artifact> (a collection with single artifact) to the executor, but
this time the executor is actually a DirectExecutor which actually runs in main
thread. Please check code below.
if ( maxThreads <= 1 )
{
return DirectExecutor.INSTANCE;
}
int tasks = safe( artifacts ).size() + safe( metadatas ).size();
if ( tasks <= 1 )
{
return DirectExecutor.INSTANCE;
}
##########
src/site/markdown/configuration.md:
##########
@@ -52,7 +52,6 @@ Option | Type | Description | Default Value | Supports Repo
ID Suffix
`aether.dependencyCollector.maxExceptions` | int | Only exceptions up to the
number given in this configuration property are emitted. Exceptions which
exceed that number are swallowed. | `50` | no
`aether.dependencyCollector.impl` | String | The name of the dependency
collector implementation to use: depth-first (original) named `df`, and
breadth-first (new in 1.8.0) named `bf`. Both collectors produce equivalent
results, but they may differ performance wise, depending on project being
applied to. Our experience shows that existing `df` is well suited for smaller
to medium size projects, while `bf` may perform better on huge projects with
many dependencies. Experiment (and come back to us!) to figure out which one
suits you the better. | `"df"` | no
Review Comment:
Shall we add more info about "BF will download poms/metadata.xml in
parallel"?
> Introduce resolver wide "shared" executor service
> -------------------------------------------------
>
> Key: MRESOLVER-283
> URL: https://issues.apache.org/jira/browse/MRESOLVER-283
> Project: Maven Resolver
> Issue Type: Improvement
> Components: Resolver
> Reporter: Tamas Cservenak
> Assignee: Tamas Cservenak
> Priority: Major
> Fix For: 1.9.0
>
>
> More and more component in resolver does parallel processing (BF collector,
> MD resolver, basic connector), and they all create, maintain their own
> executor instance.
> Instead of this, create one shared "thread pool" (executor) component and
> just share it accross resovler.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
