[
https://issues.apache.org/jira/browse/SPARK-31693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17111561#comment-17111561
]
Shane Knapp commented on SPARK-31693:
-------------------------------------
ok, i had a nagging suspicion that i knew what it was, and i was proven right!
'''
Here's an example of the query string
/dyn/closer.lua?action=download&filename=/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
that resulted in the ban. There are also a lot of connections to
archive.apache.org which have not (yet) hit the ban limit. These artifacts
should be cached locally as well if you are using them in a CI environment. I
unblocked the IP for now, but it may get re-banned if the traffic continues, as
it is automated.
'''
each and every build, a call is made to download maven 3.6.3. i can have
jenkins manage this, and have just about finished setting this up.
this will lessen the load by a LOT, and then we can cherry-pick other
unnecessary calls later.
> Investigate AmpLab Jenkins server network issue
> -----------------------------------------------
>
> Key: SPARK-31693
> URL: https://issues.apache.org/jira/browse/SPARK-31693
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
> Affects Versions: 3.1.0
> Reporter: Dongjoon Hyun
> Priority: Critical
>
> Given the series of failures in Spark packaging Jenkins job, it seems that
> there is a network issue in AmbLab Jenkins cluster.
> -
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-maven-snapshots/
> - The node failed to talk to GitBox. (SPARK-31687) -> GitHub is okay.
> - The node failed to download the maven mirror. (SPARK-31691) -> The primary
> host is okay.
> - The node failed to communicate repository.apache.org. (Current master
> branch Jenkins job failure)
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-deploy-plugin:3.0.0-M1:deploy (default-deploy)
> on project spark-parent_2.12: ArtifactDeployerException: Failed to retrieve
> remote metadata
> org.apache.spark:spark-parent_2.12:3.1.0-SNAPSHOT/maven-metadata.xml: Could
> not transfer metadata
> org.apache.spark:spark-parent_2.12:3.1.0-SNAPSHOT/maven-metadata.xml from/to
> apache.snapshots.https
> (https://repository.apache.org/content/repositories/snapshots): Transfer
> failed for
> https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.12/3.1.0-SNAPSHOT/maven-metadata.xml:
> Connect to repository.apache.org:443 [repository.apache.org/207.244.88.140]
> failed: Connection timed out (Connection timed out) -> [Help 1]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
