Brian E Fox created SPARK-59229:
-----------------------------------
Summary: Use a clear Spark user agent
Key: SPARK-59229
URL: https://issues.apache.org/jira/browse/SPARK-59229
Project: Spark
Issue Type: Improvement
Components: Spark Submit
Affects Versions: 4.2.0
Reporter: Brian E Fox
I have been investigating massive amounts of traffic to Maven Central that
seems to be related to Apache Spark downloads. I found months ago that
Databricks Clusters are a large amount of the traffic, but I only recently
discovered that pyspark is apparently another major source of this traffic.
While investigating the caching and potential url overrides I discovered the
location where it seems to make sense to use a more precise user agent.
Currently this only appears to use as Apache Ivy 2.5.1 which makes it very
difficult to understand what's going on. Following the guidance on Maven
Central for tools [https://central.sonatype.org/faq/429-tooling-provider/] I'm
submitting an enhancement to the User Agent to make this easier in the future
to isolate from other traffic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
