Bo Zhang created SPARK-34757:
--------------------------------
Summary: Spark submit should ignore cache for SNAPSHOT dependencies
Key: SPARK-34757
URL: https://issues.apache.org/jira/browse/SPARK-34757
Project: Spark
Issue Type: Bug
Components: Deploy, Spark Core
Affects Versions: 3.1.1
Reporter: Bo Zhang
When spark-submit is executed with --packages, it will not download the
dependency jars when they are available in cache (e.g. ivy cache), even when
the dependencies are SNAPSHOTs.
This might block developers who work on external modules in Spark (e.g.
spark-avro), since they need to remove the cache manually every time when they
update the code during developments (which generates SNAPSHOT jars). Without
knowing this, they could be blocked wondering why their code changes are not
reflected in spark-submit executions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
