This is an automated email from the ASF dual-hosted git repository.

srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 3909223  [MINOR][DOCS] Clarify that Spark apps should mark Spark as a 
'provided' dependency, not package it
3909223 is described below

commit 39092236819da097e9c8a3b2fa975105f08ae5b9
Author: Sean Owen <sean.o...@databricks.com>
AuthorDate: Tue Mar 5 08:26:30 2019 -0600

    [MINOR][DOCS] Clarify that Spark apps should mark Spark as a 'provided' 
dependency, not package it
    
    ## What changes were proposed in this pull request?
    
    Spark apps do not need to package Spark. In fact it can cause problems in 
some cases. Our examples should show depending on Spark as a 'provided' 
dependency.
    
    Packaging Spark makes the app much bigger by tens of megabytes. It can also 
bring in conflicting dependencies that wouldn't otherwise be a problem. 
https://issues.apache.org/jira/browse/SPARK-26146 was what reminded me of this.
    
    ## How was this patch tested?
    
    Doc build
    
    Closes #23938 from srowen/Provided.
    
    Authored-by: Sean Owen <sean.o...@databricks.com>
    Signed-off-by: Sean Owen <sean.o...@databricks.com>
---
 docs/cloud-integration.md           | 1 +
 docs/quick-start.md                 | 1 +
 docs/streaming-programming-guide.md | 3 ++-
 3 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/docs/cloud-integration.md b/docs/cloud-integration.md
index 5368e13..a014d6f 100644
--- a/docs/cloud-integration.md
+++ b/docs/cloud-integration.md
@@ -87,6 +87,7 @@ is set to the chosen version of Spark:
     <groupId>org.apache.spark</groupId>
     <artifactId>hadoop-cloud_{{site.SCALA_BINARY_VERSION}}</artifactId>
     <version>${spark.version}</version>
+    <scope>provided</scope>
   </dependency>
   ...
 </dependencyManagement>
diff --git a/docs/quick-start.md b/docs/quick-start.md
index 28186c1..8bf0d6e 100644
--- a/docs/quick-start.md
+++ b/docs/quick-start.md
@@ -341,6 +341,7 @@ Note that Spark artifacts are tagged with a Scala version.
       <groupId>org.apache.spark</groupId>
       <artifactId>spark-sql_{{site.SCALA_BINARY_VERSION}}</artifactId>
       <version>{{site.SPARK_VERSION}}</version>
+      <scope>provided</scope>
     </dependency>
   </dependencies>
 </project>
diff --git a/docs/streaming-programming-guide.md 
b/docs/streaming-programming-guide.md
index 94c6120..30a0160 100644
--- a/docs/streaming-programming-guide.md
+++ b/docs/streaming-programming-guide.md
@@ -385,11 +385,12 @@ Similar to Spark, Spark Streaming is available through 
Maven Central. To write y
         <groupId>org.apache.spark</groupId>
         <artifactId>spark-streaming_{{site.SCALA_BINARY_VERSION}}</artifactId>
         <version>{{site.SPARK_VERSION}}</version>
+        <scope>provided</scope>
     </dependency>
 </div>
 <div data-lang="SBT" markdown="1">
 
-       libraryDependencies += "org.apache.spark" % 
"spark-streaming_{{site.SCALA_BINARY_VERSION}}" % "{{site.SPARK_VERSION}}"
+       libraryDependencies += "org.apache.spark" % 
"spark-streaming_{{site.SCALA_BINARY_VERSION}}" % "{{site.SPARK_VERSION}}" % 
"provided"
 </div>
 </div>
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

Reply via email to