[
https://issues.apache.org/jira/browse/SPARK-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169629#comment-14169629
]
Sean Owen commented on SPARK-1317:
----------------------------------
PS if you're still interested in this, I am pretty sure #1 is the correct
answer. I would use my own sbt (or really, the SBT support in my IDE perhaps,
or Maven) to build my own app.
> sbt doesn't work for building Spark programs
> --------------------------------------------
>
> Key: SPARK-1317
> URL: https://issues.apache.org/jira/browse/SPARK-1317
> Project: Spark
> Issue Type: Bug
> Components: Build, Documentation
> Affects Versions: 0.9.0
> Reporter: Diana Carroll
>
> I don't know if this is a doc bug or a product bug, because I don't know how
> it is supposed to work.
> The Spark quick start guide page has a section that walks you through
> creating a "standalone" Spark app in Scala. I think the instructions worked
> in 0.8.1 but I can't get them to work in 0.9.0.
> The instructions have you create a directory structure in the "canonical" sbt
> format, but do not tell you where to locate this directory. However, after
> setting up the structure, the tutorial then instructs you to use the command
> {code}sbt/sbt package{code}
> which implies that the working directory must be SPARK_HOME.
> I tried it both ways: creating a "mysparkapp" directory right in SPARK_HOME
> and creating it in my home directory. Neither worked, with different results:
> - if I create a "mysparkapp" directory as instructed in SPARK_HOME, cd to
> SPARK_HOME and run the command sbt/sbt package as specified, it packages ALL
> of Spark...but does not build my own app.
> - if I create a "mysparkapp" directory elsewhere, cd to that directory, and
> run the command there, I get an error:
> {code}
> $SPARK_HOME/sbt/sbt package
> awk: cmd. line:1: fatal: cannot open file `./project/build.properties' for
> reading (No such file or directory)
> Attempting to fetch sbt
> /usr/lib/spark/sbt/sbt: line 33: sbt/sbt-launch-.jar: No such file or
> directory
> /usr/lib/spark/sbt/sbt: line 33: sbt/sbt-launch-.jar: No such file or
> directory
> Our attempt to download sbt locally to sbt/sbt-launch-.jar failed. Please
> install sbt manually from http://www.scala-sbt.org/
> {code}
> So, either:
> 1: the Spark distribution of sbt can only be used to build Spark itself, not
> you own code...in which case the quick start guide is wrong, and should
> instead say that users should install sbt separately
> OR
> 2: the Spark distribution of sbt CAN be used, with property configuration, in
> which case that configuration should be documented (I wasn't able to figure
> it out, but I didn't try that hard either)
> OR
> 3: the Spark distribution of sbt is *supposed* to be able to build Spark
> apps, but is configured incorrectly in the product, in which case there's a
> product bug rather than a doc bug
> Although this is not a show-stopper, because the obvious workaround is to
> simply install sbt separately, I think at least updating the docs is pretty
> high priority, because most people learning Spark start with that Quick Start
> page, which doesn't work.
> (If it's doc issue #1, let me know, and I'll fix the docs myself. :-) )
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
