Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39659222
I tested this in maven and sbt builds. Seemed to work for me!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as we
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r11324595
--- Diff: bin/compute-classpath.sh ---
@@ -59,7 +45,7 @@ if [ -f "$ASSEMBLY_DIR"/spark-assembly*hadoop*-deps.jar
]; then
CLASSPATH="$CLASSPATH:$FWDIR/
Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r11324565
--- Diff: bin/compute-classpath.sh ---
@@ -59,7 +45,7 @@ if [ -f "$ASSEMBLY_DIR"/spark-assembly*hadoop*-deps.jar
]; then
CLASSPATH="$CLASSPATH:$FWDIR/
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r11324292
--- Diff: bin/compute-classpath.sh ---
@@ -71,6 +57,23 @@ else
CLASSPATH="$CLASSPATH:$ASSEMBLY_JAR"
fi
+# When Hive support is needed, Da
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r11324287
--- Diff: bin/compute-classpath.sh ---
@@ -59,7 +45,7 @@ if [ -f "$ASSEMBLY_DIR"/spark-assembly*hadoop*-deps.jar
]; then
CLASSPATH="$CLASSPATH:$FWDIR/
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r11324290
--- Diff: bin/compute-classpath.sh ---
@@ -71,6 +57,23 @@ else
CLASSPATH="$CLASSPATH:$ASSEMBLY_JAR"
fi
+# When Hive support is needed, Da
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39655971
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39655972
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13811/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39655300
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39655298
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39655244
Also fixes SPARK-1309.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39654658
Build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39654659
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13809/
---
If your project is set up for it, you can r
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39654636
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39654631
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r11323799
--- Diff: project/SparkBuild.scala ---
@@ -395,8 +404,6 @@ object SparkBuild extends Build {
// assembly jar.
def hiveSettings = sharedSettings +
Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-39654566
Alright, I changed up the solution by having Maven also download the
datanucleus jars into lib_managed/jars/ (unlike SBT, Maven will only download
these jars, and only wh
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38881838
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your proj
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r10996156
--- Diff: project/SparkBuild.scala ---
@@ -395,8 +404,6 @@ object SparkBuild extends Build {
// assembly jar.
def hiveSettings = sharedSettings +
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38718243
Ah sorry I didn't read your description. Indeed, this wouldn't work for
maven. Having a file flag seems potentially complicated if someone does
multiple builds some with
Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38716186
One issue with the "including datanucleus if it exists" approach is that we
need to at some point download datanucleus for the user. Right now sbt does
that via lib_manag
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38715665
One other option is to have a file flag like we do for `RELEASE` that is
used in addition to the env var.
---
If your project is set up for it, you can reply to this ema
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38715395
We could also include them and log a message like "Including Datanucleus
jars needed for Hive" and then if someone happens to not want them (they
previously did a Hive as
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38715156
Thanks Aaron - looks good. From what I can tell you've just copied the
model used for ganglia and previously YARN.
The only ugliness is that SPARK_HIVE is now nee
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r10988066
--- Diff: sql/hive/pom.xml ---
@@ -53,6 +53,12 @@
${hive.version}
+
+org.codehaus.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38662413
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38662415
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13466/
---
If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38658678
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not hav
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38658679
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/237#issuecomment-38656860
@marmbrus @pwendell Please take a look, my build-foo is not strong.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/237#discussion_r10966616
--- Diff: pom.xml ---
@@ -373,7 +373,6 @@
org.apache.derby
derby
10.4.2.0
-test
--- End diff --
GitHub user aarondav opened a pull request:
https://github.com/apache/spark/pull/237
[WIP] SPARK-1314: Use SPARK_HIVE to determine if we include Hive in
packaging
Previously, we based our decision regarding including datanucleus jars
based on the existence of a spark-hive-assembly
32 matches
Mail list logo
