[GitHub] spark issue #18888: [Spark-17025][ML][Python] Persistence for Pipelines with...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #80547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80547/testReport)** for PR 1 at commit [`18c902c`](https://github.com/apache/spark/commit/18

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18925 **[Test build #80546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80546/testReport)** for PR 18925 at commit [`d2f7e60`](https://github.com/apache/spark/commit/d2

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-08-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15009 I kinda have up waiting for feedback to be addressed here. I'm fixing some things in the launcher library and will work on SPARK-11035 next, replacing part of this PR. --- If your project is set up

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-11 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18901 Note I'll file a separate jira for this but even in client mode it will download and then reupload the file to hdfs for yarn mode, thus defeating the purpose of it being in hdfs. --- If your pr

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18926 **[Test build #80544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80544/testReport)** for PR 18926 at commit [`753dbe1`](https://github.com/apache/spark/commit/75

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/18925 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if th

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/18925 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18918: [SPARK-21707][SQL]Improvement a special case for non-det...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18918 **[Test build #80545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80545/testReport)** for PR 18918 at commit [`b79b9af`](https://github.com/apache/spark/commit/b7

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread joseph-torres
Github user joseph-torres commented on the issue: https://github.com/apache/spark/pull/18925 @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18918: [SPARK-21707][SQL]Improvement a special case for non-det...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18918 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18918: [SPARK-21707][SQL]Improvement a special case for non-det...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18918 How about data source tables? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request #18926: [SPARK-21712] [PySpark] Clarify type error for Co...

2017-08-11 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/18926 [SPARK-21712] [PySpark] Clarify type error for Column.substr() Proposed changes: * Clarify the type error that `Column.substr()` gives. Test plan: * Tested this manually. * Tes

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-11 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18901 Generally its expected that where you launch your job from has all the necessary jars already, thus all the jar would be available for the Client. here we download all jars that the user expli

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18921 Hm, looks like MiMa is running out of memory for some reason now. I'm not so familiar with the `dev/mima` script but maybe the sbt call there can just increase the heap size? --- If your project is

[GitHub] spark pull request #18914: [MINOR][SQL][TEST]no uncache table in joinsuite t...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18914#discussion_r132754247 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala --- @@ -141,6 +141,7 @@ class JoinSuite extends QueryTest with SharedSQLContext {

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18921 **[Test build #3888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3888/testReport)** for PR 18921 at commit [`b15334e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80543/ Test FAILed. ---

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80543/testReport)** for PR 18907 at commit [`ec8b465`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18724 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark pull request #18849: [SPARK-21617][SQL] Store correct table metadata w...

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18849#discussion_r132751607 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -1175,6 +1205,27 @@ private[spark] class HiveExternalCatalog(con

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-11 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/18899 Ok, it's fairly safe since it's limited to `private[linalg]`. The confusion for me is that this method introduces all sorts of edge cases which have behavior that is not at all obvious or clear. If w

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18925 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread joseph-torres
Github user joseph-torres commented on the issue: https://github.com/apache/spark/pull/18925 @tdas - please review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18900 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #18925: [SPARK-21713][SC] Replace streaming bit with Outp...

2017-08-11 Thread joseph-torres
GitHub user joseph-torres opened a pull request: https://github.com/apache/spark/pull/18925 [SPARK-21713][SC] Replace streaming bit with OutputMode ## What changes were proposed in this pull request? * Replace LogicalPlan.isStreaming with output mode. * Replace Deduplica

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18900 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80528/ Test FAILed. ---

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18900 **[Test build #80528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80528/testReport)** for PR 18900 at commit [`c833ce7`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80543/testReport)** for PR 18907 at commit [`ec8b465`](https://github.com/apache/spark/commit/ec

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18921 **[Test build #3888 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3888/testReport)** for PR 18921 at commit [`b15334e`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18872 @ProtD this needs a JIRA or else needs to be linked to whatever one you opened, in the title --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18924 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80524/ Test PASSed. ---

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18924 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18924 **[Test build #80524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80524/testReport)** for PR 18924 at commit [`f81f1cd`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80527/ Test FAILed. ---

[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80527/testReport)** for PR 18904 at commit [`b349668`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18921 **[Test build #3887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3887/testReport)** for PR 18921 at commit [`b15334e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18872 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80533/ Test PASSed. ---

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18872 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18872 **[Test build #80533 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80533/testReport)** for PR 18872 at commit [`38da77d`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #18421: [SPARK-21213][SQL] Support collecting partition-level st...

2017-08-11 Thread mbasmanova
Github user mbasmanova commented on the issue: https://github.com/apache/spark/pull/18421 @gatorsmile , thank you for reviewing this PR. I'm exciting that it is getting very close to be merged. It is not obvious to me why would you prefer to follow Hive and restrict the user

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132744793 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self): JavaParams._new_java_obj("org.apache.spark.ml.Pipelin

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132744462 --- Diff: python/pyspark/ml/pipeline.py --- @@ -16,14 +16,15 @@ # import sys +import os if sys.version > '3': bases

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132599249 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self): JavaParams._new_java_obj("org.apache.spark.ml.Pipelin

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132742064 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self): JavaParams._new_java_obj("org.apache.spark.ml.Pipelin

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132743731 --- Diff: python/pyspark/ml/tests.py --- @@ -1142,6 +1142,35 @@ def test_nested_pipeline_persistence(self): except OSError:

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132744343 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self): JavaParams._new_java_obj("org.apache.spark.ml.Pipelin

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132599646 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self): JavaParams._new_java_obj("org.apache.spark.ml.Pipelin

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132744158 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +327,65 @@ def _to_java(self): JavaParams._new_java_obj("org.apache.spark.ml.Pipelin

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132599624 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self): JavaParams._new_java_obj("org.apache.spark.ml.Pipelin

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18901 **[Test build #80542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80542/testReport)** for PR 18901 at commit [`24c80d1`](https://github.com/apache/spark/commit/24

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18921 **[Test build #3887 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3887/testReport)** for PR 18921 at commit [`b15334e`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18901 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18756 @lvdongr We need to introduce the concept of `default value` to our schema at first. Could you hold this PR until we finish that? Thanks! --- If your project is set up for it, you can reply to t

[GitHub] spark pull request #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18519#discussion_r132741326 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -53,9 +53,16 @@ pr

[GitHub] spark pull request #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18519#discussion_r132741250 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -53,9 +53,16 @@ pr

[GitHub] spark pull request #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18519#discussion_r132740552 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -41,9 +45,11 @@ import org.apache.spark.util.{Rp

[GitHub] spark pull request #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18519#discussion_r132741194 --- Diff: resource-managers/mesos/pom.xml --- @@ -74,6 +74,17 @@ test + --- End diff -- Is this really needed?

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132740595 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -370,6 +370,14 @@ case class WholeStageCodegenExec(c

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132740569 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging o

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18907 @shaneknapp thanks for quick response! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18887 hmm... @squito @ajbozarth @jerryshao --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-08-11 Thread justinuang
Github user justinuang commented on the issue: https://github.com/apache/spark/pull/15009 @kishorvpatil this will be quite useful for us! To avoid the 3s cost of spinning up a new jvm just for yarn-cluster --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark pull request #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for st...

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18630#discussion_r132738772 --- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala --- @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80526/ Test FAILed. ---

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80526/testReport)** for PR 18907 at commit [`3820442`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18756 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18756 **[Test build #80538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80538/testReport)** for PR 18756 at commit [`adbb8ee`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18756 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80538/ Test FAILed. ---

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18899 This isn't a public method though. The dv.toSparseWithSize(2) error will never come up unless Spark causes it, and there's no contract for its behavior in that case. It's probably over-specifying thi

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-11 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/18899 I think there _is_ new functionality, a new method that needs its functionality defined. One specific example, we need a test like: scala test("toSparseWithSize") { val dv =

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2017-08-11 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/18903 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132731370 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging or -1

[GitHub] spark issue #18895: [SPARK-21658][SQL][PYSPARK] Add default None for value i...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18895 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/18907 thanks for the heads up @hvanhovell -- looks like there was some gunk in the pipes and now we've got ~10 pull request builds running. :) --- If your project is set up for it, you can reply to t

[GitHub] spark issue #18895: [SPARK-21658][SQL][PYSPARK] Add default None for value i...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18895 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80530/ Test FAILed. ---

[GitHub] spark issue #18895: [SPARK-21658][SQL][PYSPARK] Add default None for value i...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18895 **[Test build #80530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80530/testReport)** for PR 18895 at commit [`5d8d4b6`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #80541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80541/testReport)** for PR 16985 at commit [`7171b58`](https://github.com/apache/spark/commit/71

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18756 **[Test build #80538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80538/testReport)** for PR 18756 at commit [`adbb8ee`](https://github.com/apache/spark/commit/ad

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18899 **[Test build #80529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80529/testReport)** for PR 18899 at commit [`4404b10`](https://github.com/apache/spark/commit/44

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18846 **[Test build #80535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80535/testReport)** for PR 18846 at commit [`afc07ee`](https://github.com/apache/spark/commit/af

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18724 **[Test build #80539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80539/testReport)** for PR 18724 at commit [`5792fd6`](https://github.com/apache/spark/commit/57

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #80540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80540/testReport)** for PR 18555 at commit [`3135235`](https://github.com/apache/spark/commit/31

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18900 **[Test build #80528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80528/testReport)** for PR 18900 at commit [`c833ce7`](https://github.com/apache/spark/commit/c8

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #80532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80532/testReport)** for PR 18875 at commit [`f83a110`](https://github.com/apache/spark/commit/f8

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18872 **[Test build #80533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80533/testReport)** for PR 18872 at commit [`38da77d`](https://github.com/apache/spark/commit/38

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80537/testReport)** for PR 18810 at commit [`32813b0`](https://github.com/apache/spark/commit/32

[GitHub] spark issue #18855: [SPARK-3151] [Block Manager] DiskStore.getBytes fails fo...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18855 **[Test build #80534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80534/testReport)** for PR 18855 at commit [`6cbe8d0`](https://github.com/apache/spark/commit/6c

[GitHub] spark issue #18843: [SPARK-21595] Separate thresholds for buffering and spil...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18843 **[Test build #80536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80536/testReport)** for PR 18843 at commit [`ab5cd2e`](https://github.com/apache/spark/commit/ab

[GitHub] spark issue #18895: [SPARK-21658][SQL][PYSPARK] Add default None for value i...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18895 **[Test build #80530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80530/testReport)** for PR 18895 at commit [`5d8d4b6`](https://github.com/apache/spark/commit/5d

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80526/testReport)** for PR 18907 at commit [`3820442`](https://github.com/apache/spark/commit/38

[GitHub] spark issue #18892: [SPARK-21520][SQL]Improvement a special case for non-det...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18892 **[Test build #80531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80531/testReport)** for PR 18892 at commit [`1ee1a76`](https://github.com/apache/spark/commit/1e

[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80527/testReport)** for PR 18904 at commit [`b349668`](https://github.com/apache/spark/commit/b3

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80525/testReport)** for PR 18914 at commit [`b537ce0`](https://github.com/apache/spark/commit/b5

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18924 **[Test build #80524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80524/testReport)** for PR 18924 at commit [`f81f1cd`](https://github.com/apache/spark/commit/f8

[GitHub] spark issue #18916: [SPARK-21705][CORE][DOC]Add spark.internal.config parame...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18916 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #18915: [SPARK-21176][WEB UI] Format worker page links to work w...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18915 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #18922: [SPARK-21701][CORE] Enable RPC client to use SO_RCVBUF, ...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18922 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

<    1   2   3   4   >