[GitHub] spark pull request #18900: [SPARK-21687][SQL] Spark SQL should set createTim...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18900#discussion_r132727520 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -97,7 +97,9 @@ object CatalogStorageFormat {

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-11 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/18899 Ok, it's fairly safe since it's limited to `private[linalg]`. The confusion for me is that this method introduces all sorts of edge cases which have behavior that is not at all obvious or clear. If

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18907 **[Test build #80543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80543/testReport)** for PR 18907 at commit

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80543/ Test FAILed. ---

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18907 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread joseph-torres
Github user joseph-torres commented on the issue: https://github.com/apache/spark/pull/18925 @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #18903: [SPARK-21590][SS]Window start time should support...

2017-08-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/18903#discussion_r132729005 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/TimeWindowSuite.scala --- @@ -77,6 +77,19 @@ class TimeWindowSuite extends

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18899 This isn't a public method though. The dv.toSparseWithSize(2) error will never come up unless Spark causes it, and there's no contract for its behavior in that case. It's probably over-specifying

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-08-11 Thread justinuang
Github user justinuang commented on the issue: https://github.com/apache/spark/pull/15009 @kishorvpatil this will be quite useful for us! To avoid the 3s cost of spinning up a new jvm just for yarn-cluster --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132740569 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132740595 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -370,6 +370,14 @@ case class

[GitHub] spark pull request #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18519#discussion_r132740552 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -41,9 +45,11 @@ import

[GitHub] spark pull request #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18519#discussion_r132741194 --- Diff: resource-managers/mesos/pom.xml --- @@ -74,6 +74,17 @@ test + --- End diff -- Is this really

[GitHub] spark pull request #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18519#discussion_r132741326 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -53,9 +53,16 @@

[GitHub] spark pull request #18519: [SPARK-16742] Mesos Kerberos Support

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18519#discussion_r132741250 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -53,9 +53,16 @@

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18724 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18901 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18924 **[Test build #80524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80524/testReport)** for PR 18924 at commit

[GitHub] spark issue #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not coll...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18924 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80524/ Test PASSed. ---

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18921 **[Test build #3888 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3888/testReport)** for PR 18921 at commit

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18872 @ProtD this needs a JIRA or else needs to be linked to whatever one you opened, in the title --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18724 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80539/ Test PASSed. ---

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18724 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/18907 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/18907 sometimes jobs don't like to trigger and there's nothing in the logs as to exactly why. since nothing was building, i decided to kick jenkins and then retrigger this build. --- If your

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/18907 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18810 @cloud-fan I am thinking about how we can split a super long function into multiple function as `CodeGenerator.splitExpressions` does. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should n...

2017-08-11 Thread akopich
GitHub user akopich opened a pull request: https://github.com/apache/spark/pull/18924 [SPARK-14371] [MLLIB] OnlineLDAOptimizer should not collect stats for each doc in mini-batch to driver Hi, as it was proposed by Joseph K. Bradley, gammat are not collected to the

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/18887 hmm... @squito @ajbozarth @jerryshao --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132744343 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self):

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132599249 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self):

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132742064 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self):

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132744462 --- Diff: python/pyspark/ml/pipeline.py --- @@ -16,14 +16,15 @@ # import sys +import os if sys.version > '3':

[GitHub] spark issue #18918: [SPARK-21707][SQL]Improvement a special case for non-det...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18918 How about data source tables? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18926: [SPARK-21712] [PySpark] Clarify type error for Co...

2017-08-11 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/18926 [SPARK-21712] [PySpark] Clarify type error for Column.substr() Proposed changes: * Clarify the type error that `Column.substr()` gives. Test plan: * Tested this manually. *

[GitHub] spark issue #18918: [SPARK-21707][SQL]Improvement a special case for non-det...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18918 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18888: [Spark-17025][ML][Python] Persistence for Pipelines with...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #80547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80547/testReport)** for PR 1 at commit

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18907 restest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18907 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18907 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #18810: [SPARK-21603][SQL]The wholestage codegen will be ...

2017-08-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18810#discussion_r132731370 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -572,6 +572,14 @@ object SQLConf { "disable logging or -1

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2017-08-11 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/18903 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18756 @lvdongr We need to introduce the concept of `default value` to our schema at first. Could you hold this PR until we finish that? Thanks! --- If your project is set up for it, you can reply to

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18901 **[Test build #80542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80542/testReport)** for PR 18901 at commit

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18872 **[Test build #80533 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80533/testReport)** for PR 18872 at commit

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18872 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18872: [MLlib] Fix writing LibSVM (key not found: numFeatures)

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18872 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80533/ Test PASSed. ---

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18921 **[Test build #3887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3887/testReport)** for PR 18921 at commit

[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80527/testReport)** for PR 18904 at commit

[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80527/ Test FAILed. ---

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18900 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80528/ Test FAILed. ---

[GitHub] spark pull request #18925: [SPARK-21713][SC] Replace streaming bit with Outp...

2017-08-11 Thread joseph-torres
GitHub user joseph-torres opened a pull request: https://github.com/apache/spark/pull/18925 [SPARK-21713][SC] Replace streaming bit with OutputMode ## What changes were proposed in this pull request? * Replace LogicalPlan.isStreaming with output mode. * Replace

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18900 **[Test build #80528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80528/testReport)** for PR 18900 at commit

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18920 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18922: [SPARK-21701][CORE] Enable RPC client to use SO_RCVBUF, ...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18922 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18917: [SPARK-21704][DOC] Add the description of 'sbin/stop-sla...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18917 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18915: [SPARK-21176][WEB UI] Format worker page links to work w...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18915 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18918: [SPARK-21707][SQL]Improvement a special case for non-det...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18918 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18921 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18923: [SPARK-21710][StSt] Fix OOM on ConsoleSink with large in...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18923 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18756 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18756 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80538/ Test FAILed. ---

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18756 **[Test build #80538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80538/testReport)** for PR 18756 at commit

[GitHub] spark issue #18421: [SPARK-21213][SQL] Support collecting partition-level st...

2017-08-11 Thread mbasmanova
Github user mbasmanova commented on the issue: https://github.com/apache/spark/pull/18421 @gatorsmile , thank you for reviewing this PR. I'm exciting that it is getting very close to be merged. It is not obvious to me why would you prefer to follow Hive and restrict the

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132744793 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self):

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/18925 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/18925 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18918: [SPARK-21707][SQL]Improvement a special case for non-det...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18918 **[Test build #80545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80545/testReport)** for PR 18918 at commit

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18926 **[Test build #80544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80544/testReport)** for PR 18926 at commit

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18900 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2017-08-11 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/18903 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18895: [SPARK-21658][SQL][PYSPARK] Add default None for value i...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18895 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/18907 thanks for the heads up @hvanhovell -- looks like there was some gunk in the pipes and now we've got ~10 pull request builds running. :) --- If your project is set up for it, you can reply to

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #80541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80541/testReport)** for PR 16985 at commit

[GitHub] spark issue #18895: [SPARK-21658][SQL][PYSPARK] Add default None for value i...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18895 **[Test build #80530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80530/testReport)** for PR 18895 at commit

[GitHub] spark issue #18895: [SPARK-21658][SQL][PYSPARK] Add default None for value i...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18895 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80530/ Test FAILed. ---

[GitHub] spark pull request #18630: [SPARK-12559][SPARK SUBMIT] fix --packages for st...

2017-08-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18630#discussion_r132738772 --- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala --- @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-11 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18901 Note I'll file a separate jira for this but even in client mode it will download and then reupload the file to hdfs for yarn mode, thus defeating the purpose of it being in hdfs. --- If your

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18925 **[Test build #80546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80546/testReport)** for PR 18925 at commit

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18724 **[Test build #80539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80539/testReport)** for PR 18724 at commit

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/18907 @shaneknapp thanks for quick response! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18921 **[Test build #3887 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3887/testReport)** for PR 18921 at commit

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132743731 --- Diff: python/pyspark/ml/tests.py --- @@ -1142,6 +1142,35 @@ def test_nested_pipeline_persistence(self): except OSError:

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132599646 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self):

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132744158 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +327,65 @@ def _to_java(self):

[GitHub] spark pull request #18888: [Spark-17025][ML][Python] Persistence for Pipelin...

2017-08-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r132599624 --- Diff: python/pyspark/ml/pipeline.py --- @@ -242,3 +319,74 @@ def _to_java(self):

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18900 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18925 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18925: [SPARK-21713][SC] Replace streaming bit with OutputMode

2017-08-11 Thread joseph-torres
Github user joseph-torres commented on the issue: https://github.com/apache/spark/pull/18925 @tdas - please review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18724: [SPARK-21519][SQL] Add an option to the JDBC data source...

2017-08-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18724 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18921: [SPARK-21709][Build] sbt 0.13.16 and some plugin updates

2017-08-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18921 Hm, looks like MiMa is running out of memory for some reason now. I'm not so familiar with the `dev/mima` script but maybe the sbt call there can just increase the heap size? --- If your project

[GitHub] spark issue #18901: [SPARK-21689][YARN] Download user jar from remote in cas...

2017-08-11 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/18901 Generally its expected that where you launch your job from has all the necessary jars already, thus all the jar would be available for the Client. here we download all jars that the user

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-08-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15009 I kinda have up waiting for feedback to be addressed here. I'm fixing some things in the launcher library and will work on SPARK-11035 next, replacing part of this PR. --- If your project is set

[GitHub] spark pull request #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added...

2017-08-11 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16985#discussion_r132620278 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ReorderJoinPredicates.scala --- @@ -0,0 +1,93 @@ +/* + * Licensed to

[GitHub] spark issue #18915: [SPARK-21176][WEB UI] Format worker page links to work w...

2017-08-11 Thread aosagie
Github user aosagie commented on the issue: https://github.com/apache/spark/pull/18915 Hey @cloud-fan and @ajbozarth Thanks for checking my last PR. Any chance I could get one more look at this clean up? Some of the existing link handling was different between the Applications

[GitHub] spark pull request #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added...

2017-08-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16985#discussion_r132624167 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala --- @@ -543,6 +551,68 @@ abstract class BucketedReadSuite extends

[GitHub] spark pull request #18916: [CORE][DOC]Add spark.internal.config parameter de...

2017-08-11 Thread heary-cao
GitHub user heary-cao opened a pull request: https://github.com/apache/spark/pull/18916 [CORE][DOC]Add spark.internal.config parameter description ## What changes were proposed in this pull request? Currently, some of the configuration parameters of spark.internal.config

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2017-08-11 Thread debugger87
Github user debugger87 commented on the issue: https://github.com/apache/spark/pull/18900 My changes is not enough to support `createTime` in CatalogTablePartition, I will check and re-commit again. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18843: [SPARK-21595] Separate thresholds for buffering and spil...

2017-08-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18843 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

  1   2   3   4   >