[GitHub] spark issue #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Redact se...

2017-08-07 Thread dmvieira
Github user dmvieira commented on the issue: https://github.com/apache/spark/pull/18802 Thank you @vanzin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Re...

2017-08-07 Thread dmvieira
Github user dmvieira closed the pull request at: https://github.com/apache/spark/pull/18802 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #80372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80372/testReport)** for PR 18875 at commit

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131809065 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -596,14 +596,15 @@ object CollapseProject extends

[GitHub] spark pull request #18813: [SPARK-21567][SQL] Dataset should work with type ...

2017-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18813#discussion_r131814856 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -34,6 +34,11 @@ import org.apache.spark.sql.types._ case class

[GitHub] spark issue #18813: [SPARK-21567][SQL] Dataset should work with type alias

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18813 **[Test build #80377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80377/testReport)** for PR 18813 at commit

[GitHub] spark issue #18492: [SPARK-19326] Speculated task attempts do not get launch...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18492 **[Test build #80378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80378/testReport)** for PR 18492 at commit

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-07 Thread bravo-zhang
Github user bravo-zhang commented on the issue: https://github.com/apache/spark/pull/18820 Hi @HyukjinKwon @gatorsmile @viirya I addressed your comments, added more test coverage and provided more info in PR description. One thing that is not clear to user is that they can still

[GitHub] spark pull request #18862: [SPARK-21640][FOLLOW-UP][SQL] added errorifexists...

2017-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18862 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18861 **[Test build #80373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80373/testReport)** for PR 18861 at commit

[GitHub] spark pull request #18870: [SPARK-19270][FOLLOW-UP][ML] PySpark GLR model.su...

2017-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18870 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17373: [SPARK-12664] Expose probability in mlp model

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17373 **[Test build #80371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80371/testReport)** for PR 17373 at commit

[GitHub] spark issue #17373: [SPARK-12664] Expose probability in mlp model

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17373 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80371/ Test PASSed. ---

[GitHub] spark issue #17373: [SPARK-12664] Expose probability in mlp model

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17373 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18799: [SPARK-21596][SS]Ensure places calling HDFSMetada...

2017-08-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/18799#discussion_r131804073 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala --- @@ -186,7 +190,11 @@ abstract class

[GitHub] spark issue #18813: [SPARK-21567][SQL] Dataset should work with type alias

2017-08-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18813 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18837: [Spark-20812][Mesos] Add secrets support to the d...

2017-08-07 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/18837#discussion_r131806416 --- Diff: docs/running-on-mesos.md --- @@ -479,6 +479,35 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r131808674 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18652 @gatorsmile @cloud-fan Do you have more comments or thoughts on this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18687: [SPARK-21484][SQL] Fix inconsistent query plans of Datas...

2017-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18687 ping @cloud-fan Please help review this too. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18865 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r131820476 --- Diff: core/src/main/java/org/apache/spark/SparkFirehoseListener.java --- @@ -140,6 +140,16 @@ public void onBlockUpdated(SparkListenerBlockUpdated

[GitHub] spark issue #18813: [SPARK-21567][SQL] Dataset should work with type alias

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18813 **[Test build #80374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80374/testReport)** for PR 18813 at commit

[GitHub] spark issue #18813: [SPARK-21567][SQL] Dataset should work with type alias

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18813 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18818: [SPARK-21110][SQL] Structs, arrays, and other ord...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18818#discussion_r131824915 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -453,6 +453,14 @@ case class Or(left:

[GitHub] spark issue #18862: [SPARK-21640][FOLLOW-UP][SQL] added errorifexists on Ill...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18862 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18813: [SPARK-21567][SQL] Dataset should work with type ...

2017-08-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18813#discussion_r131803127 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -775,7 +775,7 @@ object ScalaReflection extends

[GitHub] spark issue #18813: [SPARK-21567][SQL] Dataset should work with type alias

2017-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18813 @cloud-fan I identified only one place `getClassFromType` we can remove the dealias. As there is only one and it's a public method, I prefer to keep it in case the future caller doesn't do dealias,

[GitHub] spark pull request #18799: [SPARK-21596][SS]Ensure places calling HDFSMetada...

2017-08-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/18799#discussion_r131804168 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -396,6 +400,7 @@ object HDFSMetadataLog {

[GitHub] spark pull request #18648: [SPARK-21428] Turn IsolatedClientLoader off while...

2017-08-07 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/18648#discussion_r131807040 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala --- @@ -312,7 +323,7 @@ private[spark] object HiveUtils extends Logging {

[GitHub] spark issue #14326: [SPARK-3181] [ML] Implement RobustRegression with huber ...

2017-08-07 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14326 @WeichenXu123 Thanks for your comment. I will update my PR ASAP. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-07 Thread bravo-zhang
Github user bravo-zhang commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131814622 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/types/DataTypeSuite.scala --- @@ -145,8 +145,8 @@ class DataTypeSuite extends SparkFunSuite

[GitHub] spark pull request #18648: [SPARK-21428] Turn IsolatedClientLoader off while...

2017-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18648#discussion_r131814534 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -269,6 +232,8 @@ private[hive] class HiveClientImpl(

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-07 Thread bravo-zhang
Github user bravo-zhang commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131814746 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -314,6 +316,7 @@ final class DataFrameNaFunctions

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18652 > As said in previous discussion, we can't avoid few issues regarding non-deterministic non equi join condition. We can simply allow it, but it faces inconsistency due to different join

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-07 Thread bravo-zhang
Github user bravo-zhang commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131817518 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -366,11 +370,15 @@ final class DataFrameNaFunctions

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r131818512 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -572,20 +572,35 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r131819639 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -53,6 +53,12 @@ case class SparkListenerTaskStart(stageId: Int,

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r131820095 --- Diff: core/src/main/java/org/apache/spark/SparkFirehoseListener.java --- @@ -140,6 +140,16 @@ public void onBlockUpdated(SparkListenerBlockUpdated

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18865 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80375/ Test PASSed. ---

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18865 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18875 @goldmedal, before proceeding the review, assuming from the PR description, I guess your PR targets SQL and Scala APIs for now, right? --- If your project is set up for it, you can reply to

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18875 @HyukjinKwon Yeah, we focus on SQL and Scala APIs for now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #18813: [SPARK-21567][SQL] Dataset should work with type ...

2017-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18813#discussion_r131803516 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -34,6 +34,11 @@ import org.apache.spark.sql.types._ case class

[GitHub] spark pull request #18813: [SPARK-21567][SQL] Dataset should work with type ...

2017-08-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18813#discussion_r131806778 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -34,6 +34,11 @@ import org.apache.spark.sql.types._ case class

[GitHub] spark pull request #18813: [SPARK-21567][SQL] Dataset should work with type ...

2017-08-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18813#discussion_r131806723 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -34,6 +34,11 @@ import org.apache.spark.sql.types._ case class

[GitHub] spark issue #18799: [SPARK-21596][SS]Ensure places calling HDFSMetadataLog.g...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18799 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80370/ Test PASSed. ---

[GitHub] spark issue #18799: [SPARK-21596][SS]Ensure places calling HDFSMetadataLog.g...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18799 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18799: [SPARK-21596][SS]Ensure places calling HDFSMetadataLog.g...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18799 **[Test build #80370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80370/testReport)** for PR 18799 at commit

[GitHub] spark issue #18878: fix issue with spark-submit with java

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18878 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18861#discussion_r131808426 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -746,8 +746,20 @@ abstract class

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18865 Oh. sorry. Looks like the Jenkins run tests now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18865 **[Test build #80375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80375/testReport)** for PR 18865 at commit

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18865 @HyukjinKwon @cloud-fan May you help trigger Jenkins? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #18648: [SPARK-21428] Turn IsolatedClientLoader off while...

2017-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18648#discussion_r131814407 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveCliSessionStateSuite.scala --- @@ -0,0 +1,57 @@ +/* +

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80372/ Test PASSed. ---

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18802: [SPARK-18535][SPARK-19720][CORE][BACKPORT-2.1] Redact se...

2017-08-07 Thread markgrover
Github user markgrover commented on the issue: https://github.com/apache/spark/pull/18802 Thanks @dmvieira and @vanzin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread dmvieira
Github user dmvieira commented on the issue: https://github.com/apache/spark/pull/18873 It doesn't make sense... If you see all failed builds they're failing because doesn't support python 2.6... I think your CI is running python 2.6 in some machines and python 2.7 or higher in

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18875 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18875 yea this looks in the right direction and I guess a good reference should be https://github.com/goldmedal/spark/commit/0cdcf9114527a2c359c25e46fd6556b3855bfb28 --- If your project is set up

[GitHub] spark issue #18873: Fixing python 2.6 tests for jenkings

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18873 I think we are supposed to run the tests in Python 2.7 but it looks mistakenly used Python 2.6 for a while, which I guess caused the test failures, see

[GitHub] spark issue #18763: [SPARK-21306][ML] For branch 2.1, OneVsRest should suppo...

2017-08-07 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18763 Merged into branch-2.1. Thanks @facaiy @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18764: [SPARK-21306][ML] For branch 2.0, OneVsRest should suppo...

2017-08-07 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18764 Merged into branch-2.0. Thanks @facaiy @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #18179: [SPARK-20894][SS] Resolve the checkpoint location in dri...

2017-08-07 Thread markgrover
Github user markgrover commented on the issue: https://github.com/apache/spark/pull/18179 Sounds good to me. The JIRA was unresolved. I have resolved it with a fix version of 2.3.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #18813: [SPARK-21567][SQL] Dataset should work with type ...

2017-08-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18813#discussion_r131818435 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -34,6 +34,11 @@ import org.apache.spark.sql.types._ case class

[GitHub] spark issue #18865: [SPARK-21610][SQL] Corrupt records are not handled prope...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18865 **[Test build #80375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80375/testReport)** for PR 18865 at commit

[GitHub] spark issue #18492: [SPARK-19326] Speculated task attempts do not get launch...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18492 **[Test build #80380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80380/testReport)** for PR 18492 at commit

[GitHub] spark issue #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an emergen...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18797 **[Test build #80381 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80381/testReport)** for PR 18797 at commit

[GitHub] spark issue #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an emergen...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18797 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80369/ Test FAILed. ---

[GitHub] spark issue #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an emergen...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18797 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an emergen...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18797 **[Test build #80369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80369/testReport)** for PR 18797 at commit

[GitHub] spark issue #18877: [SPARK-17742][core] Handle child process exit in SparkLa...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18877 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80368/ Test PASSed. ---

[GitHub] spark issue #18877: [SPARK-17742][core] Handle child process exit in SparkLa...

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18877 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18877: [SPARK-17742][core] Handle child process exit in SparkLa...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18877 **[Test build #80368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80368/testReport)** for PR 18877 at commit

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/18875 @HyukjinKwon Thanks for the reference! I'll check it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #18878: fix issue with spark-submit with java

2017-08-07 Thread brandonJY
GitHub user brandonJY opened a pull request: https://github.com/apache/spark/pull/18878 fix issue with spark-submit with java ## What changes were proposed in this pull request? remove quote when specifying class using spark-submit in command line. Specifying java class

[GitHub] spark issue #18813: [SPARK-21567][SQL] Dataset should work with type alias

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18813 **[Test build #80374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80374/testReport)** for PR 18813 at commit

[GitHub] spark issue #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18861 **[Test build #80373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80373/testReport)** for PR 18861 at commit

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18652 **[Test build #80376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80376/testReport)** for PR 18652 at commit

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #80372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80372/testReport)** for PR 18875 at commit

[GitHub] spark pull request #18648: [SPARK-21428] Turn IsolatedClientLoader off while...

2017-08-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18648#discussion_r131814291 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -581,17 +581,15 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to r...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18820#discussion_r131815429 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/types/DataTypeSuite.scala --- @@ -145,8 +145,8 @@ class DataTypeSuite extends SparkFunSuite

[GitHub] spark issue #18820: [SPARK-14932][SQL] Allow DataFrame.replace() to replace ...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18820 **[Test build #80379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80379/testReport)** for PR 18820 at commit

[GitHub] spark pull request #17373: [SPARK-12664] Expose probability in mlp model

2017-08-07 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/17373#discussion_r131824713 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -82,6 +83,23 @@ class

[GitHub] spark issue #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18861 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18861 @gatorsmile ok, fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18861: [SPARK-19426][SQL] Custom coalescer for Dataset

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18861 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80373/ Test PASSed. ---

[GitHub] spark pull request #18813: [SPARK-21567][SQL] Dataset should work with type ...

2017-08-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18813#discussion_r131795429 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -864,7 +865,7 @@ trait ScalaReflection { }

[GitHub] spark pull request #18813: [SPARK-21567][SQL] Dataset should work with type ...

2017-08-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18813#discussion_r131795156 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -848,9 +848,10 @@ trait ScalaReflection { *

[GitHub] spark pull request #18865: [SPARK-21610][SQL] Corrupt records are not handle...

2017-08-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18865#discussion_r131796980 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -114,7 +114,16 @@ class JsonFileFormat

[GitHub] spark issue #18870: [SPARK-19270][FOLLOW-UP][ML] PySpark GLR model.summary s...

2017-08-07 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18870 Merged into master. Thanks for reviewing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18799: [SPARK-21596][SS]Ensure places calling HDFSMetada...

2017-08-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/18799#discussion_r131798109 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala --- @@ -169,7 +169,11 @@ abstract class

[GitHub] spark pull request #18818: [SPARK-21110][SQL] Structs, arrays, and other ord...

2017-08-07 Thread aray
Github user aray commented on a diff in the pull request: https://github.com/apache/spark/pull/18818#discussion_r131808912 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -453,6 +453,14 @@ case class Or(left: Expression,

[GitHub] spark issue #18764: [SPARK-21306][ML] For branch 2.0, OneVsRest should suppo...

2017-08-07 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18764 It seems github can't close this PR automatically. @facaiy Please feel free to close it manually. Thanks. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #18763: [SPARK-21306][ML] For branch 2.1, OneVsRest should suppo...

2017-08-07 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18763 It seems github can't close this PR automatically. @facaiy Please feel free to close it manually. Thanks. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #18813: [SPARK-21567][SQL] Dataset should work with type alias

2017-08-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18813 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80374/ Test PASSed. ---

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-07 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r131822909 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -53,6 +53,12 @@ case class SparkListenerTaskStart(stageId: Int,

[GitHub] spark pull request #18790: [SPARK-21587][SS] Added pushdown through watermar...

2017-08-07 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/18790#discussion_r131761258 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -872,6 +886,25 @@ object PushDownPredicate

[GitHub] spark issue #18874: [SPARK-21656][CORE] spark dynamic allocation should not ...

2017-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18874 **[Test build #80360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80360/testReport)** for PR 18874 at commit

  1   2   3   4   5   6   >