[GitHub] spark issue #18322: [SPARK-21115][Core]If the cores left is less than the co...

2017-06-21 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18322 @jiangxb1987 can you please help to review this PR? This is a simple code improvement to avoid some unnecessary code execution when left cores is not enough for one executor. I don't

[GitHub] spark issue #17113: [SPARK-13669][SPARK-20898][Core] Improve the blacklist m...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17113 **[Test build #78431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78431/testReport)** for PR 17113 at commit

[GitHub] spark issue #17113: [SPARK-13669][SPARK-20898][Core] Improve the blacklist m...

2017-06-21 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17113 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #9518: [SPARK-11574][Core] Add metrics StatsD sink

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/9518#discussion_r123420182 --- Diff: core/src/main/scala/org/apache/spark/metrics/sink/StatsdReporter.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #9518: [SPARK-11574][Core] Add metrics StatsD sink

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/9518#discussion_r123425292 --- Diff: core/src/main/scala/org/apache/spark/metrics/sink/StatsdReporter.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #9518: [SPARK-11574][Core] Add metrics StatsD sink

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/9518#discussion_r123424723 --- Diff: core/src/main/scala/org/apache/spark/metrics/sink/StatsdReporter.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #18371: [SPARK-20889][SparkR] Grouped documentation for MATH col...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18371 **[Test build #78430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78430/testReport)** for PR 18371 at commit

[GitHub] spark pull request #18378: [SPARK-21163][SQL] DataFrame.toPandas should resp...

2017-06-21 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18378#discussion_r123425404 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1721,7 +1721,14 @@ def toPandas(self): 15Bob """ import

[GitHub] spark pull request #18371: [SPARK-20889][SparkR] Grouped documentation for M...

2017-06-21 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/18371#discussion_r123425200 --- Diff: R/pkg/R/functions.R --- @@ -34,6 +34,30 @@ NULL #' df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))} NULL

[GitHub] spark issue #18371: [SPARK-20889][SparkR] Grouped documentation for MATH col...

2017-06-21 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18371 Made another commit that addresses your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #18371: [SPARK-20889][SparkR] Grouped documentation for M...

2017-06-21 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/18371#discussion_r123425179 --- Diff: R/pkg/R/functions.R --- @@ -1405,18 +1309,12 @@ setMethod("sha1", column(jc) }) -#' signum -#'

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18378 **[Test build #78429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78429/testReport)** for PR 18378 at commit

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18378 It sounds ok to me just except missing `_have_pandas = False` above `try:` . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #17519: [SPARK-15352][Doc] follow-up: add configuration d...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17519#discussion_r123424505 --- Diff: docs/configuration.md --- @@ -1004,14 +1004,48 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #17519: [SPARK-15352][Doc] follow-up: add configuration d...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17519#discussion_r123424493 --- Diff: docs/configuration.md --- @@ -1004,14 +1004,48 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18382 **[Test build #78428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78428/testReport)** for PR 18382 at commit

[GitHub] spark issue #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18382 Thank you @felixcheung. I checked docs and the same thing in the PR description. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request #18162: [SPARK-20923] turn tracking of TaskMetrics._updat...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18162#discussion_r123424121 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -295,4 +295,12 @@ package object config { "above this

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18378 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78427/ Test FAILed. ---

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18378 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18378 **[Test build #78427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78427/testReport)** for PR 18378 at commit

[GitHub] spark pull request #18162: [SPARK-20923] turn tracking of TaskMetrics._updat...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18162#discussion_r123424029 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala --- @@ -528,7 +528,13 @@ class JobProgressListener(conf: SparkConf)

[GitHub] spark issue #18377: [SPARK-18016][SQL][CATALYST][BRANCH-2.2] Code Generation...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18377 thanks, merging to 2.2! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18354: [SPARK-18016][SQL][CATALYST][BRANCH-2.1] Code Generation...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18354 thanks, merging to 2.1! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18362: [SPARK-20832][Core] Standalone master should expl...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18362#discussion_r123423356 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -454,6 +464,12 @@ class

[GitHub] spark pull request #18362: [SPARK-20832][Core] Standalone master should expl...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18362#discussion_r123423287 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -569,6 +569,12 @@ private[spark] class TaskSchedulerImpl

[GitHub] spark pull request #18362: [SPARK-20832][Core] Standalone master should expl...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18362#discussion_r123423207 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1432,6 +1439,26 @@ class DAGScheduler( } }

[GitHub] spark issue #18384: [SPARK-21170] [CORE] Utils.tryWithSafeFinallyAndFailureC...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18384 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18378 **[Test build #78427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78427/testReport)** for PR 18378 at commit

[GitHub] spark pull request #18362: [SPARK-20832][Core] Standalone master should expl...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18362#discussion_r123423005 --- Diff: core/src/main/scala/org/apache/spark/deploy/DeployMessage.scala --- @@ -158,6 +158,8 @@ private[deploy] object DeployMessages {

[GitHub] spark pull request #18384: [SPARK-21170] [CORE] Utils.tryWithSafeFinallyAndF...

2017-06-21 Thread devaraj-kavali
GitHub user devaraj-kavali opened a pull request: https://github.com/apache/spark/pull/18384 [SPARK-21170] [CORE] Utils.tryWithSafeFinallyAndFailureCallbacks throws IllegalArgumentException: Self-suppression not permitted ## What changes were proposed in this pull request?

[GitHub] spark pull request #18118: [SPARK-20199][ML] : Provided featureSubsetStrateg...

2017-06-21 Thread pralabhkumar
Github user pralabhkumar commented on a diff in the pull request: https://github.com/apache/spark/pull/18118#discussion_r123422892 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -192,6 +196,9 @@ object GBTClassifier extends

[GitHub] spark pull request #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls...

2017-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15417#discussion_r12342 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisTest.scala --- @@ -55,6 +55,14 @@ trait AnalysisTest extends

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18378 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78426/ Test FAILed. ---

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18378 **[Test build #78426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78426/testReport)** for PR 18378 at commit

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18378 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18378 **[Test build #78426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78426/testReport)** for PR 18378 at commit

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18118 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18118 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78420/ Test PASSed. ---

[GitHub] spark pull request #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18382#discussion_r123420890 --- Diff: R/pkg/R/context.R --- @@ -295,6 +295,23 @@ setCheckpointDirSC <- function(sc, dirName) { invisible(callJMethod(sc, "setCheckpointDir",

[GitHub] spark pull request #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18382#discussion_r123420666 --- Diff: R/pkg/NAMESPACE --- @@ -403,6 +403,7 @@ export("as.DataFrame", "refreshTable", "setCheckpointDir",

[GitHub] spark pull request #18322: [SPARK-21115][Core]If the cores left is less than...

2017-06-21 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18322#discussion_r123420930 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -258,23 +256,7 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark issue #18118: [SPARK-20199][ML] : Provided featureSubsetStrategy to GB...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18118 **[Test build #78420 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78420/testReport)** for PR 18118 at commit

[GitHub] spark pull request #18322: [SPARK-21115][Core]If the cores left is less than...

2017-06-21 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18322#discussion_r123420644 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -543,6 +545,42 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18300 **[Test build #78425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78425/testReport)** for PR 18300 at commit

[GitHub] spark issue #18347: [SPARK-20599][SS] ConsoleSink should work with (batch)

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18347 **[Test build #78424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78424/testReport)** for PR 18347 at commit

[GitHub] spark pull request #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18300#discussion_r123420282 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1764,6 +1765,70 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark pull request #18322: [SPARK-21115][Core]If the cores left is less than...

2017-06-21 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/18322#discussion_r123420083 --- Diff: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala --- @@ -704,6 +707,43 @@ class MasterSuite extends SparkFunSuite

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r123417186 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -47,9 +79,11 @@ while (TRUE) { close(inputCon) --- End diff -- please add comment

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r123416154 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,40 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r123416771 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,40 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark pull request #18320: [SPARK-21093][R] Terminate R's worker processes i...

2017-06-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18320#discussion_r123419616 --- Diff: R/pkg/inst/worker/daemon.R --- @@ -30,8 +30,40 @@ port <- as.integer(Sys.getenv("SPARKR_WORKER_PORT")) inputCon <- socketConnection(

[GitHub] spark pull request #18114: [SPARK-20889][SparkR] Grouped documentation for D...

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18114#discussion_r123419433 --- Diff: R/pkg/R/functions.R --- @@ -2414,20 +2396,23 @@ setMethod("from_json", signature(x = "Column", schema = "structType"),

[GitHub] spark pull request #18114: [SPARK-20889][SparkR] Grouped documentation for D...

2017-06-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18114#discussion_r123419080 --- Diff: R/pkg/R/functions.R --- @@ -2414,20 +2396,23 @@ setMethod("from_json", signature(x = "Column", schema = "structType"),

[GitHub] spark pull request #18329: [SPARK-19909][SS] Disabling the usage of a tempor...

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18329#discussion_r123418793 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -264,12 +281,12 @@ final class DataStreamWriter[T]

[GitHub] spark issue #18327: [SPARK-21047] Add test suites for complicated cases in C...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18327 **[Test build #78423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78423/testReport)** for PR 18327 at commit

[GitHub] spark pull request #18114: [SPARK-20889][SparkR] Grouped documentation for D...

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18114#discussion_r123416302 --- Diff: R/pkg/R/functions.R --- @@ -2414,20 +2396,23 @@ setMethod("from_json", signature(x = "Column", schema = "structType"),

[GitHub] spark pull request #18114: [SPARK-20889][SparkR] Grouped documentation for D...

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18114#discussion_r123416414 --- Diff: R/pkg/R/functions.R --- @@ -2414,20 +2396,23 @@ setMethod("from_json", signature(x = "Column", schema = "structType"),

[GitHub] spark pull request #18114: [SPARK-20889][SparkR] Grouped documentation for D...

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18114#discussion_r123417000 --- Diff: R/pkg/R/functions.R --- @@ -34,6 +34,58 @@ NULL #' df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))} NULL

[GitHub] spark pull request #18114: [SPARK-20889][SparkR] Grouped documentation for D...

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18114#discussion_r123416202 --- Diff: R/pkg/R/functions.R --- @@ -34,6 +34,58 @@ NULL #' df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))} NULL

[GitHub] spark pull request #18329: [SPARK-19909][SS] Disabling the usage of a tempor...

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18329#discussion_r123417606 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -235,6 +237,21 @@ final class DataStreamWriter[T]

[GitHub] spark pull request #18329: [SPARK-19909][SS] Disabling the usage of a tempor...

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18329#discussion_r123416837 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -17,8 +17,10 @@ package

[GitHub] spark pull request #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18300#discussion_r123416779 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1764,6 +1765,70 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark issue #17713: [SPARK-20417][SQL] Move subquery error handling to check...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17713 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78418/ Test PASSed. ---

[GitHub] spark issue #18228: [SPARK-21007][SQL]Add SQL function - RIGHT && LEFT

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18228 **[Test build #78422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78422/testReport)** for PR 18228 at commit

[GitHub] spark issue #17713: [SPARK-20417][SQL] Move subquery error handling to check...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17713 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17713: [SPARK-20417][SQL] Move subquery error handling to check...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17713 **[Test build #78418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78418/testReport)** for PR 17713 at commit

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18114 @felixcheung, would you give me a moment to double check? I am interested in this and want to help double check. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18382 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78421/ Test PASSed. ---

[GitHub] spark issue #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18382 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18382 **[Test build #78421 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78421/testReport)** for PR 18382 at commit

[GitHub] spark issue #18383: [SPARK-21167][SS] Set kafka clientId while fetch message...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18383 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18322: [SPARK-21115][Core]If the cores left is less than...

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18322#discussion_r123415327 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -543,6 +545,42 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request #18322: [SPARK-21115][Core]If the cores left is less than...

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18322#discussion_r123415688 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -258,23 +256,7 @@ private[deploy] class

[GitHub] spark pull request #18128: [SPARK-20906][SparkR]:Constrained Logistic Regres...

2017-06-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18128 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18128: [SPARK-20906][SparkR]:Constrained Logistic Regression fo...

2017-06-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18128 merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18383: [SPARK-21167][SS] Set kafka clientId while fetch ...

2017-06-21 Thread dijingran
GitHub user dijingran opened a pull request: https://github.com/apache/spark/pull/18383 [SPARK-21167][SS] Set kafka clientId while fetch messages ## What changes were proposed in this pull request? Change KafkaRDD to set kafka clientId while fetch messages, as in our case

[GitHub] spark issue #18114: [SPARK-20889][SparkR] Grouped documentation for DATETIME...

2017-06-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18114 hmm, waiting for AppVeyor --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18322: [SPARK-21115][Core]If the cores left is less than...

2017-06-21 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18322#discussion_r123413523 --- Diff: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala --- @@ -704,6 +707,43 @@ class MasterSuite extends SparkFunSuite

[GitHub] spark issue #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18382 **[Test build #78421 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78421/testReport)** for PR 18382 at commit

[GitHub] spark pull request #18382: [SPARK-21149][R] Add job description API for R

2017-06-21 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18382#discussion_r123412816 --- Diff: R/pkg/R/context.R --- @@ -295,6 +295,23 @@ setCheckpointDirSC <- function(sc, dirName) { invisible(callJMethod(sc, "setCheckpointDir",

[GitHub] spark issue #18362: [SPARK-20832][Core] Standalone master should explicitly ...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18362 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78415/ Test PASSed. ---

[GitHub] spark issue #18327: [SPARK-21047] Add test suites for complicated cases in C...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18327 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78416/ Test FAILed. ---

[GitHub] spark issue #18362: [SPARK-20832][Core] Standalone master should explicitly ...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18362 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18327: [SPARK-21047] Add test suites for complicated cases in C...

2017-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18327 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18362: [SPARK-20832][Core] Standalone master should explicitly ...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18362 **[Test build #78415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78415/testReport)** for PR 18362 at commit

[GitHub] spark issue #18327: [SPARK-21047] Add test suites for complicated cases in C...

2017-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18327 **[Test build #78416 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78416/testReport)** for PR 18327 at commit

[GitHub] spark pull request #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-21 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18300#discussion_r123412488 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1764,6 +1765,70 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123408655 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -1186,3 +1186,51 @@ case class BRound(child:

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123409892 --- Diff: sql/core/src/test/resources/sql-tests/inputs/operators.sql --- @@ -92,3 +92,8 @@ select abs(-3.13), abs('-2.19'); -- positive/negative

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123401825 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -631,3 +631,109 @@ abstract class TernaryExpression

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123410671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/MathUtils.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123411306 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/MathUtils.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123411090 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/MathUtils.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123410649 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/MathUtils.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123402665 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -631,3 +631,109 @@ abstract class TernaryExpression

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123408856 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/MathUtils.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123409716 --- Diff: sql/core/src/test/resources/sql-tests/inputs/operators.sql --- @@ -92,3 +92,8 @@ select abs(-3.13), abs('-2.19'); -- positive/negative

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123410449 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/MathUtils.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support ...

2017-06-21 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18323#discussion_r123412204 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/MathUtilsSuite.scala --- @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache

  1   2   3   4   5   6   >