[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18576 **[Test build #79445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79445/testReport)** for PR 18576 at commit [`841179f`](https://github.com/apache/spark/commit/84

[GitHub] spark pull request #18541: [SPARK-21315][SQL]Skip some spill files when gene...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18541#discussion_r126345627 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -589,29 +589,49 @@ public long getKeyPrefix()

[GitHub] spark pull request #16422: [SPARK-17642] [SQL] support DESC EXTENDED/FORMATT...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16422#discussion_r126345116 --- Diff: sql/core/src/test/resources/sql-tests/results/describe-table-column.sql.out --- @@ -0,0 +1,133 @@ +-- Automatically generated by SQLQueryTes

[GitHub] spark pull request #18576: [SPARK-21351][SQL] Update nullability based on ch...

2017-07-09 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18576#discussion_r126345144 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -132,7 +132,9 @@ abstract class Optimizer(sessionCatalo

[GitHub] spark pull request #18578: [MINOR][DOC] Remove obsolete `ec2-scripts.md`

2017-07-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18578 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18543: [SPARK-21319][SQL] Fix memory leak in UnsafeExternalRowS...

2017-07-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18543 @davies or @rxin might be worth a look. The only possible downside here is the extra cycles to clear the reference, so I raise it in case you think we need to find a more optimized way to deal with t

[GitHub] spark pull request #16422: [SPARK-17642] [SQL] support DESC EXTENDED/FORMATT...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16422#discussion_r126344763 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -626,6 +624,120 @@ case class DescribeTableCommand( }

[GitHub] spark issue #18578: [MINOR][DOC] Remove obsolete `ec2-scripts.md`

2017-07-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18578 Merged to master. Yeah this file is obsolete, a redirector, and we've just removed the link in the main site that would use it anyway. Links to it from external sites, if any, have actually been brok

[GitHub] spark pull request #16422: [SPARK-17642] [SQL] support DESC EXTENDED/FORMATT...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16422#discussion_r126344451 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -626,6 +624,120 @@ case class DescribeTableCommand( }

[GitHub] spark issue #18551: [SPARK-21305][ML][MLLIB]Add options to disable multi-thr...

2017-07-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18551 I doubt the other background tasks will consume a whole additional CPU for every task. You're saying that on a 16-core machine with 8 tasks, the other 8 CPUs are probably nearly fully utilized? I jus

[GitHub] spark pull request #16422: [SPARK-17642] [SQL] support DESC EXTENDED/FORMATT...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16422#discussion_r126344265 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -626,6 +624,120 @@ case class DescribeTableCommand( }

[GitHub] spark pull request #18427: [SPARK-21219][Core] Task retry occurs on same exe...

2017-07-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18427 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18576: [SPARK-21351][SQL] Update nullability based on ch...

2017-07-09 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18576#discussion_r126344026 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/UpdateNullabilityInAttributeReferencesSuite.scala --- @@ -0,0 +1,53 @@ +/*

[GitHub] spark issue #18551: [SPARK-21305][ML][MLLIB]Add options to disable multi-thr...

2017-07-09 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18551 hi @srowen , I understand Felix's point. I mean if you only have 1 task in C/C++, and 2 CPUs, setting native BLAS to use 2 CPUs will be faster. But in JVM env, even you only have one task, and 2 CPUs,

[GitHub] spark issue #18427: [SPARK-21219][Core] Task retry occurs on same executor d...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18427 LGTM, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark pull request #18498: [SPARK-21266][R][PYTHON] Support schema a DDL-for...

2017-07-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18498#discussion_r126342705 --- Diff: R/pkg/R/schema.R --- @@ -23,7 +23,8 @@ #' Create a structType object that contains the metadata for a SparkDataFrame. Intended for #

[GitHub] spark pull request #18555: [SPARK-21353][CORE]add checkValue in spark.intern...

2017-07-09 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18555#discussion_r126343554 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -51,29 +62,61 @@ package object config { ConfigBuilder(Spark

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #79443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79443/testReport)** for PR 18555 at commit [`3df8feb`](https://github.com/apache/spark/commit/3d

[GitHub] spark pull request #18498: [SPARK-21266][R][PYTHON] Support schema a DDL-for...

2017-07-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18498#discussion_r126342898 --- Diff: R/pkg/R/schema.R --- @@ -68,18 +73,10 @@ structType.structField <- function(x, ...) { structType(stObj) } -#' Create \cod

[GitHub] spark issue #18541: [SPARK-21315][SQL]Skip some spill files when generateIte...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18541 **[Test build #79444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79444/testReport)** for PR 18541 at commit [`3ab157d`](https://github.com/apache/spark/commit/3a

[GitHub] spark pull request #18574: [SPARK-21350] [SQL] Fix the error message when th...

2017-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18574#discussion_r126343507 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -112,7 +112,12 @@ class SimpleFunctionRegistry ext

[GitHub] spark pull request #18498: [SPARK-21266][R][PYTHON] Support schema a DDL-for...

2017-07-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18498#discussion_r126343428 --- Diff: R/pkg/R/schema.R --- @@ -76,7 +76,7 @@ structType.structField <- function(x, ...) { #' @rdname structType #' @method structType chara

[GitHub] spark pull request #18576: [SPARK-21351][SQL] Update nullability based on ch...

2017-07-09 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18576#discussion_r126343168 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/UpdateNullabilityInAttributeReferencesSuite.scala --- @@ -0,0 +1,53 @@ +/*

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #79442 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79442/testReport)** for PR 18581 at commit [`eb77c98`](https://github.com/apache/spark/commit/eb

[GitHub] spark pull request #18541: [SPARK-21315][SQL]Skip some spill files when gene...

2017-07-09 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/18541#discussion_r126342950 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -589,29 +589,51 @@ public long getKeyPrefix()

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #79441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79441/testReport)** for PR 18581 at commit [`a5410e5`](https://github.com/apache/spark/commit/a5

[GitHub] spark issue #18551: [SPARK-21305][ML][MLLIB]Add options to disable multi-thr...

2017-07-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18551 @mpjlu Felix is making a slightly different point. Let's say Spark is actually configured to use 2 CPUs per task. The right setting is not 1 thread per task but 2. Using 1 would almost surely not be

[GitHub] spark pull request #18576: [SPARK-21351][SQL] Update nullability based on ch...

2017-07-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18576#discussion_r126342208 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/UpdateNullabilityInAttributeReferencesSuite.scala --- @@ -0,0 +1,53 @@ +/*

[GitHub] spark pull request #18228: [SPARK-21007][SQL]Add SQL function - RIGHT && LEF...

2017-07-09 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/18228#discussion_r126341299 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1199,6 +1199,82 @@ case class Substring(st

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #79440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79440/testReport)** for PR 18581 at commit [`585a399`](https://github.com/apache/spark/commit/58

[GitHub] spark pull request #18581: [SPARK-21289][SQL][ML] Supports custom line separ...

2017-07-09 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/18581 [SPARK-21289][SQL][ML] Supports custom line separator for all text-based datasources ## What changes were proposed in this pull request? This PR proposes to add `lineSep` option for a

[GitHub] spark pull request #18541: [SPARK-21315][SQL]Skip some spill files when gene...

2017-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18541#discussion_r126340610 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -589,29 +589,51 @@ public long getKeyPrefix() {

[GitHub] spark pull request #18228: [SPARK-21007][SQL]Add SQL function - RIGHT && LEF...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18228#discussion_r126340190 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1199,6 +1199,82 @@ case class Substring(s

[GitHub] spark pull request #18576: [SPARK-21351][SQL] Update nullability based on ch...

2017-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18576#discussion_r126339942 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -132,7 +132,9 @@ abstract class Optimizer(sessionCatalog

[GitHub] spark pull request #18570: [SPARK-21100][SQL][Followup] cleanup code and add...

2017-07-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18570 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18570: [SPARK-21100][SQL][Followup] cleanup code and add more c...

2017-07-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18570 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark pull request #18576: [SPARK-21351][SQL] Update nullability based on ch...

2017-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18576#discussion_r126339215 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/UpdateNullabilityInAttributeReferencesSuite.scala --- @@ -0,0 +1,53 @@ +/*

[GitHub] spark issue #18570: [SPARK-21100][SQL][Followup] cleanup code and add more c...

2017-07-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18570 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark issue #18570: [SPARK-21100][SQL][Followup] cleanup code and add more c...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18570 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79433/ Test PASSed. ---

[GitHub] spark issue #18570: [SPARK-21100][SQL][Followup] cleanup code and add more c...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18570 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18570: [SPARK-21100][SQL][Followup] cleanup code and add more c...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18570 **[Test build #79433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79433/testReport)** for PR 18570 at commit [`d82475f`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #18063: [SPARK-20842] [SQL] Upgrade to 1.2.2 for Hive Metastore ...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18063 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79434/ Test PASSed. ---

[GitHub] spark issue #18063: [SPARK-20842] [SQL] Upgrade to 1.2.2 for Hive Metastore ...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18063 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18063: [SPARK-20842] [SQL] Upgrade to 1.2.2 for Hive Metastore ...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18063 **[Test build #79434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79434/testReport)** for PR 18063 at commit [`5ff0039`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79431/ Test PASSed. ---

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #79431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79431/testReport)** for PR 18555 at commit [`8257e8a`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79429/ Test PASSed. ---

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #79429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79429/testReport)** for PR 18555 at commit [`d550a68`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #18571: [SPARK-21344][SQL] BinaryType comparison does signed byt...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18571 **[Test build #79439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79439/testReport)** for PR 18571 at commit [`a84776b`](https://github.com/apache/spark/commit/a8

[GitHub] spark pull request #18571: [SPARK-21344][SQL] BinaryType comparison does sig...

2017-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18571#discussion_r126335991 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/ByteArray.java --- @@ -44,7 +44,7 @@ public static long getPrefix(byte[] bytes) {

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18576 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79430/ Test PASSed. ---

[GitHub] spark issue #18228: [SPARK-21007][SQL]Add SQL function - RIGHT && LEFT

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18228 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18576 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18228: [SPARK-21007][SQL]Add SQL function - RIGHT && LEFT

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18228 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79432/ Test PASSed. ---

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18576 **[Test build #79430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79430/testReport)** for PR 18576 at commit [`d79f976`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #18228: [SPARK-21007][SQL]Add SQL function - RIGHT && LEFT

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18228 **[Test build #79432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79432/testReport)** for PR 18228 at commit [`2eb4d0f`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18565: [SPARK-21342] Fix DownloadCallback to work well with Ret...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18565 **[Test build #79438 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79438/testReport)** for PR 18565 at commit [`15001a6`](https://github.com/apache/spark/commit/15

[GitHub] spark issue #18459: [SPARK-13534][PYSPARK] Using Apache Arrow to increase pe...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79427/ Test PASSed. ---

[GitHub] spark issue #18459: [SPARK-13534][PYSPARK] Using Apache Arrow to increase pe...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18459 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18459: [SPARK-13534][PYSPARK] Using Apache Arrow to increase pe...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18459 **[Test build #79427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79427/testReport)** for PR 18459 at commit [`26dfc82`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #18571: [SPARK-21344][SQL] BinaryType comparison does sig...

2017-07-09 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18571#discussion_r126335167 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/OrderingSuite.scala --- @@ -137,4 +137,22 @@ class OrderingSuite extends Spa

[GitHub] spark issue #18541: [SPARK-21315][SQL]Skip some spill files when generateIte...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18541 **[Test build #79437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79437/testReport)** for PR 18541 at commit [`473483d`](https://github.com/apache/spark/commit/47

[GitHub] spark issue #18571: [SPARK-21344][SQL] BinaryType comparison does signed byt...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18571 **[Test build #79436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79436/testReport)** for PR 18571 at commit [`814d4c4`](https://github.com/apache/spark/commit/81

[GitHub] spark pull request #18571: [SPARK-21344][SQL] BinaryType comparison does sig...

2017-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18571#discussion_r126335091 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/OrderingSuite.scala --- @@ -137,4 +137,22 @@ class OrderingSuite extends Spar

[GitHub] spark pull request #18571: [SPARK-21344][SQL] BinaryType comparison does sig...

2017-07-09 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18571#discussion_r126335033 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/ByteArray.java --- @@ -44,7 +44,7 @@ public static long getPrefix(byte[] bytes) {

[GitHub] spark issue #18565: [SPARK-21342] Fix DownloadCallback to work well with Ret...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18565 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18565: [SPARK-21342] Fix DownloadCallback to work well with Ret...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18565 **[Test build #79435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79435/testReport)** for PR 18565 at commit [`bb82dd3`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18565: [SPARK-21342] Fix DownloadCallback to work well with Ret...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18565 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79435/ Test FAILed. ---

[GitHub] spark issue #18565: [SPARK-21342] Fix DownloadCallback to work well with Ret...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18565 **[Test build #79435 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79435/testReport)** for PR 18565 at commit [`bb82dd3`](https://github.com/apache/spark/commit/bb

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79428/ Test FAILed. ---

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #79428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79428/testReport)** for PR 18555 at commit [`9f924e5`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #18063: [SPARK-20842] [SQL] Upgrade to 1.2.2 for Hive Metastore ...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18063 **[Test build #79434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79434/testReport)** for PR 18063 at commit [`5ff0039`](https://github.com/apache/spark/commit/5f

[GitHub] spark issue #18063: [SPARK-20842] [SQL] Upgrade to 1.2.2 for Hive Metastore ...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18063 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18541: [SPARK-21315][SQL]Skip some spill files when generateIte...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18541 LGTM except some minor comments, good catch! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-07-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18460 Hi, @cloud-fan . Could you review this again when you have sometime? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #18541: [SPARK-21315][SQL]Skip some spill files when gene...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18541#discussion_r126331052 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -589,29 +589,51 @@ public long getKeyPrefix()

[GitHub] spark pull request #18541: [SPARK-21315][SQL]Skip some spill files when gene...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18541#discussion_r126330934 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -589,29 +589,51 @@ public long getKeyPrefix()

[GitHub] spark pull request #18574: [SPARK-21350] [SQL] Fix the error message when th...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18574#discussion_r126330636 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -112,7 +112,12 @@ class SimpleFunctionRegistry

[GitHub] spark issue #18577: [SPARK-21083][SQL][branch-2.1] Store zero size and row c...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18577 thanks, merging to 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18570: [SPARK-21100][SQL][Followup] cleanup code and add more c...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18570 **[Test build #79433 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79433/testReport)** for PR 18570 at commit [`d82475f`](https://github.com/apache/spark/commit/d8

[GitHub] spark pull request #18552: [Minor][Doc] Improve the docs about how to correc...

2017-07-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18552 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18555: [SPARK-21353][CORE]add checkValue in spark.intern...

2017-07-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18555#discussion_r126329361 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -51,29 +63,66 @@ package object config { ConfigBuilder(S

[GitHub] spark pull request #18555: [SPARK-21353][CORE]add checkValue in spark.intern...

2017-07-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18555#discussion_r126329304 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -51,29 +62,61 @@ package object config { ConfigBuilder(S

[GitHub] spark pull request #18555: [SPARK-21353][CORE]add checkValue in spark.intern...

2017-07-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18555#discussion_r126329325 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -51,29 +62,61 @@ package object config { ConfigBuilder(S

[GitHub] spark issue #18552: [Minor][Doc] Improve the docs about how to correctly set...

2017-07-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18552 LGTM, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #18576: [SPARK-21351][SQL] Update nullability based on children'...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18576 **[Test build #79430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79430/testReport)** for PR 18576 at commit [`d79f976`](https://github.com/apache/spark/commit/d7

[GitHub] spark issue #18228: [SPARK-21007][SQL]Add SQL function - RIGHT && LEFT

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18228 **[Test build #79432 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79432/testReport)** for PR 18228 at commit [`2eb4d0f`](https://github.com/apache/spark/commit/2e

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #79431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79431/testReport)** for PR 18555 at commit [`8257e8a`](https://github.com/apache/spark/commit/82

[GitHub] spark pull request #18555: [SPARK-21353][CORE]add checkValue in spark.intern...

2017-07-09 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18555#discussion_r126326738 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -51,29 +62,61 @@ package object config { ConfigBuilder(Spark

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #79429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79429/testReport)** for PR 18555 at commit [`d550a68`](https://github.com/apache/spark/commit/d5

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #79428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79428/testReport)** for PR 18555 at commit [`9f924e5`](https://github.com/apache/spark/commit/9f

[GitHub] spark pull request #18228: [SPARK-21007][SQL]Add SQL function - RIGHT && LEF...

2017-07-09 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/18228#discussion_r126325282 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1199,6 +1199,85 @@ case class Substring(st

[GitHub] spark issue #18459: [SPARK-13534][PYSPARK] Using Apache Arrow to increase pe...

2017-07-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18459 **[Test build #79427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79427/testReport)** for PR 18459 at commit [`26dfc82`](https://github.com/apache/spark/commit/26

[GitHub] spark issue #18459: [SPARK-13534][PYSPARK] Using Apache Arrow to increase pe...

2017-07-09 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18459 jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18551: [SPARK-21305][ML][MLLIB]Add options to disable multi-thr...

2017-07-09 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18551 hi @felixcheung , I have tested one case, write a single thread java program, and call native blas. The performance is much better to disable native blas multi-threading (the total program performanc

[GitHub] spark pull request #18580: [SPARK-21354] [SQL] INPUT FILE related functions ...

2017-07-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18580#discussion_r126323436 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -100,6 +104,10 @@ trait CheckAnalysis extends

[GitHub] spark pull request #18580: [SPARK-21354] [SQL] INPUT FILE related functions ...

2017-07-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18580#discussion_r126323442 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala --- @@ -530,6 +530,20 @@ class ColumnExpressionSuite extends QueryT

  1   2   3   >