[GitHub] spark issue #16927: [WIP][SPARK-19571][R] Fix SparkR test break on Windows v...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16927 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16927: [WIP][SPARK-19571][R] Fix SparkR test break on Windows v...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16927 **[Test build #72876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72876/testReport)** for PR 16927 at commit

[GitHub] spark issue #16927: [WIP][SPARK-19571][R] Fix SparkR test break on Windows v...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16927 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16927: [WIP][SPARK-19571][R] Fix SparkR test break on Wi...

2017-02-14 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/16927 [WIP][SPARK-19571][R] Fix SparkR test break on Windows via AppVeyor ## What changes were proposed in this pull request? It seems wintuils for Hadoop 2.6.5 not exiting for now in

[GitHub] spark issue #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and extend DataF...

2017-02-14 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16792 Thanks for putting in the time to get this really well documented :) Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-02-14 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15125 Our connected components computation completed successfully, with performance as expected. I've created a PR against @dding3's PR branch to incorporate a couple simple things. Then I think we're

[GitHub] spark issue #15468: [SPARK-17915][SQL] Prepare a new ColumnVector implementa...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15468 **[Test build #72886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72886/testReport)** for PR 15468 at commit

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-14 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/16909 @hvanhovell @davies Correct me if I am wrong. My understanding is that following code will go though all matching rows on the right side, and put them into the BufferedRowIterator. If there is OOM

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r10632 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -617,13 +617,18 @@ class Analyzer( private

[GitHub] spark issue #16896: [SPARK-19561][Python] cast TimestampType.toInternal outp...

2017-02-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16896 Hi, @davies . Could you review this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16929: [SPARK-19595][SQL] Do not allow json array in from_json

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72878/ Test PASSed. ---

[GitHub] spark issue #16929: [SPARK-19595][SQL] Do not allow json array in from_json

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16929 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16925: [SPARK-16475][SQL] Broadcast hint for SQL Queries

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16925 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72879/ Test PASSed. ---

[GitHub] spark issue #16913: [SPARK-15531] [DEPLOY] Complement launcher JVM memory se...

2017-02-14 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16913 You should not set a global init heap size like this and this is pretty much exactly why. There is no need for the launcher to immediately requests that much memory otherwise. Please close this

[GitHub] spark issue #16925: [SPARK-16475][SQL] Broadcast hint for SQL Queries

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16925 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16918: [SPARK-19584] [SS] [DOCS] update structured strea...

2017-02-14 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16918#discussion_r101122005 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -187,50 +305,68 @@ The following options must be set for the Kafka source. The following

[GitHub] spark pull request #16925: [SPARK-16475][SQL] Broadcast hint for SQL Queries

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16925#discussion_r101124672 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -374,6 +374,16 @@ querySpecification windows?)

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-02-14 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16535 looks good to me. I'll merge this to master since it helps with fast fail but in the future I'd appreciate it if you gave some thought as to how to do these in bigger chunks so we don't have a

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16537 **[Test build #72884 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72884/testReport)** for PR 16537 at commit

[GitHub] spark issue #15468: [SPARK-17915][SQL] Prepare a new ColumnVector implementa...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15468 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15468: [SPARK-17915][SQL] Prepare a new ColumnVector implementa...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15468 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16926: [MINOR][BUILD] Fix javadoc8 break

2017-02-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16926 Hi, @HyukjinKwon . Could you fix java linter errors together here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16926: [MINOR][BUILD] Fix javadoc8 break

2017-02-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16926 nit. In the PR description, a typo: `jeykill` -> `jekyll` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16918: [SPARK-19584] [SS] [DOCS] update structured streaming do...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16918 **[Test build #72887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72887/testReport)** for PR 16918 at commit

[GitHub] spark issue #16918: [SPARK-19584] [SS] [DOCS] update structured streaming do...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16918 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16926: [MINOR][BUILD] Fix javadoc8 break

2017-02-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16926 I removed my previous comment about `java linter` errors here. Never mind about that. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #16931: [SPARK-19587][SQL] bucket sorting columns should ...

2017-02-14 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/16931 [SPARK-19587][SQL] bucket sorting columns should not be picked from partition columns ## What changes were proposed in this pull request? We will throw an exception if bucket columns

[GitHub] spark pull request #16918: [SPARK-19584] [SS] [DOCS] update structured strea...

2017-02-14 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16918#discussion_r101121799 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -187,50 +305,68 @@ The following options must be set for the Kafka source. The following

[GitHub] spark pull request #16925: [SPARK-16475][SQL] Broadcast hint for SQL Queries

2017-02-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16925#discussion_r101122924 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -1002,8 +1012,12 @@ SIMPLE_COMMENT : '--'

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK][DOCS] import sort and autopep8 on...

2017-02-14 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14830 Great, thanks for updating this :) Would be good to see if @HyukjinKwon has anything to say otherwise I'll do another pass through this tomorrow and hopefully its really close :) --- If your

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-02-14 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16578 @viirya I've added a commit to address some of your feedback. I will have another commit to address the others, but I'm not sure when I'll have it in. Hopefully by the end of next week. --- If

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL] Add udf decorator

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16533 **[Test build #72885 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72885/testReport)** for PR 16533 at commit

[GitHub] spark issue #16927: [SPARK-19571][R] Fix SparkR test break on Windows via Ap...

2017-02-14 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16927 merged to master, thanks for the fix! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16720: [SPARK-19387][SPARKR] Tests do not run with Spark...

2017-02-14 Thread felixcheung
Github user felixcheung closed the pull request at: https://github.com/apache/spark/pull/16720 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #16720: [SPARK-19387][SPARKR] Tests do not run with Spark...

2017-02-14 Thread felixcheung
GitHub user felixcheung reopened a pull request: https://github.com/apache/spark/pull/16720 [SPARK-19387][SPARKR] Tests do not run with SparkR source package in CRAN check ## What changes were proposed in this pull request? - this is cause by changes in SPARK-18444,

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL] Add udf decorator

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16533 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16918: [SPARK-19584] [SS] [DOCS] update structured strea...

2017-02-14 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16918#discussion_r101120862 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -152,7 +270,7 @@ Each row in the source has the following schema: -The

[GitHub] spark pull request #16918: [SPARK-19584] [SS] [DOCS] update structured strea...

2017-02-14 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16918#discussion_r101120977 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -187,50 +305,68 @@ The following options must be set for the Kafka source. The following

[GitHub] spark pull request #16925: [SPARK-16475][SQL] Broadcast hint for SQL Queries

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16925#discussion_r101125744 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/SubstituteHints.scala --- @@ -0,0 +1,103 @@ +/* + * Licensed to the

[GitHub] spark issue #16928: [SPARK-18699][SQL] Fill NULL in a field when detecting a...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16928 **[Test build #72877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72877/testReport)** for PR 16928 at commit

[GitHub] spark issue #16928: [SPARK-18699][SQL] Fill NULL in a field when detecting a...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16928 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72877/ Test PASSed. ---

[GitHub] spark issue #16928: [SPARK-18699][SQL] Fill NULL in a field when detecting a...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16928 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16094: [SPARK-18541][Python]Add metadata parameter to pyspark.s...

2017-02-14 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16094 Merged to master :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16925: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16925 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16918: [SPARK-19584] [SS] [DOCS] update structured streaming do...

2017-02-14 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16918 @tcondie There is a typo about `fetchOffset.numRetries` in this page: `Number of times to retry before giving up **fatch** Kafka **latest** offsets.` -> `Number of times to retry before giving up

[GitHub] spark issue #16931: [SPARK-19587][SQL] bucket sorting columns should not be ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16931 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16931: [SPARK-19587][SQL] bucket sorting columns should not be ...

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16931 cc @tejasapatil @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16918: [SPARK-19584] [SS] [DOCS] update structured strea...

2017-02-14 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/16918#discussion_r101122357 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -187,50 +305,68 @@ The following options must be set for the Kafka source. The following

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-02-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r101061331 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -685,9 +686,8 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-02-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r101083260 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -107,6 +121,30 @@ public static void setConfig(String name, String

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-02-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r101084346 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -528,13 +582,41 @@ public SparkAppHandle

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-02-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r101092206 --- Diff: core/src/main/scala/org/apache/spark/launcher/LauncherBackend.scala --- @@ -71,6 +100,9 @@ private[spark] abstract class LauncherBackend {

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-02-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r101085524 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitRunner.java --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-02-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r101062900 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -719,7 +719,23 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark pull request #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction sh...

2017-02-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16535 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16094: [SPARK-18541][Python]Add metadata parameter to pyspark.s...

2017-02-14 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16094 Thank you for updating this, looks good to me! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL] Add udf decorator

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16533 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL] Add udf decorator

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16533 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL] Add udf decorator

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16533 **[Test build #72885 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72885/testReport)** for PR 16533 at commit

[GitHub] spark issue #16918: [SPARK-19584] [SS] [DOCS] update structured streaming do...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16918 **[Test build #72887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72887/testReport)** for PR 16918 at commit

[GitHub] spark issue #16918: [SPARK-19584] [SS] [DOCS] update structured streaming do...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16918 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16918: [SPARK-19584] [SS] [DOCS] update structured streaming do...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16918 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16925: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2017-02-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16925 Except the doc you mentioned, it looks great to me, @rxin ! It's great to have this in Spark SQL finally. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16537 **[Test build #72884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72884/testReport)** for PR 16537 at commit

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72884/ Test PASSed. ---

[GitHub] spark pull request #16927: [SPARK-19571][R] Fix SparkR test break on Windows...

2017-02-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16927 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16931: [SPARK-19587][SQL] bucket sorting columns should not be ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16931 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16931: [SPARK-19587][SQL] bucket sorting columns should not be ...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16931 **[Test build #72889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72889/testReport)** for PR 16931 at commit

[GitHub] spark pull request #16925: [SPARK-16475][SQL] Broadcast hint for SQL Queries

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16925#discussion_r101122146 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -1002,8 +1012,12 @@ SIMPLE_COMMENT : '--' ~[\r\n]*

[GitHub] spark pull request #16920: [MINOR][DOCS] Add jira url in pull request descri...

2017-02-14 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16920#discussion_r101126026 --- Diff: .github/PULL_REQUEST_TEMPLATE --- @@ -8,3 +8,10 @@ (If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

[GitHub] spark pull request #16925: [SPARK-16475][SQL] Broadcast hint for SQL Queries

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16925#discussion_r101124529 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/SubstituteHints.scala --- @@ -0,0 +1,103 @@ +/* + * Licensed to the

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-02-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r101085286 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -528,13 +582,41 @@ public SparkAppHandle

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-02-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r101095913 --- Diff: core/src/main/scala/org/apache/spark/launcher/LauncherBackend.scala --- @@ -71,6 +100,9 @@ private[spark] abstract class LauncherBackend {

[GitHub] spark pull request #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and exten...

2017-02-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16792 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and extend DataF...

2017-02-14 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/16792 Thanks @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r101097919 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCWriteSuite.scala --- @@ -349,4 +349,17 @@ class JDBCWriteSuite extends SharedSQLContext

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r101097830 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCWriteSuite.scala --- @@ -349,4 +349,17 @@ class JDBCWriteSuite extends SharedSQLContext

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-14 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz Any thoughts on moving the dependency version bump to a new commit and backporting to 2.11 with the pervious versions? @zswing Any chance you could take a look at this sometime this

[GitHub] spark pull request #16922: [SPARK-19590][pyspark][ML] Update the document fo...

2017-02-14 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/16922#discussion_r101097983 --- Diff: python/pyspark/ml/feature.py --- @@ -1178,7 +1178,17 @@ class QuantileDiscretizer(JavaEstimator, HasInputCol, HasOutputCol, JavaMLReadab

[GitHub] spark issue #16891: [SPARK-19318][SQL] Fix to treat JDBC connection properti...

2017-02-14 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16891 I ran the docker tests in my local computer. Now, finally, all the tests can pass! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-14 Thread dgingrich
Github user dgingrich commented on the issue: https://github.com/apache/spark/pull/16845 Added a helper that falls back from `e.message` to `str(e)`. Also rebased to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16715: [Spark-18080][ML][PYTHON] Python API & Examples for Loca...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16715 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72881/ Test PASSed. ---

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r101110989 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -617,13 +617,18 @@ class Analyzer( private

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r10083 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala --- @@ -15,29 +15,34 @@ * limitations under the License.

[GitHub] spark issue #16739: [SPARK-19399][SPARKR] Add R coalesce API for DataFrame a...

2017-02-14 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16739 great, looking forward to that. I'm going to merge this unless anyone has a concern? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #16818: [SPARK-19451][SQL][Core] Underlying integer overf...

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16818#discussion_r101121242 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/expressions/WindowSpec.scala --- @@ -180,16 +180,20 @@ class WindowSpec private[sql](

[GitHub] spark issue #16917: [SPARK-19529][BRANCH-1.6] Backport PR #16866 to branch-1...

2017-02-14 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/16917 LGTM, so I'm going to merge this to branch-1.6. Thanks for putting together the backport! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #16818: [SPARK-19451][SQL][Core] Underlying integer overf...

2017-02-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16818#discussion_r101121167 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/expressions/WindowSpec.scala --- @@ -180,16 +180,20 @@ class WindowSpec private[sql](

[GitHub] spark issue #16932: [SPARK-19599][SS]Clean up HDFSMetadataLog for Hadoop 2.6...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16932 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16932: [SPARK-19599][SS]Clean up HDFSMetadataLog for Hadoop 2.6...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16932 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-14 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16916 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #16800: [SPARK-19456][SparkR]:Add LinearSVC R API

2017-02-14 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16800#discussion_r101120745 --- Diff: R/pkg/R/generics.R --- @@ -1380,6 +1380,10 @@ setGeneric("spark.kstest", function(data, ...) { standardGeneric("spark.kstest") #'

[GitHub] spark pull request #16800: [SPARK-19456][SparkR]:Add LinearSVC R API

2017-02-14 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16800#discussion_r101121442 --- Diff: R/pkg/R/mllib_classification.R --- @@ -39,6 +46,131 @@ setClass("MultilayerPerceptronClassificationModel", representation(jobj = "jobj"

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-02-14 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r100976816 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Instance.scala --- @@ -27,3 +27,25 @@ import org.apache.spark.ml.linalg.Vector *

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-02-14 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16699 @sethah Thanks much for your review. I've made a new commit that addressed all your comments. Please see my inline comments. Let me know if there is any other suggestions. Thanks. --- If

[GitHub] spark issue #16921: [SPARK-19589][SQL] Removal of SQLGEN files

2017-02-14 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16921 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-14 Thread sureshthalamati
Github user sureshthalamati commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r100976164 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -31,7 +31,12 @@ class JDBCOptions(

[GitHub] spark pull request #16800: [SPARK-19456][SparkR]:Add LinearSVC R API

2017-02-14 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/16800#discussion_r100976153 --- Diff: R/pkg/R/mllib_classification.R --- @@ -39,6 +46,116 @@ setClass("MultilayerPerceptronClassificationModel", representation(jobj = "jobj"

  1   2   3   4   5   6   7   >