[GitHub] spark issue #15937: [SPARK-18508][SQL] Fix documentation error for DateDiff

2016-11-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15937 LGTM Sorry, I did not catch this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15929: [SPARK-18053][SQL] compare unsafe and safe comple...

2016-11-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15929#discussion_r88774814 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -452,15 +444,7 @@ case class EqualNullSafe(left:

[GitHub] spark pull request #15929: [SPARK-18053][SQL] compare unsafe and safe comple...

2016-11-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15929#discussion_r88774467 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -481,8 +481,13 @@ class

[GitHub] spark issue #15937: [SPARK-18508][SQL] Fix documentation error for DateDiff

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15937 **[Test build #68882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68882/consoleFull)** for PR 15937 at commit

[GitHub] spark pull request #15937: [SQL] Fix documentation error for DateDiff

2016-11-18 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/15937 [SQL] Fix documentation error for DateDiff ## What changes were proposed in this pull request? The previous documentation and example for DateDiff was wrong. ## How was this patch tested?

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15901 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15901 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68881/ Test PASSed. ---

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15901 **[Test build #68881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68881/consoleFull)** for PR 15901 at commit

[GitHub] spark issue #15870: [SPARK-18425][Structured Streaming][Tests] Test `Compact...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15870 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15870: [SPARK-18425][Structured Streaming][Tests] Test `Compact...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15870 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68879/ Test PASSed. ---

[GitHub] spark issue #15870: [SPARK-18425][Structured Streaming][Tests] Test `Compact...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15870 **[Test build #68879 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68879/consoleFull)** for PR 15870 at commit

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15901 **[Test build #68881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68881/consoleFull)** for PR 15901 at commit

[GitHub] spark issue #15620: [SPARK-18091] [SQL] Deep if expressions cause Generated ...

2016-11-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15620 Looks like it is not flaky test. The test failure is due to compilation error of janino: A local variable can not be accessed in the generated function. --- If your project is set up for it, you

[GitHub] spark pull request #15637: [SPARK-18000] [SQL] Aggregation function for comp...

2016-11-18 Thread wzhfy
Github user wzhfy closed the pull request at: https://github.com/apache/spark/pull/15637 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15637: [SPARK-18000] [SQL] Aggregation function for computing b...

2016-11-18 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15637 Since the JIRA is closed, I'm closing this PR too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15901: [SPARK-18467][SQL] Extracts method for preparing ...

2016-11-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15901#discussion_r88770786 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -82,8 +148,8 @@ case class StaticInvoke(

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15874 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68880/ Test PASSed. ---

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15874 **[Test build #68880 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68880/consoleFull)** for PR 15874 at commit

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15874 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15877: [SPARK-18429] [SQL] implement a new Aggregate for...

2016-11-18 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/15877#discussion_r88770530 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CountMinSketchAgg.scala --- @@ -0,0 +1,150 @@ +/* + *

[GitHub] spark pull request #15929: [SPARK-18053][SQL] compare unsafe and safe comple...

2016-11-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15929#discussion_r88770432 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -481,8 +481,13 @@ class

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15901 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15901 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68877/ Test PASSed. ---

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15901 **[Test build #68877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68877/consoleFull)** for PR 15901 at commit

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68876/ Test PASSed. ---

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13065 **[Test build #68876 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68876/consoleFull)** for PR 13065 at commit

[GitHub] spark pull request #15596: [SPARK-18089][SQL] Remove shuffle codes in Collec...

2016-11-18 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/15596 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15596: [SPARK-18089][SQL] Remove shuffle codes in CollectLimitE...

2016-11-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15596 I'd close this now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15837: [SPARK-18395][SQL] Evaluate common subexpression like la...

2016-11-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15837 ping @cloud-fan @hvanhovell @kiszk Can you help review this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15880: [SPARK-17913][SQL] compare long and string type column m...

2016-11-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15880 The [official Hive document](https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-AllowedImplicitConversions) explains the implicit type casting rules.

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15874 **[Test build #68880 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68880/consoleFull)** for PR 15874 at commit

[GitHub] spark issue #15870: [SPARK-18425][Structured Streaming][Tests] Test `Compact...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15870 **[Test build #68879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68879/consoleFull)** for PR 15870 at commit

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-18 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/15874 Hi @sethah, grouping to a number of buckets does not really affect the independence since p is a mach larger prime. For example, in http://people.csail.mit.edu/mip/papers/kwise-lb/kwise-lb.pdf, they

[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15935 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15935 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68874/ Test PASSed. ---

[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15935 **[Test build #68874 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68874/consoleFull)** for PR 15935 at commit

[GitHub] spark pull request #15893: [SPARK-18456][ML][FOLLOWUP] Use matrix abstractio...

2016-11-18 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/15893#discussion_r88767607 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -586,15 +577,24 @@ class LogisticRegression

[GitHub] spark pull request #15923: [SPARK-4105] retry the fetch or stage if shuffle ...

2016-11-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15923#discussion_r88767299 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -305,40 +312,82 @@ final class

[GitHub] spark pull request #15923: [SPARK-4105] retry the fetch or stage if shuffle ...

2016-11-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15923#discussion_r88767669 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -305,40 +312,82 @@ final class

[GitHub] spark pull request #15923: [SPARK-4105] retry the fetch or stage if shuffle ...

2016-11-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15923#discussion_r88767018 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -56,8 +59,10 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request #15923: [SPARK-4105] retry the fetch or stage if shuffle ...

2016-11-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15923#discussion_r88767390 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -305,40 +312,82 @@ final class

[GitHub] spark pull request #15923: [SPARK-4105] retry the fetch or stage if shuffle ...

2016-11-18 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15923#discussion_r88767352 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -305,40 +312,82 @@ final class

[GitHub] spark issue #15893: [SPARK-18456][ML][FOLLOWUP] Use matrix abstraction for c...

2016-11-18 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/15893 Only minor naming. LGTM. My interest can not access ssh to merge the code, will merge later tonight. Thanks. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15891: [SPARK-18300][SQL] Fix scala 2.10 build for FoldableProp...

2016-11-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15891 Maybe, could you close this? @hvanhovell ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 @hvanhovell @rxin What do you think? Please let me know if we need this or not. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68875/ Test PASSed. ---

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13065 **[Test build #68875 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68875/consoleFull)** for PR 13065 at commit

[GitHub] spark issue #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false work w...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15820 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14854: [SPARK-17283][Core] Cancel job in RDD.take() as soon as ...

2016-11-18 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/14854 Another advantage of this PR's approach is straggler-resilience: imagine that you're going to have to compute all partitions anyways and also assume that some small number of partitions will

[GitHub] spark issue #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false work w...

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15820 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68878/ Test PASSed. ---

[GitHub] spark issue #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false work w...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15820 **[Test build #68878 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68878/consoleFull)** for PR 15820 at commit

[GitHub] spark pull request #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false...

2016-11-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15820#discussion_r88765764 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/CachedKafkaConsumer.scala --- @@ -47,40 +51,190 @@ private[kafka010] case class

[GitHub] spark pull request #15933: [SPARK-18505][SQL] Simplify AnalyzeColumnCommand

2016-11-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15933 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15934: [SPARK-18497][SS]Make ForeachSink support waterma...

2016-11-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15934 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15870: [SPARK-18425][Structured Streaming][Tests] Test `Compact...

2016-11-18 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/15870 @zsxwing thanks. I'm rebasing right now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15933: [SPARK-18505][SQL] Simplify AnalyzeColumnCommand

2016-11-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15933 Merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15828: [SPARK-18187][SS] CompactibleFileStreamLog should...

2016-11-18 Thread lw-lin
Github user lw-lin closed the pull request at: https://github.com/apache/spark/pull/15828 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15934: [SPARK-18497][SS]Make ForeachSink support watermark

2016-11-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15934 Seems like this is a correctness bug as aggregation will give wrong results when using aggregations. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15934: [SPARK-18497][SS]Make ForeachSink support watermark

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15934 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14854: [SPARK-17283][Core] Cancel job in RDD.take() as soon as ...

2016-11-18 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/14854 I thought of one significant UX improvement offered by the approach in this withdrawn PR: when the current "multiple jobs" limit executes, there's no way for the end user to know how much work

[GitHub] spark issue #15933: [SPARK-18505][SQL] Simplify AnalyzeColumnCommand

2016-11-18 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/15933 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15934: [SPARK-18497][SS]Make ForeachSink support watermark

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15934 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68873/ Test PASSed. ---

[GitHub] spark issue #15932: [SPARK-18448][CORE] SparkSession should implement java.l...

2016-11-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15932 Great! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15934: [SPARK-18497][SS]Make ForeachSink support watermark

2016-11-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15934 Merging to master, 2.1 and 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15933: [SPARK-18505][SQL] Simplify AnalyzeColumnCommand

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15933 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68872/ Test PASSed. ---

[GitHub] spark issue #15934: [SPARK-18497][SS]Make ForeachSink support watermark

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15934 **[Test build #68873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68873/consoleFull)** for PR 15934 at commit

[GitHub] spark issue #15933: [SPARK-18505][SQL] Simplify AnalyzeColumnCommand

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15933 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14812: [SPARK-17237][SQL] Remove backticks in a pivot result sc...

2016-11-18 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/14812 @gatorsmile Do u have time to check this? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15933: [SPARK-18505][SQL] Simplify AnalyzeColumnCommand

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15933 **[Test build #68872 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68872/consoleFull)** for PR 15933 at commit

[GitHub] spark pull request #15910: [SPARK-18476][SPARKR][ML]:SparkR Logistic Regress...

2016-11-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15910#discussion_r88764190 --- Diff: R/pkg/inst/tests/testthat/test_mllib.R --- @@ -674,6 +674,16 @@ test_that("spark.logit", { expect_error(summary(blr_model2))

[GitHub] spark pull request #15929: [SPARK-18053][SQL] compare unsafe and safe comple...

2016-11-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15929#discussion_r88764135 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -481,8 +481,13 @@ class

[GitHub] spark pull request #15910: [SPARK-18476][SPARKR][ML]:SparkR Logistic Regress...

2016-11-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15910#discussion_r88764104 --- Diff: R/pkg/inst/tests/testthat/test_mllib.R --- @@ -637,31 +637,31 @@ test_that("spark.isotonicRegression", { test_that("spark.logit", {

[GitHub] spark issue #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false work w...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15820 **[Test build #68878 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68878/consoleFull)** for PR 15820 at commit

[GitHub] spark pull request #15888: [SPARK-18444][SPARKR] SparkR running in yarn-clus...

2016-11-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15888#discussion_r88763342 --- Diff: R/pkg/R/sparkR.R --- @@ -373,8 +373,13 @@ sparkR.session <- function( overrideEnvs(sparkConfigMap, paramMap) } +

[GitHub] spark issue #15868: [SPARK-18413][SQL] Add `maxConnections` JDBCOption

2016-11-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15868 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15911: [SPARK-18477][SS]Enable interrupts for HDFS in HD...

2016-11-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15911 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15888 looks good. We don't have this but how about adding tests for `sparkCheckInstall`? we could at least check for the error cases it has right now? --- If your project is set up for it, you can

[GitHub] spark pull request #15929: [SPARK-18053][SQL] compare unsafe and safe comple...

2016-11-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15929#discussion_r88763155 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -481,8 +481,13 @@ class

[GitHub] spark issue #15868: [SPARK-18413][SQL] Add `maxConnections` JDBCOption

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15868 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68871/ Test PASSed. ---

[GitHub] spark issue #15868: [SPARK-18413][SQL] Add `maxConnections` JDBCOption

2016-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15868 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15868: [SPARK-18413][SQL] Add `maxConnections` JDBCOption

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15868 **[Test build #68871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68871/consoleFull)** for PR 15868 at commit

[GitHub] spark issue #15901: [SPARK-18467][SQL] Extracts method for preparing argumen...

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15901 **[Test build #68877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68877/consoleFull)** for PR 15901 at commit

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-18 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15874 @jkbradley Thanks for checking that, that is the conclusion I drew as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15911: [SPARK-18477][SS]Enable interrupts for HDFS in HDFSMetad...

2016-11-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15911 LGTM. Merging to master, 2.1, and 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-18 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15874#discussion_r88753014 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHashLSH.scala --- @@ -31,36 +31,40 @@ import org.apache.spark.sql.types.StructType /**

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-11-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13065 **[Test build #68876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68876/consoleFull)** for PR 13065 at commit

[GitHub] spark pull request #15923: [SPARK-4105] retry the fetch or stage if shuffle ...

2016-11-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15923#discussion_r88759884 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -305,40 +312,82 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request #15923: [SPARK-4105] retry the fetch or stage if shuffle ...

2016-11-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15923#discussion_r88759763 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -305,40 +312,82 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false...

2016-11-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15820#discussion_r88759061 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala --- @@ -223,52 +228,184 @@ class KafkaSourceSuite

[GitHub] spark pull request #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false...

2016-11-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15820#discussion_r88759005 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala --- @@ -223,52 +228,184 @@ class KafkaSourceSuite

[GitHub] spark pull request #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false...

2016-11-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15820#discussion_r88758879 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala --- @@ -223,52 +228,184 @@ class KafkaSourceSuite

[GitHub] spark issue #15594: [SPARK-18061][SQL][Security] Spark Thriftserver needs to...

2016-11-18 Thread cmirash
Github user cmirash commented on the issue: https://github.com/apache/spark/pull/15594 Thanks Luciano. I am looking at the changes and will add them soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false...

2016-11-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15820#discussion_r88758556 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala --- @@ -626,3 +776,106 @@ class KafkaSourceStressSuite

[GitHub] spark pull request #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-18 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15644#discussion_r88757732 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -282,6 +316,211 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-18 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15644#discussion_r88758268 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -282,6 +316,211 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-18 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15644#discussion_r88755426 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -282,6 +316,211 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #15644: [SPARK-18117][CORE] Add test for TaskSetBlacklist

2016-11-18 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15644#discussion_r88758327 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -992,6 +994,47 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #15820: [SPARK-18373][SS][Kafka]Make failOnDataLoss=false...

2016-11-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15820#discussion_r88758444 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/CachedKafkaConsumer.scala --- @@ -47,40 +51,191 @@ private[kafka010] case class

  1   2   3   4   5   >