[GitHub] spark issue #12004: [SPARK-7481][build] [WIP] Add Hadoop 2.7+ spark-cloud mo...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12004 **[Test build #64580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64580/consoleFull)** for PR 12004 at commit

[GitHub] spark issue #14784: [SPARK-17210][SPARKR] sparkr.zip is not distributed to e...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14784 **[Test build #64566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64566/consoleFull)** for PR 14784 at commit

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #64572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64572/consoleFull)** for PR 14452 at commit

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14712 **[Test build #64567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64567/consoleFull)** for PR 14712 at commit

[GitHub] spark issue #14691: [SPARK-16407][STREAMING] Allow users to supply custom st...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14691 **[Test build #64568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64568/consoleFull)** for PR 14691 at commit

[GitHub] spark issue #14435: [SPARK-16756][SQL][WIP] Add `sql` function to LogicalPla...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14435 **[Test build #64573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64573/consoleFull)** for PR 14435 at commit

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14527 **[Test build #64571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64571/consoleFull)** for PR 14527 at commit

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14803 **[Test build #64565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64565/consoleFull)** for PR 14803 at commit

[GitHub] spark issue #14858: [SPARK-17219][ML] Add NaN value handling in Bucketizer

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14858 **[Test build #64557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64557/consoleFull)** for PR 14858 at commit

[GitHub] spark issue #14805: [MINOR][DOCS] Fix minor typos in python example code

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14805 **[Test build #64564 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64564/consoleFull)** for PR 14805 at commit

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SparkR] Automate bui...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14859 **[Test build #64556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64556/consoleFull)** for PR 14859 at commit

[GitHub] spark issue #14855: [SPARK-17284] [SQL] Remove Statistics-related Table Prop...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14855 **[Test build #64560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64560/consoleFull)** for PR 14855 at commit

[GitHub] spark issue #14850: [SPARK-17279][SQL] better error message for NPE during S...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14850 **[Test build #64562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64562/consoleFull)** for PR 14850 at commit

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK] PEP8 on documentation examples

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14830 **[Test build #64563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64563/consoleFull)** for PR 14830 at commit

[GitHub] spark issue #14860: [SPARK-17264] [SQL] DataStreamWriter should document tha...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14860 **[Test build #64555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64555/consoleFull)** for PR 14860 at commit

[GitHub] spark issue #14862: [SPARK-17295][SQL] Create TestHiveSessionState use refle...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14862 **[Test build #64554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64554/consoleFull)** for PR 14862 at commit

[GitHub] spark issue #14864: [SPARK-15453] [SQL] FileSourceScanExec to extract `outpu...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14864 **[Test build #64552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64552/consoleFull)** for PR 14864 at commit

[GitHub] spark issue #14854: [SPARK-17283][WIP][Core] Cancel job in RDD.take() as soo...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14854 **[Test build #64561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64561/consoleFull)** for PR 14854 at commit

[GitHub] spark issue #14863: [SPARK-16992][PYSPARK] use map comprehension in doc

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14863 **[Test build #64553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64553/consoleFull)** for PR 14863 at commit

[GitHub] spark issue #14856: [SPARK-17241][SparkR][MLlib] SparkR spark.glm should hav...

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14856 **[Test build #64559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64559/consoleFull)** for PR 14856 at commit

[GitHub] spark issue #13231: [SPARK-15453] [SQL] Sort Merge Join to use bucketing met...

2016-08-29 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/13231 Continuing this work in a new PR : https://github.com/apache/spark/pull/14864 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14862: [SPARK-17295][SQL] Create TestHiveSessionState use refle...

2016-08-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14862 We are trying to get rid of `HiveSessionState`. Thus, I am not sure what you did here is in our direction. cc @cloud-fan @yhuai --- If your project is set up for it, you can reply to this

[GitHub] spark issue #14691: [SPARK-16407][STREAMING] Allow users to supply custom st...

2016-08-29 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/14691 jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14239: [SPARK-16593] [CORE] [WIP] Provide a pre-fetch mechanism...

2016-08-29 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14239 thanks for the explanation, this makes much more sense now. I'm still a bit concerned about the memory usage of this though, especially with external shuffle on the nodemanager. Were you

[GitHub] spark issue #14864: [SPARK-15453] [SQL] FileSourceScanExec to extract `outpu...

2016-08-29 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/14864 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #14864: [SPARK-15453] [SQL] FileSourceScanExec to extract...

2016-08-29 Thread tejasapatil
GitHub user tejasapatil opened a pull request: https://github.com/apache/spark/pull/14864 [SPARK-15453] [SQL] FileSourceScanExec to extract `outputOrdering` information ## What changes were proposed in this pull request? Extracting sort ordering information in

[GitHub] spark pull request #14863: [SPARK-16992][PYSPARK] use map comprehension in d...

2016-08-29 Thread Stibbons
GitHub user Stibbons opened a pull request: https://github.com/apache/spark/pull/14863 [SPARK-16992][PYSPARK] use map comprehension in doc Code is equivalent, but map comprehency is most of the time faster than a map. You can merge this pull request into a Git repository by

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/13065 @hvanhovell yea, thx for letting me know. I'll do that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-08-29 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13065 @maropu I have updated the PR. Want to take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14862: [SPARK-17295][SQL] Create TestHiveSessionState us...

2016-08-29 Thread jiangxb1987
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/14862 [SPARK-17295][SQL] Create TestHiveSessionState use reflect logic based on the setting of CATALOG_IMPLEMENTATION ## What changes were proposed in this pull request? Currently we create

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-29 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r76624379 --- Diff: python/pyspark/mllib/feature.py --- @@ -276,24 +276,64 @@ class ChiSqSelector(object): """ Creates a ChiSquared feature selector.

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r76622682 --- Diff: python/pyspark/mllib/feature.py --- @@ -276,24 +276,64 @@ class ChiSqSelector(object): """ Creates a ChiSquared feature selector.

[GitHub] spark issue #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector based ...

2016-08-29 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/14597 Hi @srowen , I have added Python API and test cases for ChiSqSelector. Could you kindly review it again. Thanks. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #11956: [SPARK-14098][SQL] Generate Java code that gets a float/...

2016-08-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/11956 @davies Could you please share your great opinions regarding these design questions among our community while we know you are busy? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #14830: [SPARK-16992][PYSPARK] PEP8 on documentation exam...

2016-08-29 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14830#discussion_r76608499 --- Diff: examples/src/main/python/als.py --- @@ -62,10 +62,10 @@ def update(i, mat, ratings): example. Please use pyspark.ml.recommendation.ALS

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK] autopep8 on documentation example...

2016-08-29 Thread Stibbons
Github user Stibbons commented on the issue: https://github.com/apache/spark/pull/14830 Cool I wasn't sure of it. No pbl, I can even split it into several PR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK] autopep8 on documentation example...

2016-08-29 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14830 For what its worth pep8 says: > The preferred way of wrapping long lines is by using Python's implied line continuation inside parentheses, brackets and braces. Long lines can be broken

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK] autopep8 on documentation example...

2016-08-29 Thread Stibbons
Github user Stibbons commented on the issue: https://github.com/apache/spark/pull/14830 Here is a new proposal. I've taken into account your remark, hope all $on/$off things are ok, and added some minor rework with the multiline syntax (I find using \ weird and inelegant, using

[GitHub] spark pull request #14830: [SPARK-16992][PYSPARK] autopep8 on documentation ...

2016-08-29 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14830#discussion_r76599413 --- Diff: examples/src/main/python/ml/aft_survival_regression.py --- @@ -17,9 +17,9 @@ from __future__ import print_function +from

[GitHub] spark pull request #14830: [SPARK-16992][PYSPARK] autopep8 on documentation ...

2016-08-29 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14830#discussion_r76598828 --- Diff: examples/src/main/python/ml/aft_survival_regression.py --- @@ -17,9 +17,9 @@ from __future__ import print_function +from

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14746#discussion_r76597119 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -105,7 +96,14 @@ case class CreateViewCommand( }

[GitHub] spark issue #14851: [SPARK-17281][ML][MLLib] Add treeAggregateDepth paramete...

2016-08-29 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/14851 cc @jkbradley thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14746#discussion_r76596767 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -69,23 +66,17 @@ case class CreateViewCommand(

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14746#discussion_r76596601 --- Diff: sql/core/src/main/java/org/apache/spark/sql/ViewType.java --- @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #14833: fixed a typo

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14833 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14833: fixed a typo

2016-08-29 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14833 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14833: fixed a typo

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14833 **[Test build #3235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3235/consoleFull)** for PR 14833 at commit

[GitHub] spark pull request #14830: [SPARK-16992][PYSPARK] autopep8 on documentation ...

2016-08-29 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14830#discussion_r76595360 --- Diff: examples/src/main/python/ml/binarizer_example.py --- @@ -17,9 +17,10 @@ from __future__ import print_function -from

[GitHub] spark pull request #14731: [SPARK-17159] [streaming]: optimise check for new...

2016-08-29 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/14731#discussion_r76593850 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -244,6 +244,31 @@ class SparkHadoopUtil extends Logging { }

[GitHub] spark pull request #14731: [SPARK-17159] [streaming]: optimise check for new...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14731#discussion_r76594233 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -244,6 +244,31 @@ class SparkHadoopUtil extends Logging { }

[GitHub] spark issue #14833: fixed a typo

2016-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14833 **[Test build #3235 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3235/consoleFull)** for PR 14833 at commit

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-29 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/14118 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-08-29 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/14118 > What if I am writing explicitly an empty string out? Does it become just 1,,2? Yes. It becomes `1,,2` in 2.0, and the same `1,,2` with this patch -- no behavior changes. > Can

[GitHub] spark pull request #14861: [SPARK-17287] [PySpark] Add `recursive` kwarg to ...

2016-08-29 Thread jpiper
GitHub user jpiper opened a pull request: https://github.com/apache/spark/pull/14861 [SPARK-17287] [PySpark] Add `recursive` kwarg to Java Python `SparkContext.addFile` ## What changes were proposed in this pull request? Add the ability to add entire directories using the

[GitHub] spark pull request #14830: [SPARK-16992][PYSPARK] autopep8 on documentation ...

2016-08-29 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14830#discussion_r76582404 --- Diff: examples/src/main/python/ml/binarizer_example.py --- @@ -17,9 +17,10 @@ from __future__ import print_function -from

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-29 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r76582188 --- Diff: python/pep8rc --- @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] spark pull request #14860: [SPARK-17264] [SQL] DataStreamWriter should docum...

2016-08-29 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/14860 [SPARK-17264] [SQL] DataStreamWriter should document that it only supports Parquet for now ## What changes were proposed in this pull request? Clarify that only parquet files are supported

[GitHub] spark pull request #14536: Merge pull request #1 from apache/master

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14536 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14449: [SPARK-16843][MLLIB] add the percentage ChiSquare...

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14449 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #10572: SPARK-12619 Combine small files in a hadoop direc...

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10572 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #10995: [SPARK-13120] [test-maven] Shade protobuf-java

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10995 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #12695: [SPARK-14914] Normalize Paths/URIs for windows.

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12695 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #13658: [SPARK-15937] [yarn] Improving the logic to wait ...

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13658 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14505: Branch 2.0

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14505 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14810: Branch 1.6

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14810 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #12694: [SPARK-14914] Fix Command too long for windows. E...

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12694 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #12753: [SPARK-3767] [CORE] Support wildcard in Spark pro...

2016-08-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12753 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-08-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14788 For `date_add`, `date_sub`, `add_month`, I think we should support both DateType and TimestampType, and the return type should depend on the input type. For `last_day`, `first_day`, we

[GitHub] spark pull request #14849: [BUILD] Closes some stale PRs.

2016-08-29 Thread srowen
Github user srowen closed the pull request at: https://github.com/apache/spark/pull/14849 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r76580634 --- Diff: python/pep8rc --- @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SparkR] Automate bui...

2016-08-29 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14859 Good point, I suppose there is a weak promise there that it runs on Windows. Could anyone else who knows Windows weigh in? I assume @dongjoon-hyun is on board. --- If your project is set up

[GitHub] spark pull request #14698: [SPARK-17061][SPARK-17093][SQL] `MapObjects` shou...

2016-08-29 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/14698#discussion_r76579917 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -136,7 +136,7 @@ trait

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SparkR] Automate bui...

2016-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14859 Ah, I thought Windows is already officially supported assuming from this documentation https://github.com/apache/spark/blob/master/docs/index.md#downloading. BTW, I do understand your

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-29 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14712 Looks like Jenkins doesn't work for a while. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-29 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r76577358 --- Diff: python/pep8rc --- @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-29 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r76577137 --- Diff: dev/isort.cfg --- @@ -1,9 +1,9 @@ # Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements.

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14712 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14712 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SparkR] Automate bui...

2016-08-29 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14859 Hm, we also had Travis config that isn't used now, to try to add Java style checking. I can see the value in adding Windows testing, but here we have a third CI tool involved. I'm concerned that I

[GitHub] spark pull request #14698: [SPARK-17061][SPARK-17093][SQL] `MapObjects` shou...

2016-08-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/14698#discussion_r76576622 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -136,7 +136,7 @@ trait

[GitHub] spark issue #14859: [SPARK-17200][PROJECT INFRA][BUILD][SparkR] Automate bui...

2016-08-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14859 cc @rxin @srowen (for build) @JoshRosen (for project infra) @dongjoon-hyun (who suggested AppVeyor CI) @steveloughran (who is the author of winutils) @felixcheung and

[GitHub] spark issue #14833: fixed a typo

2016-08-29 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14833 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #14859: [SPARK-17200][PROJECT INFRA][BUILD][SparkR] Autom...

2016-08-29 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/14859 [SPARK-17200][PROJECT INFRA][BUILD][SparkR] Automate building and testing on Windows (currently SparkR only) ## What changes were proposed in this pull request? This PR adds the build

[GitHub] spark issue #14836: [MINOR][MLlib][SQL] Clean up unused variables and unused...

2016-08-29 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14836 I think the other changes are trivial but not wrong. I'd generally not bother with these bitty changes. It's not that they're wrong but that it takes me some time to go think through whether they're

[GitHub] spark pull request #14836: [MINOR][MLlib][SQL] Clean up unused variables and...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14836#discussion_r76575766 --- Diff: core/src/test/scala/org/apache/spark/AccumulatorSuite.scala --- @@ -171,7 +173,7 @@ class AccumulatorSuite extends SparkFunSuite with Matchers with

[GitHub] spark pull request #14836: [MINOR][MLlib][SQL] Clean up unused variables and...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14836#discussion_r76575699 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -1452,7 +1452,6 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark issue #14650: [SPARK-17062][MESOS] add conf option to mesos dispatcher

2016-08-29 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/14650 WIP --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r76574930 --- Diff: python/pep8rc --- @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r76574875 --- Diff: dev/isort.cfg --- @@ -1,9 +1,9 @@ # Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements.

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r76574749 --- Diff: dev/isort.cfg --- @@ -1,9 +1,9 @@ # Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements.

[GitHub] spark issue #14805: [MINOR][DOCS] Fix minor typos in python example code

2016-08-29 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14805 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14826: [SPARK-16832] [MLLIB] Standard Python-Java MLlib API to ...

2016-08-29 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14826 @mengxr what do you think about this narrower change? just double-checking. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14783: SPARK-16785 R dapply doesn't return array or raw columns

2016-08-29 Thread clarkfitzg
Github user clarkfitzg commented on the issue: https://github.com/apache/spark/pull/14783 Tried some more benchmarks today. Didn't see any difference in speed before / after patch. Observing the processes as they run I see the vast majority of time spent in the local R process, while

[GitHub] spark pull request #14731: [SPARK-17159] [streaming]: optimise check for new...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14731#discussion_r76573274 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -244,6 +244,31 @@ class SparkHadoopUtil extends Logging { }

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution by dedupli...

2016-08-29 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14452 Jenkins seems not working? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #14858: [SPARK-17219][ML] Add NaN value handling in Bucke...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14858#discussion_r76572693 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -116,8 +116,7 @@ final class QuantileDiscretizer @Since("1.6.0")

[GitHub] spark pull request #14858: [SPARK-17219][ML] Add NaN value handling in Bucke...

2016-08-29 Thread VinceShieh
Github user VinceShieh commented on a diff in the pull request: https://github.com/apache/spark/pull/14858#discussion_r76572410 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -116,8 +116,7 @@ final class QuantileDiscretizer

[GitHub] spark pull request #14858: [SPARK-17219][ML] Add NaN value handling in Bucke...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14858#discussion_r76572124 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -63,7 +63,7 @@ final class Bucketizer @Since("1.4.0") (@Since("1.4.0")

[GitHub] spark pull request #14858: [SPARK-17219][ML] Add NaN value handling in Bucke...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14858#discussion_r76571886 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -129,17 +129,21 @@ object Bucketizer extends

[GitHub] spark pull request #14858: [SPARK-17219][ML] Add NaN value handling in Bucke...

2016-08-29 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14858#discussion_r76571614 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -116,8 +116,7 @@ final class QuantileDiscretizer @Since("1.6.0")

<    1   2   3   4   5   6   7   >