[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16537 **[Test build #72826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72826/testReport)** for PR 16537 at commit

[GitHub] spark pull request #16672: [SPARK-19329][SQL]Reading from or writing to a da...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16672#discussion_r100882025 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1816,4 +1816,127 @@ class DDLSuite extends QueryTest

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100881040 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/Range.scala --- @@ -0,0 +1,120 @@ +/* + *

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL] Add udf decorator

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16533 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16672: [SPARK-19329][SQL]Reading from or writing to a da...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16672#discussion_r100882612 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1816,4 +1816,127 @@ class DDLSuite extends QueryTest

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-02-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16845 +1 to @HyukjinKwon comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and extend DataF...

2017-02-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16792 thanks for putting in the time to come up with a clear warning about how the types are handled. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #16880: [SPARK-19542][SS]Delete the temp checkpoint if a ...

2017-02-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16880 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and extend DataF...

2017-02-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16792 Can you update this to the latest master? (The merge tool has some warnings). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16909 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16859: [SPARK-17714][Core][test-maven][test-hadoop2.6]Avoid usi...

2017-02-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16859 Thanks! Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16859: [SPARK-17714][Core][test-maven][test-hadoop2.6]Avoid usi...

2017-02-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16859 @vanzin I just merged this one. If you have other comments, I will address them in a separate PR. Thanks! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #16859: [SPARK-17714][Core][test-maven][test-hadoop2.6]Av...

2017-02-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16859 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16859: [SPARK-17714][Core][test-maven][test-hadoop2.6]Avoid usi...

2017-02-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16859 Nope, don't have anything else. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16909 **[Test build #72827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72827/testReport)** for PR 16909 at commit

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16916 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16916 **[Test build #72828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72828/testReport)** for PR 16916 at commit

[GitHub] spark pull request #16901: [SPARK-19565] Improve DAGScheduler tests.

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16901#discussion_r100886876 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,48 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16901: [SPARK-19565] Improve DAGScheduler tests.

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16901#discussion_r100886451 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,48 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16901: [SPARK-19565] Improve DAGScheduler tests.

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16901#discussion_r100885960 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,48 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16901: [SPARK-19565] Improve DAGScheduler tests.

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16901#discussion_r100885083 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,48 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16901: [SPARK-19565] Improve DAGScheduler tests.

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16901#discussion_r100886335 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,48 @@ class DAGSchedulerSuite extends

[GitHub] spark issue #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and extend DataF...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16792 **[Test build #72829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72829/testReport)** for PR 16792 at commit

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/16537 For me it is all about the bigger picture. I've been working with Python for quite a while right now (probably to long for my own good) and I am used to two things: - Language is

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16777 **[Test build #72819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72819/testReport)** for PR 16777 at commit

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100896576 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala --- @@ -0,0 +1,316 @@ +/*

[GitHub] spark issue #16891: [SPARK-19318][SQL] Fix to treat JDBC connection properti...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16891 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72820/ Test PASSed. ---

[GitHub] spark issue #16868: [SPARK-19115] [SQL] Supporting Create Table Like Locatio...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16868 Yes, the location can be the same or different from the original table. LGTM pending test --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100904151 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/JoinEstimationSuite.scala --- @@ -0,0 +1,314 @@ +/* + *

[GitHub] spark issue #16841: [SPARK-18871][SQL][TESTS] New test cases for IN/NOT IN s...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16841 OK to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16878: [SPARK-19539][SQL] Block duplicate temp table during cre...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16878 **[Test build #72834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72834/testReport)** for PR 16878 at commit

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16537 **[Test build #72826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72826/testReport)** for PR 16537 at commit

[GitHub] spark issue #16866: [SPARK-19529] TransportClientFactory.createClient() shou...

2017-02-13 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/16866 Branch-2.1 test compilation happens to be broken right now. Trying to fix the compilation failure first. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #16776: [SPARK-19436][SQL] Add missing tests for approxQu...

2017-02-13 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16776#discussion_r100891313 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -58,49 +58,54 @@ final class DataFrameStatFunctions

[GitHub] spark issue #16891: [SPARK-19318][SQL] Fix to treat JDBC connection properti...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16891 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16777 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100901206 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/StatsEstimationTestBase.scala --- @@ -21,14 +21,19 @@ import

[GitHub] spark pull request #16915: [SPARK-18871][SQL][TESTS] New test cases for IN/N...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16915#discussion_r100905620 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/in-subquery/in-with-cte.sql.out --- @@ -0,0 +1,368 @@ +-- Automatically generated

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16916 **[Test build #72832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72832/testReport)** for PR 16916 at commit

[GitHub] spark issue #16841: [SPARK-18871][SQL][TESTS] New test cases for IN/NOT IN s...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16841 **[Test build #72833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72833/testReport)** for PR 16841 at commit

[GitHub] spark pull request #16674: [SPARK-19331][SQL][TESTS] Improve the test covera...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16674#discussion_r100906893 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala --- @@ -15,29 +15,34 @@ * limitations under the License.

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16537 I think the overhead of doing this piecemeal removes review time available for more important changes (like places where users are actively encountering confusing error messages, incorrect

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16620#discussion_r100921953 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,58 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16620#discussion_r100922486 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,58 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16620#discussion_r100922127 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,58 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16620#discussion_r100922010 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,58 @@ class DAGSchedulerSuite extends

[GitHub] spark issue #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and extend DataF...

2017-02-13 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/16792 Done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16672: [SPARK-19329][SQL]Reading from or writing to a datasourc...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16672 Actually, I found another issue in CTAS with pre-existing location. Maybe you can take that too? https://issues.apache.org/jira/browse/SPARK-19583 --- If your project is set up for it, you can

[GitHub] spark issue #16777: [SPARK-19435][SQL] Type coercion between ArrayTypes

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16777 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/16537 And there is of course a matter of user experience. Even if failure is cheap, something like this: ```python In [4]: from pyspark.sql.functions import udf In [5]: udf(lambda

[GitHub] spark pull request #16741: [SPARK-19402][DOCS] Support LaTex inline formula ...

2017-02-13 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/16741#discussion_r100900474 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -135,13 +135,13 @@ abstract class MLWriter extends BaseReadWrite with

[GitHub] spark issue #16868: [SPARK-19115] [SQL] Supporting Create Table Like Locatio...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16868 **[Test build #72830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72830/testReport)** for PR 16868 at commit

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16537 @rdblue i think we're maybe understanding different type checks. My understanding is in this case the error is already thrown right away. It's also not that the user needs to pass a callable here,

[GitHub] spark issue #11211: [SPARK-13330][PYSPARK] PYTHONHASHSEED is not propgated t...

2017-02-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/11211 gentle re-ping - is this something you have badnwidth to work on @zjffdu? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16578 **[Test build #72836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72836/testReport)** for PR 16578 at commit

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16916 Is it possible to merge the stat cache and the symlink cache? It seems both are sort of doing the same thing. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16699: [SPARK-18710][ML] Add offset in GLM

2017-02-13 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16699 @sethah The predict method can work with new data in R. See below. Shall we focus on the current implementation, instead of discussing the details of the R behavior? :) Let me know if

[GitHub] spark issue #16771: [SPARK-19429][PYTHON][SQL] Support slice arguments in Co...

2017-02-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16771 Thanks for cleaning this up (yay removing deprecated functions from Python 2.0 :)). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and extend DataF...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16792 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16792: [SPARK-19453][PYTHON][SQL][DOC] Correct and extend DataF...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16792 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72829/ Test PASSed. ---

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100897179 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala --- @@ -0,0 +1,316 @@ +/*

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16777#discussion_r100898554 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -117,49 +115,67 @@ object TypeCoercion {

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/16537 I explore an alternative approach, with adding type hints (https://github.com/zero323/pyspark-stubs), but I doubt it'll become particularly popular, and I won't even try to push it to the main

[GitHub] spark issue #16868: [SPARK-19115] [SQL] Supporting Create Table Like Locatio...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16868 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16802: [SPARK-18872][SQL][TESTS] New test cases for EXISTS subq...

2017-02-13 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/16802 cc @hvanhovell @gatorsmile Thanks @nsyca @dongjoon-hyun for reviewing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16841: [SPARK-18871][SQL][TESTS] New test cases for IN/NOT IN s...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16841 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16917: [SPARK-19529][BRANCH-1.6] Backport PR #16866 to branch-1...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16917 **[Test build #72835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72835/consoleFull)** for PR 16917 at commit

[GitHub] spark issue #16866: [SPARK-19529] TransportClientFactory.createClient() shou...

2017-02-13 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/16866 Merged to master, branch-2.1, and branch-2.0. Files involved in branch-1.6 were moved to new directories and made it hard to cherry-pick. Created PR #16917 to backport this one to 1.6.

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16916 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72832/ Test PASSed. ---

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16537 Ah perhaps then we are simply argeeing with each-other. I'm fine with adding these types of fixes - but doing it one function at a time is just going to be too time consuming and distracting from

[GitHub] spark issue #16533: [SPARK-19160][PYTHON][SQL] Add udf decorator

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16533 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72825/ Test PASSed. ---

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72826/ Test PASSed. ---

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16916 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72828/ Test PASSed. ---

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16916 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100894110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala --- @@ -0,0 +1,316 @@ +/*

[GitHub] spark issue #16891: [SPARK-19318][SQL] Fix to treat JDBC connection properti...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16891 **[Test build #72820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72820/testReport)** for PR 16891 at commit

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100899309 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala --- @@ -0,0 +1,316 @@ +/*

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/16537 Sorry, my example was for validating the object passed to `udf` was callable, not for the use of the UDF. I still think it's a good idea not to make assumptions about how a user makes a mistake.

[GitHub] spark pull request #16228: [SPARK-17076] [SQL] Cardinality estimation for jo...

2017-02-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16228#discussion_r100902029 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/Range.scala --- @@ -0,0 +1,120 @@ +/* + *

[GitHub] spark pull request #16841: [SPARK-18871][SQL][TESTS] New test cases for IN/N...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16841#discussion_r100907556 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/in-multiple-columns.sql --- @@ -0,0 +1,127 @@ +-- A test suite for

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16535 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16535 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72831/ Test PASSed. ---

[GitHub] spark issue #16535: [SPARK-19162][PYTHON][SQL] UserDefinedFunction should va...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16535 **[Test build #72831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72831/testReport)** for PR 16535 at commit

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16909 **[Test build #72827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72827/testReport)** for PR 16909 at commit

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r100910764 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCWriteSuite.scala --- @@ -75,7 +75,7 @@ class JDBCWriteSuite extends SharedSQLContext

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16909 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72827/ Test FAILed. ---

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16909 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16917: [SPARK-19529][BRANCH-1.6] Backport PR #16866 to b...

2017-02-13 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/16917 [SPARK-19529][BRANCH-1.6] Backport PR #16866 to branch-1.6 ## What changes were proposed in this pull request? This PR backports PR #16866 to branch-1.6 ## How was this patch

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16916 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16862: [SPARK-19520][streaming] Do not encrypt data writ...

2017-02-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16862 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16916 **[Test build #72832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72832/testReport)** for PR 16916 at commit

[GitHub] spark issue #16916: [SPARK-19501][YARN] Reduce the number of HDFS RPCs durin...

2017-02-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16916 **[Test build #72828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72828/testReport)** for PR 16916 at commit

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-02-13 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/16537 I don't think it is a good idea to think that this has little use because it is a dumb mistake to pass something that isn't callable. In this case, it's easy to accidentally reuse a name for a

[GitHub] spark pull request #16891: [SPARK-19318][SQL] Fix to treat JDBC connection p...

2017-02-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16891#discussion_r100910994 --- Diff: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala --- @@ -149,4 +172,16 @@ class

[GitHub] spark pull request #16714: [SPARK-16333][Core] Enable EventLoggingListener t...

2017-02-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16714#discussion_r100917783 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -64,6 +64,12 @@ private[spark] class EventLoggingListener(

[GitHub] spark pull request #16714: [SPARK-16333][Core] Enable EventLoggingListener t...

2017-02-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16714#discussion_r100918633 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -64,6 +64,12 @@ private[spark] class EventLoggingListener(

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16620#discussion_r100922388 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,58 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16620#discussion_r100921788 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,58 @@ class DAGSchedulerSuite extends

[GitHub] spark pull request #16620: [SPARK-19263] DAGScheduler should avoid sending c...

2017-02-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16620#discussion_r100921653 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2161,6 +2161,58 @@ class DAGSchedulerSuite extends

  1   2   3   4   5   6   7   >