[GitHub] spark issue #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source trait ...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14553 **[Test build #67004 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67004/consoleFull)** for PR 14553 at commit

[GitHub] spark issue #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source trait ...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14553 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67004/ Test PASSed. ---

[GitHub] spark issue #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source trait ...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14553 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15497: [Test][SPARK-16002][Follow-up] Fix flaky test in Streami...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15497 **[Test build #67006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67006/consoleFull)** for PR 15497 at commit

[GitHub] spark issue #15497: [Test][SPARK-16002][Follow-up] Fix flaky test in Streami...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15497 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67006/ Test FAILed. ---

[GitHub] spark issue #15497: [Test][SPARK-16002][Follow-up] Fix flaky test in Streami...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15497 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14847: [SPARK-17254][SQL] Add StopAfter physical plan fo...

2016-10-15 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/14847 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15498: [SPARK-17953] [Documentation] Fix typo in SparkSe...

2016-10-15 Thread tae-jun
GitHub user tae-jun opened a pull request: https://github.com/apache/spark/pull/15498 [SPARK-17953] [Documentation] Fix typo in SparkSession scaladoc ## What changes were proposed in this pull request? ### Before: ```scala SparkSession.builder()

[GitHub] spark issue #15498: [SPARK-17953] [Documentation] Fix typo in SparkSession s...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15498 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15498: [SPARK-17953] [Documentation] Fix typo in SparkSession s...

2016-10-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15498 Thanks - merging in master/branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15497: [Test][SPARK-16002][Follow-up] Fix flaky test in Streami...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15497 **[Test build #3343 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3343/consoleFull)** for PR 15497 at commit

[GitHub] spark pull request #15498: [SPARK-17953] [Documentation] Fix typo in SparkSe...

2016-10-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15498 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrdering` gro...

2016-10-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15480 cc @ueshin want to help review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15319: [SPARK-17733][SQL] InferFiltersFromConstraints rule neve...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15319 **[Test build #67007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67007/consoleFull)** for PR 15319 at commit

[GitHub] spark issue #15319: [SPARK-17733][SQL] InferFiltersFromConstraints rule neve...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15319 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67007/ Test FAILed. ---

[GitHub] spark issue #15319: [SPARK-17733][SQL] InferFiltersFromConstraints rule neve...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15319 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15471: [WIP][SPARK-17919] Make timeout to RBackend configurable...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15471 **[Test build #67005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67005/consoleFull)** for PR 15471 at commit

[GitHub] spark issue #15474: [DO_NOT_MERGE] Test netty

2016-10-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15474 Did you clear your locally built artifacts first? maybe that's the difference. The Jenkins test here hits the same problem I was seeing. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-15 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15218 I am assuming @kayousterhout does not have comments on this. Can you please fix the conflict @zhzhan ? I will merge it in after that to master. --- If your project is set up for it, you can

[GitHub] spark issue #15497: [Test][SPARK-16002][Follow-up] Fix flaky test in Streami...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15497 **[Test build #3343 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3343/consoleFull)** for PR 15497 at commit

[GitHub] spark issue #15411: Set master URL configuration in scala example

2016-10-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15411 Hm, so I see several more examples that get included in documentation that don't set master. I am not sure that is a salient difference, because in general, when writing your own app you would not

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15450 **[Test build #67009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67009/consoleFull)** for PR 15450 at commit

[GitHub] spark issue #15471: [WIP][SPARK-17919] Make timeout to RBackend configurable...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15471 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67005/ Test FAILed. ---

[GitHub] spark issue #15471: [WIP][SPARK-17919] Make timeout to RBackend configurable...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15471 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrderi...

2016-10-15 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15480#discussion_r83528941 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala --- @@ -118,7 +118,45 @@ object

[GitHub] spark pull request #15495: [SPARK-17620][SQL] Determine Serde by hive.defaul...

2016-10-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15495#discussion_r83528964 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -587,6 +594,30 @@ class SQLQuerySuite extends QueryTest

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r83529473 --- Diff: core/src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java --- @@ -0,0 +1,142 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r83529504 --- Diff: core/src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java --- @@ -0,0 +1,142 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r83529508 --- Diff: core/src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java --- @@ -0,0 +1,142 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r83529483 --- Diff: core/src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java --- @@ -0,0 +1,142 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #15408: [SPARK-17839][CORE] Use Nio's directbuffer instea...

2016-10-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15408#discussion_r83529476 --- Diff: core/src/main/java/org/apache/spark/io/NioBufferedFileInputStream.java --- @@ -0,0 +1,142 @@ +/* + * Licensed under the Apache License,

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15450 @sethah I wanted to check how strongly against this kind of change you might be, and continue to discussion here. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrderi...

2016-10-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15480#discussion_r83530039 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala --- @@ -118,7 +118,45 @@ object

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-10-15 Thread philipphoffmann
Github user philipphoffmann commented on the issue: https://github.com/apache/spark/pull/14936 Alright, I changed the implementation to keep the existing defaults. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14936 **[Test build #67010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67010/consoleFull)** for PR 14936 at commit

[GitHub] spark pull request #15480: [SPARK-16845][SQL] `GeneratedClass$SpecificOrderi...

2016-10-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15480#discussion_r83530625 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala --- @@ -118,7 +118,45 @@ object

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15450 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67009/ Test PASSed. ---

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15450 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15450 **[Test build #67009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67009/consoleFull)** for PR 15450 at commit

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14936 **[Test build #67010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67010/consoleFull)** for PR 14936 at commit

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14936 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14936 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67010/ Test PASSed. ---

[GitHub] spark pull request #15499: [SPARK-17955][SQL] Make DataFrameReader.jdbc call...

2016-10-15 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/15499 [SPARK-17955][SQL] Make DataFrameReader.jdbc call DataFrameReader.format("jdbc").load ## What changes were proposed in this pull request? This PR proposes to make

[GitHub] spark issue #15499: [SPARK-17955][SQL] Make DataFrameReader.jdbc call DataFr...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15499 **[Test build #67011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67011/consoleFull)** for PR 15499 at commit

[GitHub] spark issue #15504: [SPARK-17812][SQL][KAFKA] Assign and specific startingOf...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15504 **[Test build #67025 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67025/consoleFull)** for PR 15504 at commit

[GitHub] spark issue #15436: [SPARK-17875] [BUILD] Remove unneeded direct dependence ...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15436 **[Test build #3344 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3344/consoleFull)** for PR 15436 at commit

[GitHub] spark issue #15500: [SPARK-17956][SQL] Fix projection output ordering

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15500 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null/long as input ...

2016-10-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15432 One more suggestion to your future PR. Whenever you submit a PR, please try to improve the test case coverage. This can help you find bugs in your codes and also benefit the whole community.

[GitHub] spark pull request #15504: [SPARK-17812][SQL][KAFKA] Assign and specific sta...

2016-10-15 Thread koeninger
GitHub user koeninger opened a pull request: https://github.com/apache/spark/pull/15504 [SPARK-17812][SQL][KAFKA] Assign and specific startingOffsets for structured stream ## What changes were proposed in this pull request? startingOffsets takes specific per-topicpartition

[GitHub] spark pull request #15503: Fix example of tf_idf with minDocFreq

2016-10-15 Thread maximerihouey
GitHub user maximerihouey opened a pull request: https://github.com/apache/spark/pull/15503 Fix example of tf_idf with minDocFreq ## What changes were proposed in this pull request? The python example for tf_idf with the parameter "minDocFreq" is not properly set up

[GitHub] spark issue #15503: Fix example of tf_idf with minDocFreq

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15503 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15436: [SPARK-17875] [BUILD] Remove unneeded direct dependence ...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15436 **[Test build #3344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3344/consoleFull)** for PR 15436 at commit

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null/long as input ...

2016-10-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15432 I didn't understand. I would like to be very sure on this to avoid mistakes in my future PRs. > the current change in this PR also allows `rand` to take an expression that returns a

[GitHub] spark issue #15500: [SPARK-17956][SQL] Fix projection output ordering

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15500 **[Test build #67022 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67022/consoleFull)** for PR 15500 at commit

[GitHub] spark issue #15500: [SPARK-17956][SQL] Fix projection output ordering

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15500 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67022/ Test PASSed. ---

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null/long as input ...

2016-10-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15432 Here, I am talking about the black box testing. If you add the new capability to any external function, you should add it in the test cases. This is very fundamental when we developing

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null/long as input ...

2016-10-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15432 Regarding the test cases of R and PythonR, I am fine if you do not add them into the code base. However, please at least run them manually. We hit many surprise bugs in the past just because

[GitHub] spark issue #15500: [SPARK-17956][SQL] Fix projection output ordering

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15500 **[Test build #67022 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67022/consoleFull)** for PR 15500 at commit

[GitHub] spark issue #15272: [SPARK-17698] [SQL] Join predicates should not contain f...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15272 **[Test build #67023 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67023/consoleFull)** for PR 15272 at commit

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-15 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15218 Merged to master, thanks @zhzhan ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15218 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15218 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67021/ Test PASSed. ---

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15218 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-15 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/15218 @mridulm Thanks for reviewing this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15503: Fix example of tf_idf with minDocFreq

2016-10-15 Thread maximerihouey
Github user maximerihouey commented on the issue: https://github.com/apache/spark/pull/15503 Yes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15475: [WIP] [SQL] Enable metastore partition pruning by defaul...

2016-10-15 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15475 Hm, seems the 1.3 client breaks all the tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14947: [SPARK-17388][SQL] Support for inferring type date/times...

2016-10-15 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14947 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15272: [SPARK-17698] [SQL] Join predicates should not contain f...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15272 **[Test build #67023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67023/consoleFull)** for PR 15272 at commit

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15218 **[Test build #67021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67021/consoleFull)** for PR 15218 at commit

[GitHub] spark issue #15493: [SPARK-17946][PYSPARK] Python crossJoin API similar to S...

2016-10-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15493 @rxin In R, CrossJoin is the default when expr is empty (https://github.com/apache/spark/blob/master/R/pkg/R/DataFrame.R#L2304) I reviewed the code and documentation I think it is sufficient

[GitHub] spark issue #15493: [SPARK-17946][PYSPARK] Python crossJoin API similar to S...

2016-10-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15493 The issue is that we want to prevent users from shooting themselves in the foot, i.e. we want to avoid accidental cross joins. The idea is unless the user explicitly says crossJoin, we should disallow

[GitHub] spark issue #15272: [SPARK-17698] [SQL] Join predicates should not contain f...

2016-10-15 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15272 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15272: [SPARK-17698] [SQL] Join predicates should not contain f...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15272 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67023/ Test PASSed. ---

[GitHub] spark issue #15272: [SPARK-17698] [SQL] Join predicates should not contain f...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15272 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14995 **[Test build #67024 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67024/consoleFull)** for PR 14995 at commit

[GitHub] spark issue #15504: [SPARK-17812][SQL][KAFKA] Assign and specific startingOf...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15504 **[Test build #67026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67026/consoleFull)** for PR 15504 at commit

[GitHub] spark issue #15272: [SPARK-17698] [SQL] Join predicates should not contain f...

2016-10-15 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15272 @cloud-fan + @rxin : Fixed the test case. Ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14531: [SPARK-17353] [SPARK-16943] [SPARK-16942] [SQL] Fix mult...

2016-10-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14531 Sure, will submit a PR for it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14531: [SPARK-17353] [SPARK-16943] [SPARK-16942] [SQL] Fix mult...

2016-10-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14531 @gatorsmile / @sitalkedia that idea sounds good (similar to Hive's) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83545541 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14995 **[Test build #67024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67024/consoleFull)** for PR 14995 at commit

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83545544 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15218 @zhzhan and @mridulm all the classes need to be private[scheduler] shouldn't they? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14995 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67024/ Test PASSed. ---

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14995 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83545613 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83545669 --- Diff: docs/configuration.md --- @@ -1334,6 +1334,17 @@ Apart from these, the following properties are also available, and may be useful Should be

[GitHub] spark issue #14124: [SPARK-16472][SQL] Inconsistent nullability in schema af...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14124 **[Test build #67029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67029/consoleFull)** for PR 14124 at commit

[GitHub] spark pull request #15474: [DO_NOT_MERGE] Test netty

2016-10-15 Thread zsxwing
Github user zsxwing closed the pull request at: https://github.com/apache/spark/pull/15474 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14627: [SPARK-16975][SQL][FOLLOWUP] Do not duplicately check fi...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14627 **[Test build #67027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67027/consoleFull)** for PR 14627 at commit

[GitHub] spark issue #15504: [SPARK-17812][SQL][KAFKA] Assign and specific startingOf...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67026/ Test PASSed. ---

[GitHub] spark issue #15504: [SPARK-17812][SQL][KAFKA] Assign and specific startingOf...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15504 **[Test build #67026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67026/consoleFull)** for PR 15504 at commit

[GitHub] spark issue #15504: [SPARK-17812][SQL][KAFKA] Assign and specific startingOf...

2016-10-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15504 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83545645 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -109,6 +109,72 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83545638 --- Diff: docs/configuration.md --- @@ -1334,6 +1334,17 @@ Apart from these, the following properties are also available, and may be useful Should be

[GitHub] spark issue #14451: [SPARK-16848][SQL] Make jdbc() and read.format("jdbc") c...

2016-10-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14451 **[Test build #67028 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67028/consoleFull)** for PR 14451 at commit

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15218 @zhzhan in general it'd be great to have proper documentation on the classes. For example, it is important to document the behavior of the various assigners, and even more importantly, document the

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83545573 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #15493: [SPARK-17946][PYSPARK] Python crossJoin API similar to S...

2016-10-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15493 That's a great point. Currently R is the same as Python in that when `joinExpr` is `NULL` (R) or `on` is `None` (Python), CrossJoin is assumed. (Python

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83545554 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache Software Foundation

  1   2   >