[GitHub] spark pull request #16016: Branch 2.1

2016-11-25 Thread horo90
Github user horo90 closed the pull request at: https://github.com/apache/spark/pull/16016 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16016: Branch 2.1

2016-11-25 Thread horo90
GitHub user horo90 reopened a pull request: https://github.com/apache/spark/pull/16016 Branch 2.1 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch

[GitHub] spark pull request #16016: Branch 2.1

2016-11-25 Thread horo90
Github user horo90 closed the pull request at: https://github.com/apache/spark/pull/16016 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16003: [SPARK-18482][SQL] make sure Spark can access the table ...

2016-11-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16003 How about Spark 2.1 altering the table metadata created by Spark 2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15975 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15975 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69180/ Test PASSed. ---

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15975 **[Test build #69180 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69180/consoleFull)** for PR 15975 at commit

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16015 **[Test build #69181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69181/consoleFull)** for PR 16015 at commit

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16015 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16015 The only one test failure is irrelevant to this PR. ``` [info] - set spark.sql.warehouse.dir *** FAILED *** (5 minutes, 0 seconds) [info] Timeout of './bin/spark-submit' '--class'

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15975 @gatorsmile NP. Thank you for informing that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16016: Branch 2.1

2016-11-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16016 Cloud you please close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16016: Branch 2.1

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16016 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16016: Branch 2.1

2016-11-25 Thread horo90
GitHub user horo90 opened a pull request: https://github.com/apache/spark/pull/16016 Branch 2.1 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch

[GitHub] spark issue #15662: [SPARK-18141][SQL] Fix to quote column names in the pred...

2016-11-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15662 @sureshthalamati Could you resolve the conflict? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15662: [SPARK-18141][SQL] Fix to quote column names in t...

2016-11-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15662#discussion_r89666271 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -172,7 +172,7 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16013 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69179/ Test FAILed. ---

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16013 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16013 **[Test build #69179 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69179/consoleFull)** for PR 16013 at commit

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15975 **[Test build #69180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69180/consoleFull)** for PR 15975 at commit

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15975 @dongjoon-hyun Will not add test cases for the write path in this PR, because it requires code changes on the source codes. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16015 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16015 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69178/ Test FAILed. ---

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16015 **[Test build #69178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69178/consoleFull)** for PR 16015 at commit

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89665558 --- Diff: core/src/main/scala/org/apache/spark/rdd/DoubleRDDFunctions.scala --- @@ -155,7 +155,7 @@ class DoubleRDDFunctions(self: RDD[Double]) extends

[GitHub] spark pull request #16007: [SPARK-18583][SQL] Fix nullability of InputFileNa...

2016-11-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16007 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Merging in master/branch-2.1. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89665095 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2063,6 +2063,7 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89664964 --- Diff: core/src/main/scala/org/apache/spark/Accumulator.scala --- @@ -26,7 +26,7 @@ package org.apache.spark * * An accumulator is created

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89664921 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -262,8 +262,9 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89664911 --- Diff: core/src/main/scala/org/apache/spark/Accumulator.scala --- @@ -26,7 +26,7 @@ package org.apache.spark * * An accumulator is created

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16013 **[Test build #69179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69179/consoleFull)** for PR 16013 at commit

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 Forget to say, of course, this example will thrown the exception only running in "test". Other developers would possibly encounter this when they write test codes in the future. If we could

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16015 **[Test build #69178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69178/consoleFull)** for PR 16015 at commit

[GitHub] spark issue #16012: [SPARK-17251][SQL] Support `OuterReference` in projectio...

2016-11-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16012 Thank you for review, @hvanhovell and @nsyca . I agree with you. We need enough time for this. So, the option one for 2.1 is spun off into #16015. --- If your project is set up for

[GitHub] spark pull request #16015: [SPARK-17251][SQL] Improve `OuterReference` to be...

2016-11-25 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/16015 [SPARK-17251][SQL] Improve `OuterReference` to be `NamedExpression` ## What changes were proposed in this pull request? Currently, `OuterReference` is not `NamedExpression`. So, it

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16007 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69177/ Test PASSed. ---

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16007 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16007 **[Test build #69177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69177/consoleFull)** for PR 16007 at commit

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15975 Will update it tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 The test case I added in this pr: val rng = new scala.util.Random(42) val data = sparkContext.parallelize(Seq.tabulate(100) { i => Row(Array.fill(10)(rng.nextInt(10)))

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89664100 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -989,7 +989,7 @@ class Analyzer(

[GitHub] spark issue #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE and DESC ...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15358 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69176/ Test PASSed. ---

[GitHub] spark issue #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE and DESC ...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15358 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE and DESC ...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15358 **[Test build #69176 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69176/consoleFull)** for PR 15358 at commit

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16014 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16014 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69175/ Test PASSed. ---

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16014 **[Test build #69175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69175/consoleFull)** for PR 16014 at commit

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Thanks - LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16007 **[Test build #69177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69177/consoleFull)** for PR 16007 at commit

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 I see, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Yes! That's what I meant -- change it false and add some documentation and one require to force that contract. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16014 @shivaram --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15916 Can you show an example of a leak that would happen in Executor but not in the callback? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 @rxin BTW, I see you merged #15989 to downgrade error message level in TaskMemoryManager. I'd like to modify the error message in Executor too, because the current one is little confusing

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 @rxin Sorry but finally we can change the nullable value to `false`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE and DESC ...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15358 **[Test build #69176 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69176/consoleFull)** for PR 15358 at commit

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15975 @gatorsmile did you update this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15736: [SPARK-18224] [CORE] Optimise PartitionedPairBuffer impl...

2016-11-25 Thread a-roberts
Github user a-roberts commented on the issue: https://github.com/apache/spark/pull/15736 I've conducted a lot of performance tests and gathered .hcd files so I can investigate this next week, but it looks like either the first commit is the best for performance or my current

[GitHub] spark pull request #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss ...

2016-11-25 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15994#discussion_r89662083 --- Diff: project/MimaExcludes.scala --- @@ -529,6 +529,7 @@ object MimaExcludes {

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Alright I looked more into this -- I think your approach might be better actually. Can you add an require in `InputFileNameHolder.setInputFileName` to verify the input is not null, and then

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 @rxin And also we should modify the generated code to check the value is null or not, shouldn't we? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #16008: [SPARK-18585][SQL] Use `ev.isNull = "false"` if p...

2016-11-25 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/16008#discussion_r89661757 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -61,7 +61,6 @@ case class

[GitHub] spark pull request #15916: [SPARK-18487][SQL] Add completion listener to Has...

2016-11-25 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/15916 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15916: [SPARK-18487][SQL] Add completion listener to HashAggreg...

2016-11-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15916 @rxin Thanks. Appreciate your feedback. I could close this now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 I see, I'll revert this and add the comment. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #16009: [SPARK-18318][ML] ML, Graph 2.1 QA: API: New Scal...

2016-11-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16009#discussion_r89661577 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -49,15 +49,13 @@ private[feature] trait ChiSqSelectorParams

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 I wouldn't change the default as it might break compatibility. That said, I don't think it is safe to just set this to non-nullable because it is a very implicit assumption, and setting it to be

[GitHub] spark pull request #16014: [SPARK-18590][SPARKR] build R source package when...

2016-11-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16014#discussion_r89661428 --- Diff: dev/create-release/release-build.sh --- @@ -189,6 +189,9 @@ if [[ "$1" == "package" ]]; then SHA512 $PYTHON_DIST_NAME > \

[GitHub] spark pull request #16014: [SPARK-18590][SPARKR] build R source package when...

2016-11-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16014#discussion_r89661414 --- Diff: R/pkg/NAMESPACE --- @@ -3,7 +3,7 @@ importFrom("methods", "setGeneric", "setMethod", "setOldClass") importFrom("methods", "is",

[GitHub] spark pull request #16014: [SPARK-18590][SPARKR] build R source package when...

2016-11-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16014#discussion_r89661364 --- Diff: R/pkg/DESCRIPTION --- @@ -1,28 +1,27 @@ Package: SparkR Type: Package -Title: R Frontend for Apache Spark Version: 2.1.0

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16014 **[Test build #69175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69175/consoleFull)** for PR 16014 at commit

[GitHub] spark pull request #16014: [SPARK-18590][SPARKR] build R source package when...

2016-11-25 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/16014 [SPARK-18590][SPARKR] build R source package when making distribution ## What changes were proposed in this pull request? We should include in Spark distribution the built source

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15998 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69174/ Test PASSed. ---

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15998 **[Test build #69174 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69174/consoleFull)** for PR 15998 at commit

[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.

2016-11-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 @rxin The default value is `""` (`UTF8String.fromString("")`) if input file name is not set for now.

[GitHub] spark issue #16012: [SPARK-17251][SQL] Support `OuterReference` in projectio...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69173/ Test PASSed. ---

[GitHub] spark issue #16012: [SPARK-17251][SQL] Support `OuterReference` in projectio...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16012 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16012: [SPARK-17251][SQL] Support `OuterReference` in projectio...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16012 **[Test build #69173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69173/consoleFull)** for PR 16012 at commit

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89657974 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -989,7 +989,7 @@ class Analyzer(

[GitHub] spark pull request #15998: [SPARK-18572][SQL] Add a method `listPartitionNam...

2016-11-25 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15998#discussion_r89656487 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/InMemoryCatalog.scala --- @@ -482,6 +482,19 @@ class InMemoryCatalog( }

[GitHub] spark pull request #15998: [SPARK-18572][SQL] Add a method `listPartitionNam...

2016-11-25 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15998#discussion_r89656749 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -922,6 +923,29 @@ private[spark] class HiveExternalCatalog(conf:

[GitHub] spark pull request #15998: [SPARK-18572][SQL] Add a method `listPartitionNam...

2016-11-25 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15998#discussion_r89656509 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -189,6 +189,21 @@ abstract class ExternalCatalog {

[GitHub] spark pull request #15998: [SPARK-18572][SQL] Add a method `listPartitionNam...

2016-11-25 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15998#discussion_r89656787 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -922,6 +923,29 @@ private[spark] class HiveExternalCatalog(conf:

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89656824 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -989,7 +989,7 @@ class Analyzer(

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89656679 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -356,10 +356,17 @@ case class

[GitHub] spark pull request #15977: [SPARK-18436][SQL] isin causing SQL syntax error ...

2016-11-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15977 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15977: [SPARK-18436][SQL] isin causing SQL syntax error with JD...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15977 LGTM. Merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89656204 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -989,7 +989,7 @@ class Analyzer(

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89655153 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -356,10 +356,17 @@ case class

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89655174 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -356,10 +356,17 @@ case class

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15998 **[Test build #69174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69174/consoleFull)** for PR 15998 at commit

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-11-25 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15998 CC @ericl @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15736: [SPARK-18224] [CORE] Optimise PartitionedPairBuffer impl...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15736 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69172/ Test FAILed. ---

[GitHub] spark issue #15736: [SPARK-18224] [CORE] Optimise PartitionedPairBuffer impl...

2016-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15736 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15736: [SPARK-18224] [CORE] Optimise PartitionedPairBuffer impl...

2016-11-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15736 **[Test build #69172 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69172/consoleFull)** for PR 15736 at commit

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89652990 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292 @@ +/* + *

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89647985 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292 @@ +/* + *

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89646058 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292 @@ +/* + *

  1   2   3   >