[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9593#issuecomment-166178142 **[Test build #48091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48091/consoleFull)** for PR 9593 at commit

[GitHub] spark pull request: [SPARK-12293][SQL] Support UnsafeRow in LocalT...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10283#issuecomment-166184502 **[Test build #48094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48094/consoleFull)** for PR 10283 at commit

[GitHub] spark pull request: [SPARK-12443][SQL] encoderFor should support D...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10399#discussion_r48114340 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ScalaReflectionRelationSuite.scala --- @@ -138,4 +144,16 @@ class ScalaReflectionRelationSuite

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9593#issuecomment-166185197 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48115321 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -131,6 +131,7 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12437][SQL] [WIP] Encapsulate the table...

2015-12-20 Thread naveenminchu
Github user naveenminchu commented on the pull request: https://github.com/apache/spark/pull/10403#issuecomment-166203008 @rxin Agree 100% --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48117154 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -131,6 +131,7 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166212601 **[Test build #48095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48095/consoleFull)** for PR 10398 at commit

[GitHub] spark pull request: [SPARK-6624][SQL]Convert filters into CNF for ...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/8200#issuecomment-166212614 It sounds like multiple PRs are blocked by this PR. I will submit a PR for fixing it tomorrow. Thanks! --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166212830 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12153][SPARK-7617][MLlib]add support of...

2015-12-20 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/10152#issuecomment-166215753 Pinging @jkbradley @mengxr @MechCoder again for a final review - could you give this a look and confirm you're in agreement with my comments above. Also

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166215771 Yeah, let's use this PR for the runtime check. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166169310 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166170888 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166170840 **[Test build #48089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48089/consoleFull)** for PR 10405 at commit

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-20 Thread echoTomei
GitHub user echoTomei opened a pull request: https://github.com/apache/spark/pull/10407 [SPARK-12396][Core]Once driver connect to a master successfully, stop it connect to master again. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-12443][SQL] encoderFor should support D...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10399#issuecomment-166181467 cc @cloud-fan @marmbrus @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10391#issuecomment-166181451 cc @cloud-fan @marmbrus @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12438][SQL] Add SQLUserDefinedType supp...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10390#discussion_r48113653 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/UserDefinedTypeSuite.scala --- @@ -89,6 +94,23 @@ class UserDefinedTypeSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10408#issuecomment-166184144 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166195414 I'm with @markhamstra here. It is unclear what ! or ? mean. They are unintuitive, and are not general symbols for schema construction or nullability. --- If your

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166195514 hi @markhamstra , defining a schema is very frequent when writing test, do you feel it's a good idea to put this functionality only in test scope? --- If your

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166201463 Thank you! @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/10331#discussion_r48116699 --- Diff: sql/core/src/test/resources/log4j.properties --- @@ -33,7 +33,7 @@ log4j.appender.FA.layout=org.apache.log4j.PatternLayout

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48117166 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Generate.scala --- @@ -51,9 +52,12 @@ case class Generate( join: Boolean,

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10393#issuecomment-166203846 **[Test build #48096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48096/consoleFull)** for PR 10393 at commit

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48119148 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -131,6 +131,7 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12085] [SQL] The join condition hidden ...

2015-12-20 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10087#issuecomment-166212178 @flyson Great work though, you'd better to work with #8200 . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-12153][SPARK-7617][MLlib]add support of...

2015-12-20 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/10152#discussion_r48119953 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -77,6 +77,20 @@ class Word2Vec extends Serializable with Logging {

[GitHub] spark pull request: [SPARK-12153][SPARK-7617][MLlib]add support of...

2015-12-20 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/10152#issuecomment-166215472 @ygcao just one final comment on the `setMaxSentenceLength` setter comment to address, thanks! --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread ajbozarth
GitHub user ajbozarth opened a pull request: https://github.com/apache/spark/pull/10405 [SPARK-12339] [WebUI] Added a null check that was removed in SPARK-11206 Updates made in SPARK-11206 missed an edge case which cause's a NullPointerException when a task is killed. In some cases

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166169309 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/10408 [SPARK-12327][SPARKR] fix code for lintr warning for commented code @shivaram You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10408#issuecomment-166184145 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10408#issuecomment-166184083 **[Test build #48092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48092/consoleFull)** for PR 10408 at commit

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166195733 Note that we also have the "ColumnName" implicit. Using that you can already define a struct field using: `'fieldName.int`. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166199376 **[Test build #48093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48093/consoleFull)** for PR 10346 at commit

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166199432 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166197805 @cloud-fan You're kind of hinting at my point: To me, this DSL seems to make life easier for Spark developers, not Spark users. In that kind of trade-off, we

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48118001 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -131,6 +131,7 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10409#issuecomment-166211388 **[Test build #48097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48097/consoleFull)** for PR 10409 at commit

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10393#issuecomment-166213409 **[Test build #48098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48098/consoleFull)** for PR 10393 at commit

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread ajbozarth
Github user ajbozarth commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166165475 FYI the line in JobProgressListener.onTaskEnd that initializes the null value is 387. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread ajbozarth
Github user ajbozarth commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166165392 Looping in those involved with SPARK-11206: @carsonwang @JoshRosen @vanzin --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread junhaoMg
Github user junhaoMg commented on a diff in the pull request: https://github.com/apache/spark/pull/9593#discussion_r48112803 --- Diff: docs/configuration.md --- @@ -1523,6 +1523,15 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166184298 Thanks for catching this. I think the null check here is necessary, and it seems the code that really pass a null taskMetrcis is from the `TaskSetManager` line 796

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166195988 @rxin you are right, an example is [`'b.struct('a.int,

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/10406 [SPARK-12232] New R API for read.table to avoid name conflict @shivaram sorry it took longer to fix some conflicts, this is the change to add an alias for `table` You can merge this pull

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166169270 **[Test build #48090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48090/consoleFull)** for PR 10406 at commit

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166191466 LGTM except a minor comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10398#discussion_r48115141 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -253,6 +254,18 @@ class DatasetSuite extends QueryTest with

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166201679 Should we just keep the runtime part of changes? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166213937 @yhuai Do you think we should move analysis phase checking into another PR or just drop that part? This check does find other nullability bugs (revealed by the

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/10331#discussion_r48119682 --- Diff: sql/catalyst/src/test/resources/log4j.properties --- @@ -16,9 +16,9 @@ # # Set everything to be logged to the file

[GitHub] spark pull request: [SPARK-12400][Shuffle] Avoid generating temp s...

2015-12-20 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/10376#issuecomment-166170360 Hi @JoshRosen , from performance point I don't think there's a big difference with this patch, since at most we will only open `200 * Cores` number of files

[GitHub] spark pull request: [SPARK-12168][SPARKR] Add automated tests for ...

2015-12-20 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/10171#issuecomment-166189995 @shivaram what do you think about adding `--vanilla` to `RRunner`

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166193507 **[Test build #2242 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2242/consoleFull)** for PR 10404 at commit

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166199431 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166210297 **[Test build #2242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2242/consoleFull)** for PR 10404 at commit

[GitHub] spark pull request: [SPARK-12232] New R API for read.table to avoi...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166168381 **[Test build #48090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48090/consoleFull)** for PR 10406 at commit

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10407#issuecomment-166175075 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12232][SPARKR] New R API for read.table...

2015-12-20 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/10406#issuecomment-166180732 How about "tableToDF" ? there are some API methods having table in their names, like "createExternalTable", "saveAsTable", "tables". "tableToDF" is shorter and

[GitHub] spark pull request: [SPARK-12327][SPARKR] fix code for lintr warni...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10408#issuecomment-166181204 **[Test build #48092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48092/consoleFull)** for PR 10408 at commit

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9593#issuecomment-166185198 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11627] Add initial input rate limit for...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9593#issuecomment-166185146 **[Test build #48091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48091/consoleFull)** for PR 9593 at commit

[GitHub] spark pull request: [SPARK-12443][SQL] encoderFor should support D...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10399#discussion_r48114262 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -61,6 +61,7 @@ object ScalaReflection extends

[GitHub] spark pull request: [SPARK-12371][SQL] Dataset nullability check

2015-12-20 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/10331#discussion_r48116475 --- Diff: sql/catalyst/src/test/resources/log4j.properties --- @@ -16,9 +16,9 @@ # # Set everything to be logged to the file

[GitHub] spark pull request: [SPARK-6624][SQL]Convert filters into CNF for ...

2015-12-20 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/8200#issuecomment-166213204 @gatorsmile +1 and great work :)) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166165926 **[Test build #48089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48089/consoleFull)** for PR 10405 at commit

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166168817 **[Test build #48088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48088/consoleFull)** for PR 10404 at commit

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166168831 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10404#issuecomment-166168830 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12339] [WebUI] Added a null check that ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10405#issuecomment-166170887 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166179434 @andrewor14 Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12392][Core] Optimize a location order ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10346#issuecomment-166181392 **[Test build #48093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48093/consoleFull)** for PR 10346 at commit

[GitHub] spark pull request: [SPARK-12438][SQL] Add SQLUserDefinedType supp...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10390#issuecomment-166181434 cc @cloud-fan @marmbrus @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12293][SQL] Support UnsafeRow in LocalT...

2015-12-20 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10283#issuecomment-166184035 @cloud-fan I think I have addressed all your comments. These bugs found in implementing UnsafeRow support in LocalTableScan are submitted as other PRs with their tests,

[GitHub] spark pull request: [SPARK-12399] Display correct error message wh...

2015-12-20 Thread carsonwang
Github user carsonwang commented on a diff in the pull request: https://github.com/apache/spark/pull/10352#discussion_r48114578 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala --- @@ -115,7 +117,17 @@ class HistoryServer( } def

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48114994 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Generate.scala --- @@ -51,9 +52,12 @@ case class Generate( join: Boolean,

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10391#issuecomment-166189382 Good catch! One mirror comment, can we write the test in `ExpressionEncoderSuite`? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10393#discussion_r48115249 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Generate.scala --- @@ -51,9 +52,12 @@ case class Generate( join: Boolean,

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-166202288 **[Test build #48095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48095/consoleFull)** for PR 10398 at commit

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-20 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10396#issuecomment-166203268 After a few tries, I am unable to create a test case that can trigger the issue. I think I am not the right person to fix it. Thus, For not wasting the reviewers'

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-20 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/10396 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/10409 [SPARK-12446][SQL] Add unit tests for JDBCRDD internal functions No tests done for JDBCRDD#compileFilter. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10409#issuecomment-166211512 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10409#issuecomment-166211511 **[Test build #48097 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48097/consoleFull)** for PR 10409 at commit

[GitHub] spark pull request: [SPARK-12446][SQL] Add unit tests for JDBCRDD ...

2015-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10409#issuecomment-166211513 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5479] [yarn] Handle --py-files correctl...

2015-12-20 Thread zjffdu
Github user zjffdu commented on the pull request: https://github.com/apache/spark/pull/6360#issuecomment-166224830 @vanzin I am reading the yarn related code specially on org.apache.spark.deploy.yarn.Client.scala Do you know where LOCAL_SCHEME("local") come from ? As I know we

[GitHub] spark pull request: [SPARK-12371][SQL] Runtime nullability check f...

2015-12-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166227395 @yhuai Narrowed down the scope of this PR. As we discussed offline, will open another one for the analysis phase check. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10156#discussion_r48123275 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -274,4 +274,12 @@ class AnalysisSuite extends

[GitHub] spark pull request: [SPARK-2331] SparkContext.emptyRDD should retu...

2015-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10394#issuecomment-166158720 **[Test build #2241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2241/consoleFull)** for PR 10394 at commit

[GitHub] spark pull request: [SPARK-12349] [ML] Make spark.ml PCAModel load...

2015-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/10327#discussion_r48101296 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/PCA.scala --- @@ -167,14 +167,37 @@ object PCAModel extends MLReadable[PCAModel] {

[GitHub] spark pull request: [SPARK-10158] [PySpark] [MLlib] ALS better err...

2015-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9361 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/10400 [SPARK-12444][SQL] A lightweight Scala DSL for schema construction This PR introduces a lightweight Scala DSL for constructing complex Spark SQL schema without introducing implicit conversion or

[GitHub] spark pull request: [SPARK-12292] [SQL] Support UnsafeRow in Gener...

2015-12-20 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10396#issuecomment-16611 @gatorsmile I think this needs a bit more work. Generate produces new rows, this means we also need to add a code path for generating ```UnsafeRow```s. I think we

[GitHub] spark pull request: [SPARK-12369][SQL]DataFrameReader fails on glo...

2015-12-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10379#issuecomment-166088383 @yanakad Thanks for your explanation, now I understand your use case. I agree that this is somewhat inconvenient under this use case. But I still tend to say this

[GitHub] spark pull request: [SPARK-12010][SQL] Spark JDBC requires support...

2015-12-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/10380#discussion_r48101417 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -60,20 +60,6 @@ object JdbcUtils extends Logging {

[GitHub] spark pull request: [SPARK-12444][SQL] A lightweight Scala DSL for...

2015-12-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10400#issuecomment-166112698 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12371][SQL] Checks Dataset nullability ...

2015-12-20 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10331#issuecomment-166142301 @yhuai Thanks a lot for the explanation, I misunderstood the scope of the JIRA ticket. Updated this PR according to @marmbrus's [comment][1] in #10296. A new

  1   2   3   >