[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75928/ Test PASSed. ---

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75930/ Test FAILed. ---

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if Hadoop 2...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17669 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if Hadoop 2...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17669 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75929/ Test FAILed. ---

[GitHub] spark issue #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if Hadoop 2...

2017-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17669 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if Hadoop 2...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17669 **[Test build #75931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75931/testReport)** for PR 17669 at commit [`0c9dd50`](https://github.com/apache/spark/commit/0c

[GitHub] spark pull request #17680: [SPARK-20364][SQL] Support Parquet predicate push...

2017-04-19 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/17680 [SPARK-20364][SQL] Support Parquet predicate pushdown on columns with dots ## What changes were proposed in this pull request? Currently, if there are dots in the column name, predicate

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17680 cc @ash211, @robert3005 and @liancheng. @liancheng, do you mind if I ask to review this please? --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17680 **[Test build #75932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75932/testReport)** for PR 17680 at commit [`297c70b`](https://github.com/apache/spark/commit/29

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17191 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17191 **[Test build #75933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75933/testReport)** for PR 17191 at commit [`58f7b80`](https://github.com/apache/spark/commit/58

[GitHub] spark issue #17679: [SPARK-20356][SQL] Pruned InMemoryTableScanExec should h...

2017-04-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17679 will we return invalid `outputPartitioning`? Think about a `InMemoryTableScanExec` that only read column `a`, and the `outputPartitioning` may be `a`, `b`, is that expected? --- If your project

[GitHub] spark issue #17679: [SPARK-20356][SQL] Pruned InMemoryTableScanExec should h...

2017-04-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17679 @cloud-fan I've raised similar question before in a PR. I remember I got an answer that an invalid `outputPartitioning` like this won't cause problem. --- If your project is set up for it, you can r

[GitHub] spark pull request #17681: [SPARK-20383][SQL] Supporting Create [temporary] ...

2017-04-19 Thread ouyangxiaochen
GitHub user ouyangxiaochen opened a pull request: https://github.com/apache/spark/pull/17681 [SPARK-20383][SQL] Supporting Create [temporary] Function with the keyword 'IF NOT EXISTS' ## What changes were proposed in this pull request? support to create [temporary] fucntio

[GitHub] spark pull request #17191: [SPARK-14471][SQL] Aliases in SELECT could be use...

2017-04-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17191#discussion_r112135554 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2349,7 +2350,31 @@ class Analyzer( }

[GitHub] spark issue #17679: [SPARK-20356][SQL] Pruned InMemoryTableScanExec should h...

2017-04-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17679 A similar example is `ProjectExec` which takes `child.outputPartitioning` as its `outputPartitioning`. But a projection can also change child's output like [a, b] -> [a]. --- If your project is set

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17681 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-04-19 Thread ouyangxiaochen
Github user ouyangxiaochen commented on the issue: https://github.com/apache/spark/pull/17681 @gatorsmile CC is it reasonable? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #17660: [SPARK-20359][SQL] Avoid unnecessary execution in Elimin...

2017-04-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17660 thanks, merging to master/2.2/2.1! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark pull request #17660: [SPARK-20359][SQL] Avoid unnecessary execution in...

2017-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17660 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17660: [SPARK-20359][SQL] Avoid unnecessary execution in Elimin...

2017-04-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17660 > As we don't know the details of UDF, shall we skip UDF for canFilterOutNull and directly return false? Then we may miss some opportunities for optimization. --- If your project is set up fo

[GitHub] spark issue #17679: [SPARK-20356][SQL] Pruned InMemoryTableScanExec should h...

2017-04-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17679 thanks, merging to master/2.2! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark issue #17679: [SPARK-20356][SQL] Pruned InMemoryTableScanExec should h...

2017-04-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17679 Thanks! @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #17679: [SPARK-20356][SQL] Pruned InMemoryTableScanExec s...

2017-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17679 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-04-19 Thread ouyangxiaochen
Github user ouyangxiaochen commented on the issue: https://github.com/apache/spark/pull/17681 I think it is necessary to support this key word in 'create [temporary] fucntion ...' clause, For example: I hava an application which has three steps as follow : 1. insert into ta

[GitHub] spark issue #17568: [SPARK-20254][SQL] Remove unnecessary data conversion fo...

2017-04-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17568 Thank you very much!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #17191: [SPARK-14471][SQL] Aliases in SELECT could be use...

2017-04-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17191#discussion_r112141941 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2349,7 +2350,31 @@ class Analyzer( }

[GitHub] spark pull request #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the ...

2017-04-19 Thread guoxiaolongzte
GitHub user guoxiaolongzte opened a pull request: https://github.com/apache/spark/pull/17682 [SPARK-20385][WEB-UI]'Submitted Time' field, the date format needs to be formatted, in running Drivers table or Completed Drivers table in master web ui. ## What changes were proposed in thi

[GitHub] spark issue #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the date fo...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17682 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #17582: [SPARK-20239][Core] Improve HistoryServer's ACL mechanis...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17582 **[Test build #75934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75934/testReport)** for PR 17582 at commit [`a8fb1e0`](https://github.com/apache/spark/commit/a8

[GitHub] spark issue #16989: [WIP][SPARK-19659] Fetch big blocks to disk when shuffle...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16989 **[Test build #75935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75935/testReport)** for PR 16989 at commit [`135c668`](https://github.com/apache/spark/commit/13

[GitHub] spark pull request #17665: [SPARK-16742] Mesos Kerberos Support

2017-04-19 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/17665#discussion_r112142810 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -564,12 +566,22 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark issue #17416: [SPARK-20075][CORE][WIP] Support classifier, packaging i...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17416 **[Test build #75936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75936/testReport)** for PR 17416 at commit [`4126702`](https://github.com/apache/spark/commit/41

[GitHub] spark issue #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the date fo...

2017-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17682 @guoxiaolongzte, I think Web UI changes require snapstops. (Also, it is not a big deal though, probably, it would be nicer if the branch is rebased and sync'ed to the Spark's master bran

[GitHub] spark issue #17416: [SPARK-20075][CORE][WIP] Support classifier, packaging i...

2017-04-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17416 Thanks @BryanCutler ! yes that's working better. I can run, for example, `./bin/spark-shell --packages edu.stanford.nlp:stanford-corenlp:jar:models:3.4.1` and I can see that it's now on the classpat

[GitHub] spark pull request #17665: [SPARK-16742] Mesos Kerberos Support

2017-04-19 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/17665#discussion_r112144166 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/ConfigurableCredentialManager.scala --- @@ -41,15 +41,17 @@ import org.apache.spark.util.U

[GitHub] spark issue #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the date fo...

2017-04-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17682 Agree with those comments, though the JIRA does show the change once you format the code. Are there other dates in the UI that aren't passed through this method? let's look for all `\${.*[dD

[GitHub] spark pull request #16781: [SPARK-12297][SQL] Hive compatibility for Parquet...

2017-04-19 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/16781#discussion_r112144554 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -498,6 +498,11 @@ object DateTimeUtils { false

[GitHub] spark issue #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the date fo...

2017-04-19 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/17682 Ok, i'm dealing right away.@srowen This function is very important to our large spark distributed system.Because this UI, the user concerned more, rather than our developers.@Hy

[GitHub] spark issue #17655: [SPARK-20156] [SQL] [FOLLOW-UP] Java String toLowerCase ...

2017-04-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17655 This change looks good. You're saying that you think the Avro issue is another instance of the same issue, just in Avro? if so then, yeah, not sure we can fix that unfortunately, not directly.

[GitHub] spark pull request #17655: [SPARK-20156] [SQL] [FOLLOW-UP] Java String toLow...

2017-04-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17655#discussion_r112146073 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala --- @@ -229,6 +229,32 @@ private[sql] trait SQLTestUtils }

[GitHub] spark issue #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the date fo...

2017-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17682 Ah, I didn't mean I am against this PR. I just wanted to make this PR better. Just FYI in case, just simply dragging and dropping the image into PR description in edit mode uploads the

[GitHub] spark issue #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the date fo...

2017-04-19 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/17682 Ok, I have not been how to upload pictures in PR, I will learn about it.Thank you.@HyukjinKwon --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #17191: [SPARK-14471][SQL] Aliases in SELECT could be use...

2017-04-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17191#discussion_r112152324 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -161,6 +161,8 @@ class Analyzer( HandleNullInp

[GitHub] spark issue #16609: [SPARK-8480] [CORE] [PYSPARK] [SPARKR] Add setName for D...

2017-04-19 Thread emlyn
Github user emlyn commented on the issue: https://github.com/apache/spark/pull/16609 I'm happy for this to be closed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the date fo...

2017-04-19 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/17682 @srowen I found 9 places. Are the following Scala files: MesosClusterPage.scala DriverPage.scala ApplicationPage.scala --- If your project is set up for it, you can reply

[GitHub] spark pull request #17658: [SPARK-20355] Add per application spark version o...

2017-04-19 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/17658#discussion_r112154490 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -243,18 +243,19 @@ private[history] class FsHistoryProvider(c

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17680 **[Test build #75932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75932/testReport)** for PR 17680 at commit [`297c70b`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17680 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75932/ Test PASSed. ---

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17680 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #17658: [SPARK-20355] Add per application spark version o...

2017-04-19 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/17658#discussion_r112157871 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListenerBus.scala --- @@ -71,7 +71,6 @@ private[spark] trait SparkListenerBus l

[GitHub] spark issue #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir are not ...

2017-04-19 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17638 cc @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17191 **[Test build #75933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75933/testReport)** for PR 17191 at commit [`58f7b80`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75933/ Test PASSed. ---

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17620: [SPARK-20305][Spark Core]Master may keep in the state of...

2017-04-19 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17620 I'm still not sure what issue you met during recovery and what will be happened if the issue is occurred? Looks from the fix you provided, what you mainly did is to shutdown rpcEnv, what

[GitHub] spark issue #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if Hadoop 2...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17669 **[Test build #75931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75931/testReport)** for PR 17669 at commit [`0c9dd50`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if Hadoop 2...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75931/ Test PASSed. ---

[GitHub] spark issue #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if Hadoop 2...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17669 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #17683: [SPARK-20386][Spark Core]modify the log info if t...

2017-04-19 Thread eatoncys
GitHub user eatoncys opened a pull request: https://github.com/apache/spark/pull/17683 [SPARK-20386][Spark Core]modify the log info if the block exists on the slave already ## What changes were proposed in this pull request? Modify the added memory size to memSize-originalMemSi

[GitHub] spark pull request #17683: [SPARK-20386][Spark Core]modify the log info if t...

2017-04-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17683#discussion_r112167792 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -497,11 +497,12 @@ private[spark] class BlockManagerInfo(

[GitHub] spark pull request #17683: [SPARK-20386][Spark Core]modify the log info if t...

2017-04-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17683#discussion_r112167850 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -520,9 +521,16 @@ private[spark] class BlockManagerInfo(

[GitHub] spark issue #17683: [SPARK-20386][Spark Core]modify the log info if the bloc...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17683 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #17683: [SPARK-20386][Spark Core]modify the log info if t...

2017-04-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17683#discussion_r112167751 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -520,9 +521,16 @@ private[spark] class BlockManagerInfo(

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17191 **[Test build #75937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75937/testReport)** for PR 17191 at commit [`21a5c85`](https://github.com/apache/spark/commit/21

[GitHub] spark issue #16989: [WIP][SPARK-19659] Fetch big blocks to disk when shuffle...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16989 **[Test build #75938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75938/testReport)** for PR 16989 at commit [`31cfee0`](https://github.com/apache/spark/commit/31

[GitHub] spark pull request #17191: [SPARK-14471][SQL] Aliases in SELECT could be use...

2017-04-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17191#discussion_r112168719 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2349,7 +2350,31 @@ class Analyzer( }

[GitHub] spark pull request #17191: [SPARK-14471][SQL] Aliases in SELECT could be use...

2017-04-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17191#discussion_r112168787 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2349,7 +2350,26 @@ class Analyzer( }

[GitHub] spark pull request #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17674#discussion_r112168948 --- Diff: R/pkg/R/functions.R --- @@ -3652,3 +3652,56 @@ setMethod("posexplode", jc <- callJStatic("org.apache.spark.sql.functions", "posex

[GitHub] spark pull request #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17674#discussion_r112169318 --- Diff: R/pkg/R/functions.R --- @@ -3652,3 +3652,56 @@ setMethod("posexplode", jc <- callJStatic("org.apache.spark.sql.functions", "posex

[GitHub] spark pull request #17683: [SPARK-20386][Spark Core]modify the log info if t...

2017-04-19 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/17683#discussion_r112169515 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -520,9 +521,16 @@ private[spark] class BlockManagerInfo(

[GitHub] spark pull request #17683: [SPARK-20386][Spark Core]modify the log info if t...

2017-04-19 Thread eatoncys
Github user eatoncys commented on a diff in the pull request: https://github.com/apache/spark/pull/17683#discussion_r112169689 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -520,9 +521,16 @@ private[spark] class BlockManagerInfo(

[GitHub] spark pull request #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17674#discussion_r112171802 --- Diff: R/pkg/R/functions.R --- @@ -3652,3 +3652,56 @@ setMethod("posexplode", jc <- callJStatic("org.apache.spark.sql.functions", "posex

[GitHub] spark pull request #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17674#discussion_r112172948 --- Diff: R/pkg/R/generics.R --- @@ -942,6 +942,14 @@ setGeneric("countDistinct", function(x, ...) { standardGeneric("countDistinct") #' @export s

[GitHub] spark pull request #17191: [SPARK-14471][SQL] Aliases in SELECT could be use...

2017-04-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17191#discussion_r112173197 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -161,6 +161,8 @@ class Analyzer( HandleNullInp

[GitHub] spark pull request #17672: [SPARK-20371][R] Add wrappers for collect_list an...

2017-04-19 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17672#discussion_r112173606 --- Diff: R/pkg/R/functions.R --- @@ -3652,3 +3652,43 @@ setMethod("posexplode", jc <- callJStatic("org.apache.spark.sql.functions", "posex

[GitHub] spark pull request #17672: [SPARK-20371][R] Add wrappers for collect_list an...

2017-04-19 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17672#discussion_r112173641 --- Diff: R/pkg/R/functions.R --- @@ -3652,3 +3652,43 @@ setMethod("posexplode", jc <- callJStatic("org.apache.spark.sql.functions", "posex

[GitHub] spark issue #17672: [SPARK-20371][R] Add wrappers for collect_list and colle...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17672 **[Test build #75940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75940/testReport)** for PR 17672 at commit [`fdc0395`](https://github.com/apache/spark/commit/fd

[GitHub] spark pull request #17672: [SPARK-20371][R] Add wrappers for collect_list an...

2017-04-19 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17672#discussion_r112173770 --- Diff: R/pkg/R/generics.R --- @@ -918,6 +918,14 @@ setGeneric("cbrt", function(x) { standardGeneric("cbrt") }) #' @export setGeneric("ceil", fu

[GitHub] spark issue #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17674 **[Test build #75939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75939/testReport)** for PR 17674 at commit [`d2b9723`](https://github.com/apache/spark/commit/d2

[GitHub] spark pull request #17672: [SPARK-20371][R] Add wrappers for collect_list an...

2017-04-19 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17672#discussion_r112173828 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1714,6 +1714,28 @@ test_that("group by, agg functions", { expect_true(abs(sd(1:2) - 0.7071

[GitHub] spark issue #17557: [SPARK-20208][R][DOCS] Document R fpGrowth support

2017-04-19 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/17557 Thanks @felixcheung! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16989: [WIP][SPARK-19659] Fetch big blocks to disk when shuffle...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16989 **[Test build #75935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75935/testReport)** for PR 16989 at commit [`135c668`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #17582: [SPARK-20239][Core] Improve HistoryServer's ACL mechanis...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17582 **[Test build #75934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75934/testReport)** for PR 17582 at commit [`a8fb1e0`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #16989: [WIP][SPARK-19659] Fetch big blocks to disk when shuffle...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16989 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75935/ Test PASSed. ---

[GitHub] spark issue #16989: [WIP][SPARK-19659] Fetch big blocks to disk when shuffle...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16989 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17582: [SPARK-20239][Core] Improve HistoryServer's ACL mechanis...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17582 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17582: [SPARK-20239][Core] Improve HistoryServer's ACL mechanis...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17582 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75934/ Test PASSed. ---

[GitHub] spark issue #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if Hadoop 2...

2017-04-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17669 Merged to master, let's see if it works --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark issue #17416: [SPARK-20075][CORE][WIP] Support classifier, packaging i...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17416 **[Test build #75936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75936/testReport)** for PR 17416 at commit [`4126702`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #17669: [SPARK-20343][BUILD] Avoid Unidoc build only if H...

2017-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17669 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17416: [SPARK-20075][CORE][WIP] Support classifier, packaging i...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17416 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17416: [SPARK-20075][CORE][WIP] Support classifier, packaging i...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17416 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75936/ Test PASSed. ---

[GitHub] spark issue #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17674 **[Test build #75939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75939/testReport)** for PR 17674 at commit [`d2b9723`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17674 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17674 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75939/ Test PASSed. ---

[GitHub] spark issue #17672: [SPARK-20371][R] Add wrappers for collect_list and colle...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17672 **[Test build #75940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75940/testReport)** for PR 17672 at commit [`fdc0395`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #17672: [SPARK-20371][R] Add wrappers for collect_list and colle...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17672 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75940/ Test PASSed. ---

  1   2   3   4   5   >