[GitHub] spark pull request #16154: [SPARK-17822] [R] Make JVMObjectTracker a member ...

2016-12-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/16154#discussion_r91018557 --- Diff: core/src/main/scala/org/apache/spark/api/r/JVMObjectTracker.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #16154: [SPARK-17822] [R] Make JVMObjectTracker a member ...

2016-12-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/16154#discussion_r91145714 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -247,7 +247,7 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark pull request #16154: [SPARK-17822] [R] Make JVMObjectTracker a member ...

2016-12-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/16154#discussion_r91136204 --- Diff: core/src/main/scala/org/apache/spark/api/r/RBackendHandler.scala --- @@ -143,12 +142,8 @@ private[r] class RBackendHandler(server: RBackend)

[GitHub] spark pull request #16154: [SPARK-17822] [R] Make JVMObjectTracker a member ...

2016-12-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/16154#discussion_r91140911 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -158,7 +158,7 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark issue #15923: [SPARK-4105] retry the fetch or stage if shuffle block i...

2016-12-06 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15923 ping @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #16174: [SPARK-18741][STREAMING] Reuse or clean-up SparkC...

2016-12-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16174#discussion_r91141519 --- Diff: streaming/src/test/java/org/apache/spark/streaming/LocalJavaStreamingContext.java --- @@ -28,6 +29,7 @@ @Before public

[GitHub] spark issue #16161: [SPARK-18717][SQL] Make code generation for Scala Map wo...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16161 **[Test build #69741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69741/consoleFull)** for PR 16161 at commit

[GitHub] spark pull request #16174: [SPARK-18741][STREAMING] Reuse or clean-up SparkC...

2016-12-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16174#discussion_r91144141 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2350,6 +2350,16 @@ object SparkContext extends Logging { } }

[GitHub] spark pull request #16068: [SPARK-18637][SQL]Stateful UDF should be consider...

2016-12-06 Thread zhzhan
Github user zhzhan commented on a diff in the pull request: https://github.com/apache/spark/pull/16068#discussion_r91142141 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala --- @@ -487,6 +488,52 @@ class HiveUDFSuite extends QueryTest with

[GitHub] spark issue #16175: [SPARK-17460][SQL]check if statistics.sizeInBytes >=0 in...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16175 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16175: [SPARK-17460][SQL]check if statistics.sizeInBytes >=0 in...

2016-12-06 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/16175 @gatorsmile Could you please take a look when you have time? Thanks a lot!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16174: [SPARK-18741][STREAMING] Reuse or clean-up SparkContext ...

2016-12-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16174 > This is resource intensive and it can lead to unneeded test failures (flakyness) when park.driver.allowMultipleContexts is disabled (this happens when the order of tests changes). Do you

[GitHub] spark pull request #16175: [SPARK-17460][SQL]check if statistics.sizeInBytes...

2016-12-06 Thread huaxingao
GitHub user huaxingao opened a pull request: https://github.com/apache/spark/pull/16175 [SPARK-17460][SQL]check if statistics.sizeInBytes >=0 in canBroadcast ## What changes were proposed in this pull request? 1. In SparkStrategies.canBroadcast, I will add the check

[GitHub] spark issue #16128: [SPARK-18671][SS][TEST] Added tests to ensure stability ...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16128 **[Test build #69740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69740/consoleFull)** for PR 16128 at commit

[GitHub] spark issue #16128: [SPARK-18671][SS][TEST] Added tests to ensure stability ...

2016-12-06 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16128 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #16172: [SPARK-18740] Log spark.app.name in driver logs

2016-12-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16172 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16172: [SPARK-18740] Log spark.app.name in driver logs

2016-12-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16172 LGTM. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16014: [SPARK-18590][SPARKR] build R source package when...

2016-12-06 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/16014#discussion_r91135457 --- Diff: dev/create-release/release-build.sh --- @@ -221,14 +235,13 @@ if [[ "$1" == "package" ]]; then # We increment the Zinc port each time

[GitHub] spark issue #16168: [SPARK-18209][SQL] More robust view canonicalization wit...

2016-12-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16168 also cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16168: [SPARK-18209][SQL] More robust view canonicalization wit...

2016-12-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16168 @jiangxb1987 We need a low level design about the changes you made here, especially document the issue you hit and the solution you choose. --- If your project is set up for it, you can reply

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #69739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69739/consoleFull)** for PR 16138 at commit

[GitHub] spark pull request #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with ...

2016-12-06 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r91134199 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1047,6 +1047,53 @@ case class

[GitHub] spark pull request #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with ...

2016-12-06 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r91134077 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1047,6 +1047,53 @@ case class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91133855 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -557,9 +557,12 @@ class SessionCatalog(

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91133644 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -509,32 +509,42 @@ class Analyzer( * Replaces

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91133302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -509,32 +509,42 @@ class Analyzer( * Replaces

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #69738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69738/consoleFull)** for PR 16138 at commit

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91131928 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -221,11 +223,14 @@ case class CatalogTable(

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91131668 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -557,9 +557,12 @@ class SessionCatalog(

[GitHub] spark issue #15819: [SPARK-18372][SQL][Branch-1.6].Staging directory fail to...

2016-12-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15819 Could you post the layout of that staging folder? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #69737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69737/consoleFull)** for PR 16138 at commit

[GitHub] spark pull request #16000: [SPARK-18537][Web UI]Add a REST api to spark stre...

2016-12-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16000#discussion_r91123740 --- Diff: streaming/src/main/java/org/apache/spark/streaming/status/api/v1/BatchStatus.java --- @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16174: [SPARK-18741][STREAMING] Reuse or clean-up SparkContext ...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16174 **[Test build #69736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69736/consoleFull)** for PR 16174 at commit

[GitHub] spark issue #16172: [SPARK-18740] Log spark.app.name in driver logs

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16172 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16172: [SPARK-18740] Log spark.app.name in driver logs

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16172 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69731/ Test PASSed. ---

[GitHub] spark issue #16172: [SPARK-18740] Log spark.app.name in driver logs

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16172 **[Test build #69731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69731/consoleFull)** for PR 16172 at commit

[GitHub] spark issue #16092: [SPARK-18662] Move resource managers to separate directo...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16092 **[Test build #69735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69735/consoleFull)** for PR 16092 at commit

[GitHub] spark pull request #16174: [SPARK-18741][STREAMING] Reuse or clean-up SparkC...

2016-12-06 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/16174 [SPARK-18741][STREAMING] Reuse or clean-up SparkContext in streaming tests ## What changes were proposed in this pull request? Tests in Spark Streaming currently create a `SparkContext` for

[GitHub] spark issue #16174: [SPARK-18741][STREAMING] Reuse or clean-up SparkContext ...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16174 cc @tdas @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16092: [SPARK-18662] Move resource managers to separate directo...

2016-12-06 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/16092 @vanzin Done. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16006: [SPARK-18580] [DStreams] [external/kafka-0-10] Us...

2016-12-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/16006#discussion_r91118893 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala --- @@ -143,9 +147,14 @@ private[spark]

[GitHub] spark pull request #16006: [SPARK-18580] [DStreams] [external/kafka-0-10] Us...

2016-12-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/16006#discussion_r91118458 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala --- @@ -67,6 +67,10 @@ private[spark]

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16173 **[Test build #69734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69734/consoleFull)** for PR 16173 at commit

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16173 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16173 **[Test build #69733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69733/consoleFull)** for PR 16173 at commit

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16173 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69733/ Test FAILed. ---

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16173 **[Test build #69733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69733/consoleFull)** for PR 16173 at commit

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r9663 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala --- @@ -448,19 +476,105 @@ class SQLViewSuite extends QueryTest with

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16173 **[Test build #69732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69732/consoleFull)** for PR 16173 at commit

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16173 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16173 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69732/ Test FAILed. ---

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16173 **[Test build #69732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69732/consoleFull)** for PR 16173 at commit

[GitHub] spark pull request #16173: [SPARK-18742][CORE]readd spark.broadcast.factory ...

2016-12-06 Thread windpiger
GitHub user windpiger opened a pull request: https://github.com/apache/spark/pull/16173 [SPARK-18742][CORE]readd spark.broadcast.factory conf to implement er-defined BroadcastFactory ## What changes were proposed in this pull request? After SPARK-12588 Remove HTTPBroadcast [1],

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91110663 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala --- @@ -448,19 +476,105 @@ class SQLViewSuite extends QueryTest with

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91108821 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -207,31 +205,56 @@ case class CreateViewCommand( }

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r9345 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala --- @@ -448,19 +476,105 @@ class SQLViewSuite extends QueryTest with

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91106867 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -151,6 +151,7 @@ case class CatalogTable(

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91112094 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLViewSuite.scala --- @@ -448,19 +476,105 @@ class SQLViewSuite extends QueryTest with

[GitHub] spark issue #16006: [SPARK-18580] [DStreams] [external/kafka-0-10] Use spark...

2016-12-06 Thread omuravskiy
Github user omuravskiy commented on the issue: https://github.com/apache/spark/pull/16006 I added new commit, the default value of `backpressureInitialRate` if `spark.streaming.backpressure.initialRate` is now taken from `spark.streaming.backpressure.pid.minRate`, or 100 if that is

[GitHub] spark issue #16168: [SPARK-18209][SQL] More robust view canonicalization wit...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16168 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16168: [SPARK-18209][SQL] More robust view canonicalization wit...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16168 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69730/ Test PASSed. ---

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91104224 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -221,11 +223,14 @@ case class CatalogTable(

[GitHub] spark issue #16168: [SPARK-18209][SQL] More robust view canonicalization wit...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16168 **[Test build #69730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69730/consoleFull)** for PR 16168 at commit

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91102867 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -557,9 +557,12 @@ class SessionCatalog( * If

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15722 @jiexiong PR descriptions are used in git commit messages, and should be clear and concise. The fix LGTM, but the description should be improved for future reference. How about we change it into

[GitHub] spark issue #16172: [SPARK-18740] Log spark.app.name in driver logs

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16172 **[Test build #69731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69731/consoleFull)** for PR 16172 at commit

[GitHub] spark pull request #16172: [SPARK-18740] Log spark.app.name in driver logs

2016-12-06 Thread peterableda
GitHub user peterableda opened a pull request: https://github.com/apache/spark/pull/16172 [SPARK-18740] Log spark.app.name in driver logs ## What changes were proposed in this pull request? Added simple logInfo line to print out the `spark.app.name` in the driver logs

[GitHub] spark issue #16169: [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API ...

2016-12-06 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16169 cc @felixcheung @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13909 @cloud-fan thank you for your thoughtful review comments. The latest one looks better. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69728/ Test PASSed. ---

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13909 **[Test build #69728 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69728/consoleFull)** for PR 13909 at commit

[GitHub] spark issue #15736: [SPARK-18224] [CORE] Optimise PartitionedPairBuffer impl...

2016-12-06 Thread a-roberts
Github user a-roberts commented on the issue: https://github.com/apache/spark/pull/15736 New data for us, inlined comparator scores here (code provided below to check I've not profiled something useless!): ``` ScalaSparkPagerank 2016-12-05 13:44:41 25992811548.149

[GitHub] spark pull request #16170: [SPARK-18634][SQL][TRIVIAL] Touch-up Generate

2016-12-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16170 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16170: [SPARK-18634][SQL][TRIVIAL] Touch-up Generate

2016-12-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16170 @hvanhovell Thanks for helping fixing this I missed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16170: [SPARK-18634][SQL][TRIVIAL] Touch-up Generate

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16170 Thanks for the review. Merging this one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16170: [SPARK-18634][SQL][TRIVIAL] Touch-up Generate

2016-12-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16170 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91054484 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -114,9 +117,26 @@ private[hive] class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91051441 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -207,31 +205,56 @@ case class CreateViewCommand( }

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91053965 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -509,32 +509,42 @@ class Analyzer( * Replaces

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91053897 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -509,32 +509,42 @@ class Analyzer( * Replaces

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91053671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -509,32 +509,42 @@ class Analyzer( * Replaces

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91078953 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -151,6 +151,7 @@ case class CatalogTable(

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91053117 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -151,6 +151,7 @@ case class CatalogTable(

[GitHub] spark issue #16168: [SPARK-18209][SQL] More robust view canonicalization wit...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16168 also cc @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91064615 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -126,6 +146,55 @@ private[hive] class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91051621 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -207,31 +205,56 @@ case class CreateViewCommand( }

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91051761 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -275,20 +297,26 @@ case class AlterViewAsCommand(

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91065113 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -126,6 +146,55 @@ private[hive] class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91078407 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -114,9 +117,26 @@ private[hive] class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91066027 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -126,6 +146,55 @@ private[hive] class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91052968 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -207,31 +205,56 @@ case class CreateViewCommand( }

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91051675 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -275,20 +297,26 @@ case class AlterViewAsCommand(

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91064795 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -114,9 +117,26 @@ private[hive] class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91064962 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -114,9 +117,26 @@ private[hive] class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91065717 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -114,9 +117,26 @@ private[hive] class

[GitHub] spark pull request #16168: [SPARK-18209][SQL] More robust view canonicalizat...

2016-12-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16168#discussion_r91064259 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -126,6 +146,55 @@ private[hive] class

[GitHub] spark issue #16170: [SPARK-18634][SQL][TRIVIAL] Touch-up Generate

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16170 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69726/ Test PASSed. ---

[GitHub] spark issue #16170: [SPARK-18634][SQL][TRIVIAL] Touch-up Generate

2016-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16170 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16170: [SPARK-18634][SQL][TRIVIAL] Touch-up Generate

2016-12-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16170 **[Test build #69726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69726/consoleFull)** for PR 16170 at commit

<    1   2   3   4   5   6   >