[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #73535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73535/testReport)** for PR 17077 at commit

[GitHub] spark issue #16557: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16557 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16557: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16557 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73525/ Test PASSed. ---

[GitHub] spark issue #17088: [SPARK-19753][CORE] All shuffle files on a host should b...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17088 **[Test build #73533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73533/testReport)** for PR 17088 at commit

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16867 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73514/ Test FAILed. ---

[GitHub] spark issue #17031: [SPARK-19702][MESOS] Add suppress/revive support to the ...

2017-02-27 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/17031 Ok I see. LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17087 **[Test build #73530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73530/testReport)** for PR 17087 at commit

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-02-27 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16867 This looks like a real test failure resulting from this change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #13143: [SPARK-15359] [Mesos] Mesos dispatcher should handle DRI...

2017-02-27 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/13143 What whole function is designed poorly. We need to totally change it instead of tacking this on. We shouldn't be calling `driver.run()` in a separate thread. We should be calling

[GitHub] spark issue #13932: [SPARK-15354] [CORE] Topology aware block replication st...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13932 **[Test build #73534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73534/testReport)** for PR 13932 at commit

[GitHub] spark issue #13932: [SPARK-15354] [CORE] Topology aware block replication st...

2017-02-27 Thread shubhamchopra
Github user shubhamchopra commented on the issue: https://github.com/apache/spark/pull/13932 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17085: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17085 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17072: [MINOR][BUILD] Fix lint-java breaks in Java

2017-02-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17072 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17072: [MINOR][BUILD] Fix lint-java breaks in Java

2017-02-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17072 Thank you @srowen. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #17082: [SPARK-19749][SS] Name socket source with a meani...

2017-02-27 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/17082 [SPARK-19749][SS] Name socket source with a meaningful name ## What changes were proposed in this pull request? Name socket source with a meaningful name ## How was this patch

[GitHub] spark issue #17082: [SPARK-19749][SS] Name socket source with a meaningful n...

2017-02-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17082 OK, how about other sources? it'd be nice to given them toString methods consistently along with other related classes, if you're bothering to do one. --- If your project is set up for it, you can

[GitHub] spark issue #16819: [SPARK-16441][YARN] Set maxNumExecutor depends on yarn c...

2017-02-27 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/16819 I agree with others, this is not the way to do this. There are different schedulers in yarn, each with different configs that could affect the actual resources you get. If you want to

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16965 Github isn't handling the merge well, so you might try rebasing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17078#discussion_r103342317 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1431,7 +1431,12 @@ private class LogisticAggregator(

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103340921 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -40,19 +42,179 @@ abstract class PlanExpression[T

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r102167233 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1110,31 +1184,24 @@ class Analyzer( }

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103339031 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -109,6 +109,26 @@ object TypeCoercion { }

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r102167746 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1110,31 +1184,24 @@ class Analyzer( }

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103340272 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -123,19 +123,36 @@ case class Not(child:

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103337724 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2512,3 +2522,67 @@ object ResolveCreateNamedStruct

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r102165726 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -707,13 +709,85 @@ class Analyzer( }

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r102256790 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2332,6 +2337,11 @@ class Analyzer(

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103340692 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/subquery.scala --- @@ -40,19 +42,179 @@ abstract class PlanExpression[T

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103336411 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1398,42 +1399,46 @@ class Analyzer( }

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r102168672 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1110,31 +1184,24 @@ class Analyzer( }

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r102168299 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -707,13 +709,85 @@ class Analyzer( }

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r102168200 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1110,31 +1184,24 @@ class Analyzer( }

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17078#discussion_r103343872 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1431,7 +1431,12 @@ private class LogisticAggregator(

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/16965 Looks like the rebase is making it even worse. I will reopen a PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16965 **[Test build #73538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73538/testReport)** for PR 16965 at commit

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17078 **[Test build #73537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73537/testReport)** for PR 17078 at commit

[GitHub] spark pull request #16965: [SPARK-18450][ML] Scala API Change for LSH AND-am...

2017-02-27 Thread Yunni
Github user Yunni closed the pull request at: https://github.com/apache/spark/pull/16965 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14273: [SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch

2017-02-27 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/14273 OK apologies @MechCoder for the delay. I guess we can close this issue, and someone else can open up a PR based on yours. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #14273: [SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch

2017-02-27 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/14273 Sorry about the delay here. Do you still have time to work on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17015: [SPARK-19678][SQL] remove MetastoreRelation

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17015 **[Test build #73536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73536/testReport)** for PR 17015 at commit

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17078#discussion_r103342591 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -456,6 +456,32 @@ class LogisticRegressionSuite

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17078#discussion_r103342093 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1447,7 +1447,7 @@ private class LogisticAggregator(

[GitHub] spark issue #17079: [SPARK-19748][SQL]refresh function has a wrong order to ...

2017-02-27 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/17079 cc @cloud-fan @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16959: [SPARK-19631][CORE] OutputCommitCoordinator should not a...

2017-02-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16959 Looks ok to me, but let me ping some others @squito @kayousterhout --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73539/testReport)** for PR 17081 at commit

[GitHub] spark issue #14273: [SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch

2017-02-27 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14273 @jkbradley I do not think @MechCoder is working on Spark for the time being. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103346762 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1110,31 +1184,24 @@ class Analyzer( }

[GitHub] spark issue #17012: [SPARK-19677][SS] Renaming a file atop an existing one s...

2017-02-27 Thread hejix
Github user hejix commented on the issue: https://github.com/apache/spark/pull/17012 Just some feedback that I did some initial regression testing with this pull request on a full YARN (v2.7.3) 4 node cluster on GCP and it appears to have fixed the two issues we had- our structured

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-02-27 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103347540 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2332,6 +2337,11 @@ class Analyzer(

[GitHub] spark issue #17075: [SPARK-19727][SQL] Fix for round function that modifies ...

2017-02-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17075 I dont' know the code well enough to really evaluate this, but, I see that `.clone()` is called in a similar context in `decimalExpressions`. There are also similar usages of `changePrecision` in

[GitHub] spark issue #17082: [SPARK-19749][SS] Name socket source with a meaningful n...

2017-02-27 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/17082 @srowen I think this is the only one souce forgotten to name. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #17071: [SPARK-15615][SQL][BUILD][FOLLOW-UP] Replace deprecated ...

2017-02-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17071 I like it, though, regarding still testing the deprecated method -- maybe it's best to even have a test that is explicitly just for testing the old method? that may be clearer than just picking some

[GitHub] spark pull request #15505: [SPARK-18890][CORE] Move task serialization from ...

2017-02-27 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r103174824 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -23,7 +23,10 @@ import java.util.Properties import

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-02-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17059 That's compelling regarding performance. It's not big but not trivial. My remaining concern is whether you're handling all the cases the original did. `Number` covers a lot but does it include

[GitHub] spark pull request #17083: [SPARK-19750][UI][branch-2.1] Fix redirect issue ...

2017-02-27 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/17083 [SPARK-19750][UI][branch-2.1] Fix redirect issue from http to https ## What changes were proposed in this pull request? If spark ui port (4040) is not set, it will choose port number 0,

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-02-27 Thread thunterdb
Github user thunterdb commented on the issue: https://github.com/apache/spark/pull/15770 Note that any of these formats would cause trouble for a graph with high centrality (lady gaga in the twitter graph). That being said, I do not have a strong opinion as to which option we pick,

[GitHub] spark issue #17039: [SPARK-19710][SQL][TESTS] Fix ordering of rows in query ...

2017-02-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17039 @hvanhovell Is that possible the SQL queries are used to verify the behavior of ORDER BY? Do you think we should explicitly leave a comment to say SQLQueryTestSuite will not be used for this

[GitHub] spark pull request #17064: [SPARK-19736][SQL] refreshByPath should clear all...

2017-02-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17064#discussion_r103348444 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -168,15 +168,16 @@ class CacheManager extends Logging {

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
Github user sueann commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103349080 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17089: [SPARK-19756][SQL] drop the table cache after ins...

2017-02-27 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/17089 [SPARK-19756][SQL] drop the table cache after inserting into a data source table ## What changes were proposed in this pull request? When we inserting into a table, we should uncache it

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
Github user sueann commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103349139 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
GitHub user sueann opened a pull request: https://github.com/apache/spark/pull/17090 [Spark-19535][ML] RecommendForAllUsers RecommendForAllItems for ALS on Dataframe ## What changes were proposed in this pull request? This is a simple implementation of

[GitHub] spark issue #17089: [SPARK-19756][SQL] drop the table cache after inserting ...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17089 cc @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17075: [SPARK-19727][SQL] Fix for round function that modifies ...

2017-02-27 Thread wojtek-szymanski
Github user wojtek-szymanski commented on the issue: https://github.com/apache/spark/pull/17075 I have just started refactoring of `changePrecission` in order to make it immutable. My idea was to change the signature from: `def changePrecision(precision: Int, scale: Int,

[GitHub] spark pull request #17064: [SPARK-19736][SQL] refreshByPath should clear all...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17064#discussion_r103349345 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -168,15 +168,16 @@ class CacheManager extends Logging {

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103349429 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #16959: [SPARK-19631][CORE] OutputCommitCoordinator shoul...

2017-02-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16959#discussion_r103348105 --- Diff: core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala --- @@ -48,25 +48,29 @@ private[spark] class

[GitHub] spark pull request #17079: [SPARK-19748][SQL]refresh function has a wrong or...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17079#discussion_r103349639 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala --- @@ -178,6 +178,34 @@ class FileIndexSuite extends

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103349590 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -248,18 +248,18 @@ class ALSModel private[ml] ( @Since("1.3.0")

[GitHub] spark issue #17090: [Spark-19535][ML] RecommendForAllUsers RecommendForAllIt...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17090 **[Test build #73540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73540/testReport)** for PR 17090 at commit

[GitHub] spark issue #17089: [SPARK-19756][SQL] drop the table cache after inserting ...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17089 **[Test build #73541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73541/testReport)** for PR 17089 at commit

[GitHub] spark pull request #17079: [SPARK-19748][SQL]refresh function has a wrong or...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17079#discussion_r103350023 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala --- @@ -178,6 +178,34 @@ class FileIndexSuite extends

[GitHub] spark issue #17079: [SPARK-19748][SQL]refresh function has a wrong order to ...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17079 good catch! Can you show a real example that fails because of this bug? I'm wondering why the existing unit tests didn't expose this bug... --- If your project is set up for it, you can reply to

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17078 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17091: [SPARK-19757][CORE] Executor with task scheduled ...

2017-02-27 Thread jxiang
GitHub user jxiang opened a pull request: https://github.com/apache/spark/pull/17091 [SPARK-19757][CORE] Executor with task scheduled could be killed due to idleness ## What changes were proposed in this pull request? In makeOffers, put in one synchronization block to check if

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
Github user sueann commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103350410 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/TopByKeyAggregator.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17078 Thanks. Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17090: [Spark-19535][ML] RecommendForAllUsers RecommendF...

2017-02-27 Thread sueann
Github user sueann commented on a diff in the pull request: https://github.com/apache/spark/pull/17090#discussion_r103350775 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -248,18 +248,18 @@ class ALSModel private[ml] ( @Since("1.3.0")

[GitHub] spark pull request #17064: [SPARK-19736][SQL] refreshByPath should clear all...

2017-02-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17064#discussion_r103351494 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -168,15 +168,16 @@ class CacheManager extends Logging {

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307491 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -146,4 +107,153 @@ private[hive] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103338672 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SessionStateSuite.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103305709 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -90,110 +208,29 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r10330 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SessionStateSuite.scala --- @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103337066 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -136,6 +139,26 @@ private[sql] class SharedState(val

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103302329 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -65,22 +82,118 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103295639 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1178,4 +1181,36 @@ class SessionCatalog(

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307776 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -146,4 +107,153 @@ private[hive] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103328320 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSessionStateSuite.scala --- @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103303272 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -90,110 +203,37 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103295692 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1178,4 +1181,36 @@ class SessionCatalog(

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103306212 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -136,6 +139,26 @@ private[sql] class SharedState(val

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307299 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -212,3 +247,31 @@ private[sql] class HiveSessionCatalog(

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103331408 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -17,89 +17,50 @@ package org.apache.spark.sql.hive

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307383 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -146,4 +107,153 @@ private[hive] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103329699 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala --- @@ -144,11 +145,37 @@ private[hive] class TestHiveSparkSession(

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103298123 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ExperimentalMethods.scala --- @@ -46,4 +46,10 @@ class ExperimentalMethods private[sql]() {

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103297916 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1196,4 +1198,28 @@ class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103336696 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -65,22 +82,118 @@ private[sql] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103307420 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -146,4 +107,153 @@ private[hive] class

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-02-27 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r103336676 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -65,22 +82,118 @@ private[sql] class

<    1   2   3   4   5   6   7   >