[GitHub] spark pull request: [SPARK-14025][STREAMING][WEBUI] Fix streaming ...

2016-03-19 Thread lw-lin
Github user lw-lin commented on the pull request: https://github.com/apache/spark/pull/11845#issuecomment-198708658 Besides, the blue/green bar in the event line itself is a clickable, linking to the specific job page. The `` thing is superfluous, let's figure out how to remove it.

[GitHub] spark pull request: [SPARK-14025][STREAMING][WEBUI] Fix streaming ...

2016-03-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11845#issuecomment-198709371 Ah I thought this was a tooltip, where HTML can't render, but it's not. I wonder, is the problem just that this is rendered as text and not HTML? if it's controlled by

[GitHub] spark pull request: [SPARK-7425] [ML] spark.ml Predictor should su...

2016-03-19 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/10355#discussion_r56465357 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala --- @@ -49,10 +50,7 @@ private[ml] trait PredictorParams extends Params

[GitHub] spark pull request: [SPARK-13977] [SQL] Brings back Shuffled hash ...

2016-03-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11788#discussion_r56548669 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/BenchmarkWholeStageCodegen.scala --- @@ -247,7 +247,26 @@ class BenchmarkWholeStageCodegen

[GitHub] spark pull request: [SPARK-11011][SQL] Narrow type of UDT serializ...

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11379 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [Minor][DOC] Fix nits in JavaStreamingTestExam...

2016-03-19 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/11821#discussion_r56642833 --- Diff: examples/src/main/java/org/apache/spark/examples/mllib/JavaStreamingTestExample.java --- @@ -94,22 +97,21 @@ public BinarySample call(String

[GitHub] spark pull request: [SPARK-13761] [ML] Remove remaining uses of va...

2016-03-19 Thread hhbyyh
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/11790#issuecomment-198032457 Looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13972] [SQL] [FOLLOW-UP] When creating ...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11825#issuecomment-198528072 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13928] Move org.apache.spark.Logging in...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11764#issuecomment-197817129 **[Test build #53403 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53403/consoleFull)** for PR 11764 at commit

[GitHub] spark pull request: [SPARK-13901][CORE]correct the logDebug inform...

2016-03-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11719#discussion_r56393833 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -555,9 +555,9 @@ private[spark] class TaskSetManager( // Jump

[GitHub] spark pull request: [SPARK-13981][SQL] Defer evaluating variables ...

2016-03-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11792#discussion_r56590134 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala --- @@ -110,23 +114,34 @@ case class Filter(condition: Expression,

[GitHub] spark pull request: [SPARK-13742][Core] Add non-iterator interface...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11578#issuecomment-198259579 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13826][SQL] Revises Dataset ScalaDoc

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11769 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-13991 - Extend the enforcer plugin Maven...

2016-03-19 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11803#issuecomment-198140635 Why should we do this? It's one less variable / environment thing for us to have to worry about if we can rely on a fixed Maven version. Alternatively, why

[GitHub] spark pull request: [SPARK-13613] [ML] Provide ignored tests to ex...

2016-03-19 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/11463#issuecomment-197552267 Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-13922][SQL] Filter rows with null attri...

2016-03-19 Thread nongli
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/11749#issuecomment-197553112 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-13579][build][wip] Stop building the ma...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11796#issuecomment-198092462 **[Test build #53471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53471/consoleFull)** for PR 11796 at commit

[GitHub] spark pull request: [SPARK-14025][STREAMING][WEBUI] Fix streaming ...

2016-03-19 Thread lw-lin
Github user lw-lin commented on the pull request: https://github.com/apache/spark/pull/11845#issuecomment-198707142 @srowen thanks for looking at this! I believe job descriptions were intended to contains only plain texts at first, but HTMLs were introduced in for streaming

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-19 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11841#issuecomment-198706323 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [MINOR][DOCS] Use `spark-submit` instead of `s...

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11842 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-14025][STREAMING][WEBUI] Fix streaming ...

2016-03-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11845#issuecomment-198702731 I agree it's a problem, but this seems like a hacky way to band-aid it, with a second version of the description and a flag passed around. Is the problem not just that

[GitHub] spark pull request: [SPARK-13973] [PySpark]: `ipython notebook` is...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11829#issuecomment-198703377 **[Test build #53616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53616/consoleFull)** for PR 11829 at commit

[GitHub] spark pull request: [MINOR] Typo fixes

2016-03-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11802#discussion_r56749101 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -246,9 +246,7 @@ class StreamingContext private[streaming] (

[GitHub] spark pull request: [SPARK-13973] [PySpark]: `ipython notebook` is...

2016-03-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11829#issuecomment-198703257 LGTM @davies any concerns about this? it would change beahvior if you don't specify an ipython driver, have both an old and new ipython installed. You'd use jupyter now

[GitHub] spark pull request: [SPARK-13973] [PySpark]: `ipython notebook` is...

2016-03-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11829#issuecomment-198703225 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Added transitive closure transformation to Cat...

2016-03-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11777#issuecomment-198701153 @antonoal you'd have to close this PR. See https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-13019][Docs] Replace example code in ml...

2016-03-19 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/11108#discussion_r56434426 --- Diff: examples/src/main/java/org/apache/spark/examples/mllib/JavaSummaryStatisticsExample.java --- @@ -0,0 +1,54 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-13993][PySpark] Add pyspark Rformula/Rf...

2016-03-19 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/11807#discussion_r56727407 --- Diff: python/pyspark/ml/feature.py --- @@ -2385,7 +2385,31 @@ class RFormula(JavaEstimator, HasFeaturesCol, HasLabelCol): |0.0|0.0|

[GitHub] spark pull request: [SPARK-13986][CORE][MLLIB] Make `@DeveloperApi...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11797#issuecomment-198142725 **[Test build #53486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53486/consoleFull)** for PR 11797 at commit

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-19 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/11242#discussion_r56432618 --- Diff: core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala --- @@ -62,7 +64,21 @@ class UnionRDD[T: ClassTag]( var rdds: Seq[RDD[T]])

[GitHub] spark pull request: [SPARK-13977] [SQL] Brings back Shuffled hash ...

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11788 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12789]Support order by index and group ...

2016-03-19 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10731#issuecomment-198018058 What you said about order by is right. The most tricky part is `*`. When we are doing select (*) in DB2, the position number is based on the table definition in

[GitHub] spark pull request: [SPARK-8884][MLlib] 1-sample Anderson-Darling ...

2016-03-19 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/11780#discussion_r56476908 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/test/AndersonDarlingTest.scala --- @@ -0,0 +1,278 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-12719][SQL] SQL generation support for ...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11768#issuecomment-197505315 **[Test build #53337 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53337/consoleFull)** for PR 11768 at commit

[GitHub] spark pull request: [SPARK-13923] [SQL] Implement SessionCatalog

2016-03-19 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11750#issuecomment-197586729 Yea, let's merge this once it passes tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13805][SQL] Generate code that get a va...

2016-03-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11636#discussion_r56691971 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala --- @@ -217,9 +222,14 @@ private[sql] case class DataSourceScan(

[GitHub] spark pull request: [SPARK-13924][SQL] officially support multi-in...

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11754 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-13930][SQL] Apply fast serialization on...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11759#issuecomment-198219667 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13816][Graphx] Add parameter checks for...

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11655 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11891] Model export/import for RFormula...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9884#issuecomment-197705684 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13950] [SQL] generate code for sort mer...

2016-03-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11771#issuecomment-197591337 Do you have breakdown on the time spent in sorting vs join? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-13629] [ML] Add binary toggle Param to ...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11536#issuecomment-197763607 **[Test build #53408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53408/consoleFull)** for PR 11536 at commit

[GitHub] spark pull request: [SPARK-13019][Docs] Replace example code in ml...

2016-03-19 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/11108#discussion_r56434127 --- Diff: examples/src/main/java/org/apache/spark/examples/mllib/JavaStratifiedSamplingExample.java --- @@ -0,0 +1,81 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-13761] [ML] Deprecate validateParams

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11620#issuecomment-197618561 **[Test build #53375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53375/consoleFull)** for PR 11620 at commit

[GitHub] spark pull request: [SPARK-11891] Model export/import for RFormula...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9884#issuecomment-197695687 **[Test build #53390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53390/consoleFull)** for PR 9884 at commit

[GitHub] spark pull request: [SPARK-13858] [SQL] fix the data type cast iss...

2016-03-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11789#issuecomment-198138980 Yea it seems like comparing floating point numbers are difficult to compare, and if users want to do those, they should use decimal in the schema. --- If your project

[GitHub] spark pull request: [SPARK-14001][SQL] support multi-children Unio...

2016-03-19 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/11818#issuecomment-198260505 cc @liancheng @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7425] [ML] spark.ml Predictor should su...

2016-03-19 Thread BenFradet
Github user BenFradet commented on the pull request: https://github.com/apache/spark/pull/10355#issuecomment-197755584 @MLnick yup thanks, I'll get to those once I'm done with this one. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [MINOR] Typo fixes

2016-03-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11802#discussion_r56638877 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -132,7 +132,7 @@ class StreamingContext private[streaming] (

[GitHub] spark pull request: [SPARK-13845][CORE]Using onBlockUpdated to rep...

2016-03-19 Thread jeanlyn
Github user jeanlyn commented on the pull request: https://github.com/apache/spark/pull/11779#issuecomment-197786170 This PR is the same as #11679 , but i came across with some accidents when rebasing the PR. So i create a new one. /cc @andrewor14 --- If your project is set up

[GitHub] spark pull request: [SPARK-13972][SQ][WIP] hive tests should fail ...

2016-03-19 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11782#issuecomment-198318615 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-13919] [SQL] [WIP] Resolving the Confli...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11745#issuecomment-197752855 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13928] Move org.apache.spark.Logging in...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11764#issuecomment-197646339 **[Test build #53385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53385/consoleFull)** for PR 11764 at commit

[GitHub] spark pull request: [SPARK-13972] [SQL] [FOLLOW-UP] When creating ...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11825#issuecomment-198528069 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2208] Fix for local metrics tests can f...

2016-03-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11747#issuecomment-198701173 OK, makes sense. Yeah, whatever makes it pass comfortably. 10ms? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-14004][SQL][MINOR] AttributeReference a...

2016-03-19 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/11820 [SPARK-14004][SQL][MINOR] AttributeReference and Alias should only use the first qualifier to generate SQL strings ## What changes were proposed in this pull request? Current

[GitHub] spark pull request: [SPARK-13905][SPARKR] Change signature of as.d...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11811#issuecomment-198191234 **[Test build #53497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53497/consoleFull)** for PR 11811 at commit

[GitHub] spark pull request: [SPARK-14025][STREAMING][WEBUI] Fix streaming ...

2016-03-19 Thread lw-lin
Github user lw-lin commented on the pull request: https://github.com/apache/spark/pull/11845#issuecomment-198695846 @andrewor14 @zsxwing would you mind taking a look at this when you have time? Thanks! --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-13845][CORE]Using onBlockUpdated to rep...

2016-03-19 Thread jeanlyn
Github user jeanlyn commented on the pull request: https://github.com/apache/spark/pull/11679#issuecomment-197674341 All test failure is relevant with `HistoryServerSuite`, the reason is we remove the `onTaskEnd`, and it's used to replay the storage page of history server from the

[GitHub] spark pull request: [SPARK-13761] [ML] Deprecate validateParams

2016-03-19 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/11620#discussion_r56545500 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -549,7 +548,9 @@ trait Params extends Identifiable with Serializable { *

[GitHub] spark pull request: [SPARK-13928] Move org.apache.spark.Logging in...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11764#issuecomment-197410911 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14007] [SQL] Cleanup hash2

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11826#issuecomment-198474472 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13889][YARN] Fix integer overflow when ...

2016-03-19 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/11713#issuecomment-197923149 Thanks @srowen . There is no integer overflow in 1.6 but the max number of executor failure is also 3 if dynamic allocation is enabled. It should use Int.MaxValue

[GitHub] spark pull request: [SPARK-13840] [SQL] Split Optimizer Rule Colum...

2016-03-19 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11682#issuecomment-197725609 @davies Sorry for that. Could you review another PR: https://github.com/apache/spark/pull/11745 ? That is built on this PR to resolve all the issues.

[GitHub] spark pull request: [SPARK-13972][SQ] hive tests should fail if SQ...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11782#issuecomment-197902489 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10788][MLLIB][ML] Remove duplicate bins...

2016-03-19 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9474#issuecomment-197982632 Note: I ran some local tests using only unordered categorical features, and as expected, it cut the communication in half. --- If your project is set up for it, you

[GitHub] spark pull request: [MINOR][SQL][BUILD] Remove duplicated lines

2016-03-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11773 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-13904][Scheduler]Add support for plugga...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11723#issuecomment-197898255 **[Test build #53424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53424/consoleFull)** for PR 11723 at commit

[GitHub] spark pull request: [SPARK-13844][SQL] Generate better code for fi...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11684#issuecomment-198112027 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13118][SQL] Expression encoding for opt...

2016-03-19 Thread jodersky
Github user jodersky commented on the pull request: https://github.com/apache/spark/pull/11708#issuecomment-197495548 it's due to the way arrays are handled in option types (as classes). I'm working on a fix --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-14025][STREAMING][WEBUI] Fix streaming ...

2016-03-19 Thread lw-lin
GitHub user lw-lin opened a pull request: https://github.com/apache/spark/pull/11845 [SPARK-14025][STREAMING][WEBUI] Fix streaming job descriptions on the event line ## What changes were proposed in this pull request? Removed the extra `...` for each streaming job's

[GitHub] spark pull request: [SPARK-14025][STREAMING][WEBUI] Fix streaming ...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11845#issuecomment-198695774 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Added transitive closure transformation to Cat...

2016-03-19 Thread antonoal
Github user antonoal commented on the pull request: https://github.com/apache/spark/pull/11777#issuecomment-198692338 It does cover all my tests and looks a lot neater, so feel free to decline this PR. Also do you know of the top of your head if there is a jira for converting an

[GitHub] spark pull request: [SPARK-13942][CORE][DOCS] Remove Shark-related...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11770#issuecomment-197593212 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Spark-13034] PySpark ml.classification suppor...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11707#issuecomment-197606612 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13826][SQL] Revises Dataset ScalaDoc

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11769#issuecomment-197802173 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13808][test-maven] Don't build assembly...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11701#issuecomment-198010170 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-13991 - Extend the enforcer plugin Maven...

2016-03-19 Thread jbonofre
GitHub user jbonofre opened a pull request: https://github.com/apache/spark/pull/11803 SPARK-13991 - Extend the enforcer plugin Maven version rule ## What changes were proposed in this pull request? Extend of the Maven version in the enforcer plugin configuration.

[GitHub] spark pull request: [SPARK-14004][SQL][MINOR] AttributeReference a...

2016-03-19 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11820#issuecomment-198433516 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11891] Model export/import for RFormula...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9884#issuecomment-197705628 **[Test build #53390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53390/consoleFull)** for PR 9884 at commit

[GitHub] spark pull request: [SPARK-13764][SQL] Parse modes in JSON data so...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11756#issuecomment-198212315 **[Test build #53504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53504/consoleFull)** for PR 11756 at commit

[GitHub] spark pull request: [SPARK-11891] Model export/import for RFormula...

2016-03-19 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9884#discussion_r56428287 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala --- @@ -247,6 +314,51 @@ private class ColumnPruner(columnsToPrune: Set[String])

[GitHub] spark pull request: [SPARK-13845][CORE]Using onBlockUpdated to rep...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11679#issuecomment-197841436 **[Test build #53411 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53411/consoleFull)** for PR 11679 at commit

[GitHub] spark pull request: [SPARK-13764][SQL] Parse modes in JSON data so...

2016-03-19 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/11756#issuecomment-197299127 Overall LGTM, thanks for working on it! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13930][SQL] Apply fast serialization on...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11759#issuecomment-197273468 **[Test build #53310 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53310/consoleFull)** for PR 11759 at commit

[GitHub] spark pull request: [SPARK-13921] Store serialized blocks as multi...

2016-03-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11748#issuecomment-197586228 Looks good from my side. Would be good for somebody else to take a look at this too. E.g. @andrewor14 ? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-13010] [ML] [SparkR] Implement a simple...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11447#issuecomment-198293656 **[Test build #53528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53528/consoleFull)** for PR 11447 at commit

[GitHub] spark pull request: [SPARK-13970][MLlib] Add Non-Negative Matrix F...

2016-03-19 Thread zhengruifeng
Github user zhengruifeng commented on the pull request: https://github.com/apache/spark/pull/11781#issuecomment-198153193 @mengxr ok. Thanks for your explanation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [WIP][SPARK-13809][SQL] State store for stream...

2016-03-19 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/11645#discussion_r56375631 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreSuite.scala --- @@ -0,0 +1,471 @@ +/* + * Licensed to

[GitHub] spark pull request: [SPARK-13808][test-maven] Don't build assembly...

2016-03-19 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11701#issuecomment-197531138 BTW, I'm just waiting for this patch before working on SPARK-13579; so it's probably ok to keep building the assembly in the maven build, because it will become a lot

[GitHub] spark pull request: [SPARK-13774][SQL] - Improve error message for...

2016-03-19 Thread skambha
Github user skambha commented on a diff in the pull request: https://github.com/apache/spark/pull/11775#discussion_r56689434 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -205,7 +205,16 @@ case class DataSource(

[GitHub] spark pull request: [SPARK-13930][SQL] Apply fast serialization on...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11759#issuecomment-197392401 **[Test build #53320 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53320/consoleFull)** for PR 11759 at commit

[GitHub] spark pull request: [SPARK-8971][MLLIB][ML] Support balanced class...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8112#issuecomment-197755583 **[Test build #53406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53406/consoleFull)** for PR 8112 at commit

[GitHub] spark pull request: [SPARK-11891] Model export/import for RFormula...

2016-03-19 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9884#discussion_r56428307 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala --- @@ -298,4 +412,51 @@ private class VectorAttributeRewriter( }

[GitHub] spark pull request: [SPARK-13989][SQL] Remove non-vectorized/unsaf...

2016-03-19 Thread nongli
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/11799#issuecomment-198137446 Patch looks good. Can you look for places where we enable unsafe row and disable vectorized (maybe just grep files that have both flags) and remove what is not

[GitHub] spark pull request: [SPARK-13826][SQL] Addendum: update documentat...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11814#issuecomment-198243299 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12469][CORE][WIP/RFC] Consistent accumu...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11105#issuecomment-198039872 **[Test build #53457 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53457/consoleFull)** for PR 11105 at commit

[GitHub] spark pull request: [SPARK-12869] Implemented an improved version ...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10839#issuecomment-197826124 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13840] [SQL] Split Optimizer Rule Colum...

2016-03-19 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11682#discussion_r56459598 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -410,6 +401,24 @@ object ColumnPruning extends

[GitHub] spark pull request: [SPARK-3308][SQL][FOLLOW-UP] Parse JSON rows h...

2016-03-19 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11752#discussion_r56434051 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/TestJsonData.scala --- @@ -209,6 +209,11 @@ private[json] trait

<    5   6   7   8   9   10   11   12   13   14   >