[GitHub] spark pull request: [SPARK-2393][SQL] Cost estimation optimization...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-49223292 QA tests have started for PR 1238. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16745/consoleFull --- If

[GitHub] spark pull request: SPARK-2465. Use long as user / item ID for ALS

2014-07-16 Thread srowen
Github user srowen closed the pull request at: https://github.com/apache/spark/pull/1393 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: Added t2 instance types

2014-07-16 Thread 24601
GitHub user 24601 opened a pull request: https://github.com/apache/spark/pull/1446 Added t2 instance types New t2 instance types require HVM amis, bailout assumption of pvm causes failures when using t2 instance types. You can merge this pull request into a Git repository by run

[GitHub] spark pull request: Added t2 instance types

2014-07-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1446#issuecomment-49225163 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: SPARK-1097: Do not introduce deadlock while fi...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1409#issuecomment-49227548 Okay I'm going to merge this into master and 1.0. We can cut a new patch release shortly for this. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread sryza
GitHub user sryza opened a pull request: https://github.com/apache/spark/pull/1447 SPARK-2519 part 2. Remove pattern matching on Tuple2 in critical section... ...s of CoGroupedRDD and PairRDDFunctions This also removes an unnecessary tuple creation in cogroup. You can merge

[GitHub] spark pull request: Added t2 instance types

2014-07-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1446#discussion_r15027927 --- Diff: ec2/spark_ec2.py --- @@ -240,7 +240,10 @@ def get_spark_ami(opts): "r3.xlarge": "hvm", "r3.2xlarge": "hvm",

[GitHub] spark pull request: SPARK-2294: fix locality inversion bug in Task...

2014-07-16 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/1313#issuecomment-49227891 en...that makes sense actually we can get the situation about locality in the taskSet easily through myLocalityLevels, which is calculated when a new execut

[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread concretevitamin
GitHub user concretevitamin opened a pull request: https://github.com/apache/spark/pull/1448 [SPARK-2531] [SQL] Make BroadcastNestedLoopJoin take into account a BuildSide JIRA ticket: https://issues.apache.org/jira/browse/SPARK-2531 You can merge this pull request into a Git reposi

[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/1448#issuecomment-49227975 Jenkins, this is okay to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request: SPARK-1097: Do not introduce deadlock while fi...

2014-07-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1409 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49227999 I have updated the screenshot. Anything else? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-2524] missing document about spark.depl...

2014-07-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1443#discussion_r15028096 --- Diff: docs/spark-standalone.md --- @@ -157,6 +157,20 @@ SPARK_MASTER_OPTS supports the following system properties: Property NameDefaultMeanin

[GitHub] spark pull request: [SPARK-2299] Consolidate various stageIdTo* ha...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1262#issuecomment-49228272 Yeah I think @rxin is going to change this so that we index on both the stage and attempt. Also, we'll need to extend the listener interface to give both the attempt and

[GitHub] spark pull request: [SPARK-2154] Schedule next Driver when one com...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1405#issuecomment-49228359 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-2154] Schedule next Driver when one com...

2014-07-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1405 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SQL] Add HiveDecimal & HiveVarchar support in...

2014-07-16 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/1436#issuecomment-49228593 Hey @chenghao-intel -- can you create a JIRA ticket for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49228680 Looks good, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15028681 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49230195 Hey actually - could this differentiate between the case where the event log was not enabled (and give a message like "Application did not enable event logging") and a c

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15030155 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request: [SPARK-2393][SQL] Cost estimation optimization...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-49233766 QA results for PR 1238:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: SPARK-2294: fix locality inversion bug in Task...

2014-07-16 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/1313#issuecomment-49234491 Thinking about this more, if I am not wrong, current scheduler can cause suboptimal schedules when there are multiple tasksetmanagers. Particularly relevant to grap

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1440#issuecomment-49238579 QA tests have started for PR 1440. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16749/consoleFull --- If

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15033041 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15033178 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/1448#issuecomment-49239568 Jenkins, test this please... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-2294: fix locality inversion bug in Task...

2014-07-16 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/1313#issuecomment-49239705 @mridulm due to the delay scheduling, this situation has been mitigated in the case that the TaskSetManager originally contains all levels and becomes RACK_LOCAL/ANY-on

[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1448#issuecomment-49239764 QA tests have started for PR 1448. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16750/consoleFull --- If

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15033708 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request: Async in progress

2014-07-16 Thread pbailis
GitHub user pbailis opened a pull request: https://github.com/apache/spark/pull/1449 Async in progress You can merge this pull request into a Git repository by running: $ git pull https://github.com/pbailis/spark async-in-progress Alternatively you can review and apply these

[GitHub] spark pull request: Async in progress

2014-07-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1449#issuecomment-49241621 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: Async in progress

2014-07-16 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/1449#issuecomment-49241702 Please create a JIRA issue and a description for this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1450 [SPARK-2534] Avoid pulling in the entire RDD in groupByKey. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark agg-closure Alternatively y

[GitHub] spark pull request: Async in progress

2014-07-16 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1449#issuecomment-49242538 Do you mind adding some description to the PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1450#issuecomment-49242499 Jenkins, why are you so slow --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: Async in progress

2014-07-16 Thread pbailis
Github user pbailis closed the pull request at: https://github.com/apache/spark/pull/1449 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: Async in progress

2014-07-16 Thread pbailis
Github user pbailis commented on the pull request: https://github.com/apache/spark/pull/1449#issuecomment-49242578 Sorry, this PR was for a private branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-2535][SQL] Add StringComparison case to...

2014-07-16 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/1451 [SPARK-2535][SQL] Add StringComparison case to NullPropagation. `StringComparison` expressions including `null` literal cases could be added to `NullPropagation`. You can merge this pull request int

[GitHub] spark pull request: [SPARK-2535][SQL] Add StringComparison case to...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1451#issuecomment-49243557 QA tests have started for PR 1451. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16752/consoleFull --- If

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object once per Tas...

2014-07-16 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1452 [SPARK-2521] Broadcast RDD object once per TaskSet (instead of sending it for every task). This should substantially reduces task size for large jobs (with large closures). It should improve task disp

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object once per Tas...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1452#discussion_r15035365 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1195,21 +1195,28 @@ abstract class RDD[T: ClassTag]( /** * Return whether

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object once per Tas...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1452#discussion_r15035409 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1195,21 +1195,28 @@ abstract class RDD[T: ClassTag]( /** * Return whether this

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1452#issuecomment-49243895 QA tests have started for PR 1452. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16753/consoleFull --- If

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1440#discussion_r15035454 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -93,6 +93,10 @@ class HiveCompatibilitySuite exte

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1440#discussion_r15035556 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -93,6 +93,10 @@ class HiveCompatibilitySuite exte

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15036312 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1440#issuecomment-49245732 QA results for PR 1440:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15036448 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark pull request: [SQL] Add HiveDecimal & HiveVarchar support in...

2014-07-16 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1436#issuecomment-49246024 Jira Ticket are created: https://issues.apache.org/jira/browse/SPARK-2540 --- If your project is set up for it, you can reply to this email and have your reply app

[GitHub] spark pull request: [SPARK-2540] [SQL] Add HiveDecimal & HiveVarch...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1436#issuecomment-49246071 QA tests have started for PR 1436. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16754/consoleFull --- If

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1452#discussion_r15036617 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1195,21 +1195,28 @@ abstract class RDD[T: ClassTag]( /** * Return whether

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15036690 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/1453 [SPARK-2433][MLLIB] fix NaiveBayesModel.predict This is the same as https://github.com/apache/spark/pull/463 , which I forgot to merge into branch-0.9. You can merge this pull request into a Git rep

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/1440#discussion_r15036800 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -93,6 +93,10 @@ class HiveCompatibilitySuite extends

[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1448#issuecomment-49246618 QA results for PR 1448:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1452#discussion_r15036841 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1195,21 +1195,28 @@ abstract class RDD[T: ClassTag]( /** * Return whether this

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15036918 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037091 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -216,17 +216,17 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037098 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -216,17 +216,17 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037121 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -413,11 +413,11 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) * par

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037123 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -430,11 +430,11 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) */

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037134 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -571,12 +571,7 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) thr

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037164 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -712,8 +701,8 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) va

[GitHub] spark pull request: SPARK-2277: make TaskScheduler track hosts on ...

2014-07-16 Thread lirui-intel
Github user lirui-intel commented on the pull request: https://github.com/apache/spark/pull/1212#issuecomment-49248835 Thanks everybody :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037689 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -712,8 +701,8 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) v

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49249163 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [SPARK-2524] missing document about spark.depl...

2014-07-16 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/1443#issuecomment-49249536 @pwendell thanks. i address your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-2524] missing document about spark.depl...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1443#issuecomment-49249565 QA tests have started for PR 1443. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16757/consoleFull --- If

[GitHub] spark pull request: [SPARK-2535][SQL] Add StringComparison case to...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1451#issuecomment-49249573 QA results for PR 1451:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49249934 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15038052 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -216,17 +216,17 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: Added t2 instance types

2014-07-16 Thread 24601
Github user 24601 commented on a diff in the pull request: https://github.com/apache/spark/pull/1446#discussion_r15038068 --- Diff: ec2/spark_ec2.py --- @@ -240,7 +240,10 @@ def get_spark_ami(opts): "r3.xlarge": "hvm", "r3.2xlarge": "hvm", "

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49250079 QA tests have started for PR 1453. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16758/consoleFull --- If

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49250081 QA tests have started for PR 1336. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16759/consoleFull --- If

[GitHub] spark pull request: Feat kryo max buffersize

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-49250226 Jenkins, retest this please. @koertkuipers mind creating a JIRA for this (we're trying to track all changes now). I can merge this in once we have an issue for it. ---

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1450#discussion_r15038170 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -361,11 +361,11 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: SPARK-2481: The environment variables SPARK_HI...

2014-07-16 Thread tsudukim
Github user tsudukim commented on the pull request: https://github.com/apache/spark/pull/1341#issuecomment-49250255 Looks good, but this patch seems to includes some unrelated diffs to SPARK-2481. * conf/spark-env.sh.template * docs/spark-standalone.md * sbin/spark-config.s

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1440#discussion_r15038187 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -93,6 +93,10 @@ class HiveCompatibilitySuite exte

[GitHub] spark pull request: Feat kryo max buffersize

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-49250323 QA tests have started for PR 735. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16760/consoleFull --- If y

[GitHub] spark pull request: Added t2 instance types

2014-07-16 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1446#issuecomment-49250359 Adding the commit to the branch made it already update this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1450#discussion_r15038311 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -361,11 +361,11 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) // g

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15038340 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -216,17 +216,17 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: Fix JIRA-983 and support exteranl sort for sor...

2014-07-16 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/931#issuecomment-49251303 @pwendell @xiajunluan I think I'm going to send a new PR based on this because I want to use some of the changes to ExternalAppendOnlyMap in sort-based shuffle. I also noti

[GitHub] spark pull request: Fix JIRA-983 and support exteranl sort for sor...

2014-07-16 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/931#discussion_r15038532 --- Diff: core/src/main/scala/org/apache/spark/rdd/OrderedRDDFunctions.scala --- @@ -56,15 +62,56 @@ class OrderedRDDFunctions[K : Ordering : ClassTag,

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49251472 QA results for PR 1453:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1450#issuecomment-49251632 Pushed a new version. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-2540] [SQL] Add HiveDecimal & HiveVarch...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1436#issuecomment-49251619 QA results for PR 1436:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: Feat kryo max buffersize

2014-07-16 Thread koertkuipers
Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-49251684 https://issues.apache.org/jira/browse/SPARK-2543 On Wed, Jul 16, 2014 at 9:53 PM, Apache Spark QA wrote: > QA tests have started for PR 735.

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1450#issuecomment-49251845 QA tests have started for PR 1450. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16762/consoleFull --- If

[GitHub] spark pull request: SPARK-2277: clear host->rack info properly

2014-07-16 Thread lirui-intel
GitHub user lirui-intel opened a pull request: https://github.com/apache/spark/pull/1454 SPARK-2277: clear host->rack info properly Hi @mridulm, I just think of this issue of [#1212](https://github.com/apache/spark/pull/1212): I added FakeRackUtil to hold the host -> rack mapping.

[GitHub] spark pull request: SPARK-2277: clear host->rack info properly

2014-07-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1454#issuecomment-49252730 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-2479][MLlib] Comparing floating-point n...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1425#issuecomment-49252992 QA tests have started for PR 1425. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16763/consoleFull --- If

[GitHub] spark pull request: [SPARK-2479][MLlib] Comparing floating-point n...

2014-07-16 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/1425#issuecomment-49253108 @mengxr and @srowen What do you think `assert((0.0001 !~== 0.0) +- 1E-5)`? We have `~==` and `~==` which will have the error message in the latest commit from my co-work

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread mengxr
Github user mengxr closed the pull request at: https://github.com/apache/spark/pull/1453 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49254135 Merged into branch-0.9. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2524] missing document about spark.depl...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1443#issuecomment-49254241 QA results for PR 1443:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: Fix JIRA-983 and support exteranl sort for sor...

2014-07-16 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/931#issuecomment-49254607 Hey, so I rebased this PR and made it mergeable in my own branch, https://github.com/mateiz/spark/tree/spark-931. However, in doing this I realized that there might be some

[GitHub] spark pull request: [SPARK-2125] Add sort flag and move sort into ...

2014-07-16 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1210#issuecomment-49254626 @jerryshao after looking more at #931, I'd actually like to hold off on merging that the way it's set up, so would you mind updating this now? I can merge this as is (with

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49254656 QA results for PR 1336:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

<    1   2   3   4   >