[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/1448#issuecomment-49227975 Jenkins, this is okay to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-1097: Do not introduce deadlock while fi...

2014-07-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1409 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49227999 I have updated the screenshot. Anything else? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2524] missing document about spark.depl...

2014-07-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1443#discussion_r15028096 --- Diff: docs/spark-standalone.md --- @@ -157,6 +157,20 @@ SPARK_MASTER_OPTS supports the following system properties: table class=table

[GitHub] spark pull request: [SPARK-2299] Consolidate various stageIdTo* ha...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1262#issuecomment-49228272 Yeah I think @rxin is going to change this so that we index on both the stage and attempt. Also, we'll need to extend the listener interface to give both the attempt

[GitHub] spark pull request: [SPARK-2154] Schedule next Driver when one com...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1405#issuecomment-49228359 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-2154] Schedule next Driver when one com...

2014-07-16 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1405 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SQL] Add HiveDecimal HiveVarchar support in...

2014-07-16 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/1436#issuecomment-49228593 Hey @chenghao-intel -- can you create a JIRA ticket for this? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49228680 Looks good, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15028681 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49230195 Hey actually - could this differentiate between the case where the event log was not enabled (and give a message like Application did not enable event logging) and a

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15030155 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2393][SQL] Cost estimation optimization...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-49233766 QA results for PR 1238:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: SPARK-2294: fix locality inversion bug in Task...

2014-07-16 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/1313#issuecomment-49234491 Thinking about this more, if I am not wrong, current scheduler can cause suboptimal schedules when there are multiple tasksetmanagers. Particularly relevant to

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1440#issuecomment-49238579 QA tests have started for PR 1440. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16749/consoleFull ---

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15033041 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15033178 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/1448#issuecomment-49239568 Jenkins, test this please... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-2294: fix locality inversion bug in Task...

2014-07-16 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/1313#issuecomment-49239705 @mridulm due to the delay scheduling, this situation has been mitigated in the case that the TaskSetManager originally contains all levels and becomes

[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1448#issuecomment-49239764 QA tests have started for PR 1448. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16750/consoleFull ---

[GitHub] spark pull request: Async in progress

2014-07-16 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/1449#issuecomment-49241702 Please create a JIRA issue and a description for this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1450 [SPARK-2534] Avoid pulling in the entire RDD in groupByKey. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark agg-closure Alternatively

[GitHub] spark pull request: Async in progress

2014-07-16 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1449#issuecomment-49242538 Do you mind adding some description to the PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1450#issuecomment-49242499 Jenkins, why are you so slow --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Async in progress

2014-07-16 Thread pbailis
Github user pbailis closed the pull request at: https://github.com/apache/spark/pull/1449 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Async in progress

2014-07-16 Thread pbailis
Github user pbailis commented on the pull request: https://github.com/apache/spark/pull/1449#issuecomment-49242578 Sorry, this PR was for a private branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-2535][SQL] Add StringComparison case to...

2014-07-16 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/1451 [SPARK-2535][SQL] Add StringComparison case to NullPropagation. `StringComparison` expressions including `null` literal cases could be added to `NullPropagation`. You can merge this pull request

[GitHub] spark pull request: [SPARK-2535][SQL] Add StringComparison case to...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1451#issuecomment-49243557 QA tests have started for PR 1451. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16752/consoleFull ---

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object once per Tas...

2014-07-16 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/1452 [SPARK-2521] Broadcast RDD object once per TaskSet (instead of sending it for every task). This should substantially reduces task size for large jobs (with large closures). It should improve task

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object once per Tas...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1452#discussion_r15035365 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1195,21 +1195,28 @@ abstract class RDD[T: ClassTag]( /** * Return whether

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object once per Tas...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1452#discussion_r15035409 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1195,21 +1195,28 @@ abstract class RDD[T: ClassTag]( /** * Return whether

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1452#issuecomment-49243895 QA tests have started for PR 1452. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16753/consoleFull ---

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1440#discussion_r15035454 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -93,6 +93,10 @@ class HiveCompatibilitySuite

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1440#discussion_r15035556 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -93,6 +93,10 @@ class HiveCompatibilitySuite

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15036312 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1440#issuecomment-49245732 QA results for PR 1440:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15036448 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SQL] Add HiveDecimal HiveVarchar support in...

2014-07-16 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1436#issuecomment-49246024 Jira Ticket are created: https://issues.apache.org/jira/browse/SPARK-2540 --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-2540] [SQL] Add HiveDecimal HiveVarch...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1436#issuecomment-49246071 QA tests have started for PR 1436. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16754/consoleFull ---

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1452#discussion_r15036617 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1195,21 +1195,28 @@ abstract class RDD[T: ClassTag]( /** * Return whether

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15036690 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/1453 [SPARK-2433][MLLIB] fix NaiveBayesModel.predict This is the same as https://github.com/apache/spark/pull/463 , which I forgot to merge into branch-0.9. You can merge this pull request into a Git

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/1440#discussion_r15036800 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -93,6 +93,10 @@ class HiveCompatibilitySuite extends

[GitHub] spark pull request: [SPARK-2531] [SQL] Make BroadcastNestedLoopJoi...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1448#issuecomment-49246618 QA results for PR 1448:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2521] Broadcast RDD object (instead of ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1452#discussion_r15036841 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1195,21 +1195,28 @@ abstract class RDD[T: ClassTag]( /** * Return whether

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread dorx
Github user dorx commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15036918 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037091 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -216,17 +216,17 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037098 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -216,17 +216,17 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037121 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -413,11 +413,11 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) *

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037134 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -571,12 +571,7 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037164 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -712,8 +701,8 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: SPARK-2277: make TaskScheduler track hosts on ...

2014-07-16 Thread lirui-intel
Github user lirui-intel commented on the pull request: https://github.com/apache/spark/pull/1212#issuecomment-49248835 Thanks everybody :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15037689 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -712,8 +701,8 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49249163 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2524] missing document about spark.depl...

2014-07-16 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/1443#issuecomment-49249536 @pwendell thanks. i address your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2535][SQL] Add StringComparison case to...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1451#issuecomment-49249573 QA results for PR 1451:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49249934 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15038052 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -216,17 +216,17 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: Added t2 instance types

2014-07-16 Thread 24601
Github user 24601 commented on a diff in the pull request: https://github.com/apache/spark/pull/1446#discussion_r15038068 --- Diff: ec2/spark_ec2.py --- @@ -240,7 +240,10 @@ def get_spark_ami(opts): r3.xlarge: hvm, r3.2xlarge: hvm,

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49250079 QA tests have started for PR 1453. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16758/consoleFull ---

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49250081 QA tests have started for PR 1336. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16759/consoleFull ---

[GitHub] spark pull request: Feat kryo max buffersize

2014-07-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-49250226 Jenkins, retest this please. @koertkuipers mind creating a JIRA for this (we're trying to track all changes now). I can merge this in once we have an issue for it.

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1450#discussion_r15038170 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -361,11 +361,11 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: SPARK-2481: The environment variables SPARK_HI...

2014-07-16 Thread tsudukim
Github user tsudukim commented on the pull request: https://github.com/apache/spark/pull/1341#issuecomment-49250255 Looks good, but this patch seems to includes some unrelated diffs to SPARK-2481. * conf/spark-env.sh.template * docs/spark-standalone.md *

[GitHub] spark pull request: [SPARK-2190][SQL] Specialized ColumnType for T...

2014-07-16 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1440#discussion_r15038187 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCompatibilitySuite.scala --- @@ -93,6 +93,10 @@ class HiveCompatibilitySuite

[GitHub] spark pull request: Feat kryo max buffersize

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-49250323 QA tests have started for PR 735. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16760/consoleFull ---

[GitHub] spark pull request: Added t2 instance types

2014-07-16 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1446#issuecomment-49250359 Adding the commit to the branch made it already update this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1450#discussion_r15038311 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -361,11 +361,11 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) //

[GitHub] spark pull request: SPARK-2519 part 2. Remove pattern matching on ...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1447#discussion_r15038340 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -216,17 +216,17 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: Fix JIRA-983 and support exteranl sort for sor...

2014-07-16 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/931#discussion_r15038532 --- Diff: core/src/main/scala/org/apache/spark/rdd/OrderedRDDFunctions.scala --- @@ -56,15 +62,56 @@ class OrderedRDDFunctions[K : Ordering : ClassTag,

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49251472 QA results for PR 1453:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1450#issuecomment-49251632 Pushed a new version. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2540] [SQL] Add HiveDecimal HiveVarch...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1436#issuecomment-49251619 QA results for PR 1436:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: Feat kryo max buffersize

2014-07-16 Thread koertkuipers
Github user koertkuipers commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-49251684 https://issues.apache.org/jira/browse/SPARK-2543 On Wed, Jul 16, 2014 at 9:53 PM, Apache Spark QA notificati...@github.com wrote: QA tests

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1450#issuecomment-49251845 QA tests have started for PR 1450. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16762/consoleFull ---

[GitHub] spark pull request: SPARK-2277: clear host-rack info properly

2014-07-16 Thread lirui-intel
GitHub user lirui-intel opened a pull request: https://github.com/apache/spark/pull/1454 SPARK-2277: clear host-rack info properly Hi @mridulm, I just think of this issue of [#1212](https://github.com/apache/spark/pull/1212): I added FakeRackUtil to hold the host - rack mapping.

[GitHub] spark pull request: SPARK-2277: clear host-rack info properly

2014-07-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1454#issuecomment-49252730 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2479][MLlib] Comparing floating-point n...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1425#issuecomment-49252992 QA tests have started for PR 1425. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16763/consoleFull ---

[GitHub] spark pull request: [SPARK-2479][MLlib] Comparing floating-point n...

2014-07-16 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/1425#issuecomment-49253108 @mengxr and @srowen What do you think `assert((0.0001 !~== 0.0) +- 1E-5)`? We have `~==` and `~==` which will have the error message in the latest commit from my

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread mengxr
Github user mengxr closed the pull request at: https://github.com/apache/spark/pull/1453 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2433][MLLIB] fix NaiveBayesModel.predic...

2014-07-16 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1453#issuecomment-49254135 Merged into branch-0.9. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2524] missing document about spark.depl...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1443#issuecomment-49254241 QA results for PR 1443:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: Fix JIRA-983 and support exteranl sort for sor...

2014-07-16 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/931#issuecomment-49254607 Hey, so I rebased this PR and made it mergeable in my own branch, https://github.com/mateiz/spark/tree/spark-931. However, in doing this I realized that there might be

[GitHub] spark pull request: [SPARK-2125] Add sort flag and move sort into ...

2014-07-16 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1210#issuecomment-49254626 @jerryshao after looking more at #931, I'd actually like to hold off on merging that the way it's set up, so would you mind updating this now? I can merge this as is

[GitHub] spark pull request: [SPARK-2411] Add a history-not-found page to s...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1336#issuecomment-49254656 QA results for PR 1336:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: Feat kryo max buffersize

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-49254912 QA results for PR 735:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-1667] Jobs never finish successfully on...

2014-07-16 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/1383#issuecomment-49255082 My PR handles IOException as fatal but I think it's not good because IOException is not always fatal. The problem I want to solve is IOException thrown when writing

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1450#issuecomment-49256041 QA results for PR 1450:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: [SPARK-1112, 2156] (0.9 edition) Use correct a...

2014-07-16 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/1455 [SPARK-1112, 2156] (0.9 edition) Use correct akka frame size and overhead amounts. backport #1172 to branch-0.9. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-1112, 2156] (0.9 edition) Use correct a...

2014-07-16 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1455#issuecomment-49256153 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1112, 2156] (0.9 edition) Use correct a...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1455#issuecomment-49256230 QA tests have started for PR 1455. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16764/consoleFull ---

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1450#discussion_r15040306 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -214,7 +214,7 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1450#discussion_r15040327 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -214,7 +214,7 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1450#discussion_r15040336 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -214,7 +214,7 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request: [SPARK-2534] Avoid pulling in the entire RDD i...

2014-07-16 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1450#issuecomment-49256830 QA tests have started for PR 1450. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16765/consoleFull ---

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15040517 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/Correlation.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15040598 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-2359][MLlib] Correlations

2014-07-16 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1367#discussion_r15040646 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/correlation/SpearmansCorrelation.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-695] In DAGScheduler's getPreferredLocs...

2014-07-16 Thread staple
Github user staple commented on a diff in the pull request: https://github.com/apache/spark/pull/1362#discussion_r15040670 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1107,7 +1106,6 @@ class DAGScheduler( case shufDep:

[GitHub] spark pull request: [SPARK-695] In DAGScheduler's getPreferredLocs...

2014-07-16 Thread staple
Github user staple commented on a diff in the pull request: https://github.com/apache/spark/pull/1362#discussion_r15040672 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1128,6 +1126,23 @@ class DAGScheduler( */ private[spark]

<    1   2   3   4   >