[GitHub] spark pull request #15001: [SPARK-17438][WebUI] Show Application.executorLim...

2016-09-19 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/15001#discussion_r79449912 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/ui/ApplicationPage.scala --- @@ -70,6 +70,16 @@ private[ui] class ApplicationPage(parent:

[GitHub] spark issue #15144: [SPARK-17587][PYTHON][MLLIB] SparseVector __getitem__ sh...

2016-09-19 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15144 It looks like `SparseMatrix` also raises a `ValueError`. Can you fix that here also? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15144: [SPARK-17587][PYTHON][MLLIB] SparseVector __getitem__ sh...

2016-09-19 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15144 LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create externa...

2016-09-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15145 Thanks! Let me close it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create ...

2016-09-19 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/15145 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create externa...

2016-09-19 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15145 Merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15122 **[Test build #65605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65605/consoleFull)** for PR 15122 at commit

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15122 **[Test build #65604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65604/consoleFull)** for PR 15122 at commit

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15150 **[Test build #65603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65603/consoleFull)** for PR 15150 at commit

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15122 Added test using Option 2 in the end. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread willb
Github user willb commented on the issue: https://github.com/apache/spark/pull/15150 Thanks for the feedback, @srowen! I've made the changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-09-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14959 It means that if you do this: val conf = new SparkConf() val sc = new SparkContext(conf) The internal SparkConf of the context will not be the same instance as `conf`.

[GitHub] spark issue #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create externa...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15145 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create externa...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15145 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65602/ Test PASSed. ---

[GitHub] spark issue #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create externa...

2016-09-19 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15145 Thanks @gatorsmile. I am merging this fix to branch 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create externa...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15145 **[Test build #65602 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65602/consoleFull)** for PR 15145 at commit

[GitHub] spark pull request #15090: [SPARK-17073] [SQL] generate column-level statist...

2016-09-19 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/15090#discussion_r79441079 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/StatisticsSuite.scala --- @@ -101,4 +101,47 @@ class StatisticsSuite extends QueryTest with

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15122 ok, got it. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create externa...

2016-09-19 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15145 LGTM. Pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15114: [SPARK-17473][SQL] fixing docker integration test...

2016-09-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15114 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/1 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65600/ Test FAILed. ---

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/1 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #65600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65600/consoleFull)** for PR 1 at commit

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15122 @petermaxlee Thank you for the suggestions for testing. I will try out Option 1, since 2 is a bit much work for a minor PR as this. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #15122: [SPARK-17569] Make StructuredStreaming FileStreamSource ...

2016-09-19 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15122 @yhuai The suggestions are for purely testing purposes, to make sure that StructuredStreaming doesn't check for file existence twice. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15150 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14659#discussion_r79421792 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging { } }

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15150 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65601/ Test PASSed. ---

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15150 **[Test build #65601 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65601/consoleFull)** for PR 15150 at commit

[GitHub] spark issue #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create externa...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15145 **[Test build #65602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65602/consoleFull)** for PR 15145 at commit

[GitHub] spark pull request #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14659#discussion_r79419642 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala --- @@ -54,7 +54,10 @@ private[spark] abstract class Task[T]( val partitionId:

[GitHub] spark pull request #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when th...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15041#discussion_r79419305 --- Diff: core/src/main/scala/org/apache/spark/util/collection/Utils.scala --- @@ -30,10 +34,22 @@ private[spark] object Utils { * Returns the first

[GitHub] spark pull request #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14659#discussion_r79419042 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -42,7 +42,10 @@ import org.apache.spark.rdd.RDD *

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65599/ Test PASSed. ---

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #65599 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65599/consoleFull)** for PR 12819 at commit

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r79417214 --- Diff: core/src/main/scala/org/apache/spark/util/Executable.scala --- @@ -0,0 +1,25 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r79417055 --- Diff: mesos/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherArguments.scala --- @@ -18,23 +18,43 @@ package

[GitHub] spark pull request #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when th...

2016-09-19 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15041#discussion_r79416925 --- Diff: core/src/main/scala/org/apache/spark/util/collection/Utils.scala --- @@ -30,10 +34,22 @@ private[spark] object Utils { * Returns the first

[GitHub] spark pull request #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when th...

2016-09-19 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15041#discussion_r79416074 --- Diff: core/src/main/scala/org/apache/spark/util/collection/Utils.scala --- @@ -30,10 +34,22 @@ private[spark] object Utils { * Returns the first

[GitHub] spark pull request #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14659#discussion_r79415724 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging { } }

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/12819 @yanboliang What went into the decision to use RDD based aggregation? Just curious, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels'...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15149#discussion_r79412702 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/ProbabilisticClassifierSuite.scala --- @@ -56,6 +56,21 @@ class

[GitHub] spark pull request #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when th...

2016-09-19 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15041#discussion_r79413763 --- Diff: core/src/main/scala/org/apache/spark/util/collection/Utils.scala --- @@ -30,10 +34,22 @@ private[spark] object Utils { * Returns the first

[GitHub] spark pull request #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create ...

2016-09-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15145#discussion_r79413782 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala --- @@ -509,7 +509,7 @@ class MetastoreDataSourcesSuite

[GitHub] spark issue #15115: [SPARK-17558] Bump Hadoop 2.7 version from 2.7.2 to 2.7....

2016-09-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15115 The only difference was that you're also making 2.7 the default, which isn't bad or anything. Otherwise i think it was just an oversight. There's way too much traffic to keep track of unfortunately,

[GitHub] spark issue #14827: [SPARK-17259] [build] Hadoop 2.7 profile to depend on Ha...

2016-09-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14827 Go ahead and close this one but I think you deserve 'credit' for the JIRA change, if that makes any difference. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #15150: [SPARK-17595] [MLLib] Use a bounded priority queu...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15150#discussion_r79412313 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -580,7 +581,15 @@ class Word2VecModel private[spark] ( ind +=

[GitHub] spark pull request #15150: [SPARK-17595] [MLLib] Use a bounded priority queu...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15150#discussion_r79412206 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -580,7 +581,15 @@ class Word2VecModel private[spark] ( ind +=

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14659 also just to verify , did you test this with multiple tasks running in the same executor? Just want to make sure the audit log showed each task properly --- If your project is set up for it,

[GitHub] spark pull request #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when th...

2016-09-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15041#discussion_r79410775 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1384,14 +1385,15 @@ abstract class RDD[T: ClassTag]( * @param ord the implicit

[GitHub] spark pull request #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when th...

2016-09-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15041#discussion_r79410479 --- Diff: core/src/main/scala/org/apache/spark/util/collection/Utils.scala --- @@ -30,10 +34,22 @@ private[spark] object Utils { * Returns the

[GitHub] spark pull request #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels'...

2016-09-19 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/15149#discussion_r79409985 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/ProbabilisticClassifierSuite.scala --- @@ -56,6 +56,21 @@ class

[GitHub] spark issue #15115: [SPARK-17558] Bump Hadoop 2.7 version from 2.7.2 to 2.7....

2016-09-19 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/15115 This seems a duplicate of the #14827 patch I filed 3 weeks earlier. Is there some aspect of the PR submission process that I'm missing out on? I would like to get my patches in, and

[GitHub] spark issue #15146: [SPARK-17590][SQL] Analyze CTE definitions at once and a...

2016-09-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15146 Seems good, but not quite sure, cc @hvanhovell to confirm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14659#discussion_r79408637 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging { } }

[GitHub] spark issue #15150: [SPARK-17595] [MLLib] Use a bounded priority queue to fi...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15150 **[Test build #65601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65601/consoleFull)** for PR 15150 at commit

[GitHub] spark pull request #15146: [SPARK-17590][SQL] Analyze CTE definitions at onc...

2016-09-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15146#discussion_r79408325 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -76,6 +76,23 @@ class SubquerySuite extends QueryTest with

[GitHub] spark issue #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14659 So correct me if I'm missing something but this is actually going to set the caller context for everything (HDFS and YARN) correct? The caller context is a static thing you set and its just

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15149 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15149 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65597/ Test FAILed. ---

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #65600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65600/consoleFull)** for PR 1 at commit

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15149 **[Test build #65597 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65597/consoleFull)** for PR 15149 at commit

[GitHub] spark pull request #15150: Use a bounded priority queue to find synonyms in ...

2016-09-19 Thread willb
GitHub user willb opened a pull request: https://github.com/apache/spark/pull/15150 Use a bounded priority queue to find synonyms in Word2VecModel ## What changes were proposed in this pull request? The code in `Word2VecModel.findSynonyms` to choose the vocabulary elements

[GitHub] spark pull request #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector...

2016-09-19 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/14597#discussion_r79405860 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -54,11 +55,44 @@ private[feature] trait ChiSqSelectorParams extends

[GitHub] spark issue #15071: [SPARK-17517][SQL]Improve generated Code for BroadcastHa...

2016-09-19 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/15071 @hvanhovell I have added a benchmark test for this, could you please help me to review? thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/1 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #65599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65599/consoleFull)** for PR 12819 at commit

[GitHub] spark pull request #14659: [SPARK-16757] Set up Spark caller context to HDFS

2016-09-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14659#discussion_r79403168 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2420,6 +2420,44 @@ private[spark] object Utils extends Logging { } }

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65598/ Test FAILed. ---

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #65598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65598/consoleFull)** for PR 12819 at commit

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14803 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14803 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65595/ Test PASSed. ---

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #65598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65598/consoleFull)** for PR 12819 at commit

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14803 **[Test build #65595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65595/consoleFull)** for PR 14803 at commit

[GitHub] spark issue #15147: [SPARK-17545] [SQL] Handle additional time offset format...

2016-09-19 Thread nbeyer
Github user nbeyer commented on the issue: https://github.com/apache/spark/pull/15147 @HyukjinKwon I'll have to try out the 2.0.1/master changes. Is this DateTimeUtils method no longer used anywhere then? The other scenario where I ran into trouble was using 'cast' on Column class.

[GitHub] spark pull request #14803: [SPARK-17153][SQL] Should read partition data whe...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14803#discussion_r79395037 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala --- @@ -49,6 +49,14 @@ class FileStreamSource(

[GitHub] spark issue #14650: [SPARK-17062][MESOS] add conf option to mesos dispatcher

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65594/ Test PASSed. ---

[GitHub] spark issue #14650: [SPARK-17062][MESOS] add conf option to mesos dispatcher

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14650 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14650: [SPARK-17062][MESOS] add conf option to mesos dispatcher

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14650 **[Test build #65594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65594/consoleFull)** for PR 14650 at commit

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15149 **[Test build #65597 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65597/consoleFull)** for PR 15149 at commit

[GitHub] spark issue #15051: [SPARK-17499][SparkR][ML][MLLib] make the default params...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15051 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14803 I'm not so familiar with the logic here but tend to trust your analysis and the other positive review. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #14803: [SPARK-17153][SQL] Should read partition data whe...

2016-09-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14803#discussion_r79394143 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -197,10 +197,13 @@ case class DataSource(

[GitHub] spark issue #15051: [SPARK-17499][SparkR][ML][MLLib] make the default params...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15051 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65596/ Test PASSed. ---

[GitHub] spark issue #15051: [SPARK-17499][SparkR][ML][MLLib] make the default params...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15051 **[Test build #65596 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65596/consoleFull)** for PR 15051 at commit

[GitHub] spark issue #15051: [SPARK-17499][SparkR][ML][MLLib] make the default params...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15051 **[Test build #65596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65596/consoleFull)** for PR 15051 at commit

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14803 **[Test build #65595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65595/consoleFull)** for PR 14803 at commit

[GitHub] spark issue #14731: [SPARK-17159] [streaming]: optimise check for new files ...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14731 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65592/ Test PASSed. ---

[GitHub] spark issue #14731: [SPARK-17159] [streaming]: optimise check for new files ...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14731 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14731: [SPARK-17159] [streaming]: optimise check for new files ...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14731 **[Test build #65592 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65592/consoleFull)** for PR 14731 at commit

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-19 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15102 I'm not concerned about people deleting partitions before messages have been processed, because they can take care of that problem themselves, by not deleting things until consuming has

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-09-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r79376216 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -40,12 +40,12 @@ import org.apache.ivy.plugins.matcher.GlobPatternMatcher

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-09-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r79376168 --- Diff: mesos/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherArguments.scala --- @@ -73,37 +94,55 @@ private[mesos] class

[GitHub] spark pull request #14650: [SPARK-17062][MESOS] add conf option to mesos dis...

2016-09-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14650#discussion_r79376179 --- Diff: mesos/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherArguments.scala --- @@ -18,23 +18,43 @@ package

[GitHub] spark issue #14650: [SPARK-17062][MESOS] add conf option to mesos dispatcher

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14650 **[Test build #65594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65594/consoleFull)** for PR 14650 at commit

[GitHub] spark pull request #15145: [SPARK-17589] [TEST] [2.0] Fix test case `create ...

2016-09-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15145#discussion_r79375663 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala --- @@ -509,7 +509,7 @@ class MetastoreDataSourcesSuite

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15149 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65593/ Test FAILed. ---

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15149 **[Test build #65593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65593/consoleFull)** for PR 15149 at commit

<    1   2   3   4   5   >