[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72318533 [Test build #26463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26463/consoleFull) for PR 4193 at commit

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72318539 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72319231 [Test build #26464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26464/consoleFull) for PR 3976 at commit

[GitHub] spark pull request: [Minor][SQL] Little refactor DataFrame related...

2015-01-31 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/4298#issuecomment-72319354 I think we can relax the constraint of `toDataFrame`. So we can directly create a DataFrame with partial columns of a RDD of tuples. No need to call an extra `select`.

[GitHub] spark pull request: [Minor][SQL] Little refactor DataFrame related...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4298#issuecomment-72319380 [Test build #26465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26465/consoleFull) for PR 4298 at commit

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-31 Thread dibbhatt
Github user dibbhatt commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23889636 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-3778] newAPIHadoopRDD doesn't properly ...

2015-01-31 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/4292#discussion_r2348 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -820,7 +822,10 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-31 Thread dibbhatt
Github user dibbhatt commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23889938 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-31 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/4155#discussion_r23888903 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -808,6 +810,7 @@ class DAGScheduler( // will be posted, which

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72321243 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72321234 [Test build #26464 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26464/consoleFull) for PR 3976 at commit

[GitHub] spark pull request: [Minor][SQL] Little refactor DataFrame related...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4298#issuecomment-72321964 [Test build #26465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26465/consoleFull) for PR 4298 at commit

[GitHub] spark pull request: [Minor][SQL] Little refactor DataFrame related...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4298#issuecomment-72321965 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-31 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23889820 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72323941 @JoshRosen can you help me? now i add a unit test for python applicaiton on yarn cluster mode. but now there is a failed. i think its reason is environment of

[GitHub] spark pull request: [Minor][SQL] Little refactor DataFrame related...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4298#issuecomment-72317547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Minor][SQL] Little refactor DataFrame related...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4298#issuecomment-72317545 [Test build #26461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26461/consoleFull) for PR 4298 at commit

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72317452 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72317446 [Test build #26462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26462/consoleFull) for PR 3976 at commit

[GitHub] spark pull request: [MLLIB][SPARK-3278] Monotone (Isotonic) regres...

2015-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3519 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3975] Added support for BlockMatrix add...

2015-01-31 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4274#issuecomment-72309957 LGTM. Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3975] Added support for BlockMatrix add...

2015-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4274 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4193#discussion_r23887726 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala --- @@ -152,7 +152,7 @@ class RowMatrix( * storing the right

[GitHub] spark pull request: SPARK-5425: Use synchronised methods in system...

2015-01-31 Thread jacek-lewandowski
Github user jacek-lewandowski commented on the pull request: https://github.com/apache/spark/pull/4220#issuecomment-72309821 Agreed, I'll make the changes and add the clarifying comments. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/4294#discussion_r23887546 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JSONRelation.scala --- @@ -28,10 +32,10 @@ private[sql] class DefaultSource extends

[GitHub] spark pull request: [MLLIB] SPARK-5491 (ex SPARK-1473): Chi-square...

2015-01-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1484#discussion_r23887545 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/ChiSqSelector.scala --- @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5307] Add a config option for Serializa...

2015-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4297 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5307] Add a config option for Serializa...

2015-01-31 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4297#issuecomment-72309051 cc @pwendell we can add this to the release notes. If the debugger malfunctions, it can be turned off with this flag. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [MLLIB][SPARK-3278] Monotone (Isotonic) regres...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3519#issuecomment-72309076 [Test build #26458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26458/consoleFull) for PR 3519 at commit

[GitHub] spark pull request: [MLLIB][SPARK-3278] Monotone (Isotonic) regres...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3519#issuecomment-72309078 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB][SPARK-3278] Monotone (Isotonic) regres...

2015-01-31 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3519#issuecomment-72309894 LGTM. Merged into master. Thanks!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4233#discussion_r23887814 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala --- @@ -68,6 +79,65 @@ class LogisticRegressionModel (

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4233#discussion_r23887815 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala --- @@ -68,6 +79,65 @@ class LogisticRegressionModel (

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4233#discussion_r23887813 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala --- @@ -68,6 +79,65 @@ class LogisticRegressionModel (

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4233#discussion_r23887816 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala --- @@ -68,6 +79,65 @@ class LogisticRegressionModel (

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-31 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23890348 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-31 Thread dibbhatt
Github user dibbhatt commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23890423 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4193 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4964] [Streaming] Exactly-once semantic...

2015-01-31 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/3798#discussion_r23890594 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72324906 [Test build #26466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26466/consoleFull) for PR 3976 at commit

[GitHub] spark pull request: [SPARK-5341] Use maven coordinates as dependen...

2015-01-31 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-72326441 Interesting... The tests are successful on my local computer but fails in Jenkins... The end to end test that downloads spark-avro and spark-csv succeeds which is nice.

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72327242 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72328501 @srowen The changes look good to me. Are you going to push more changes? I'm thinking of merging this in to avoid merge conflicts with other PRs. --- If your project is

[GitHub] spark pull request: [SPARK-4983]exception handling about adding ta...

2015-01-31 Thread GenTang
Github user GenTang commented on a diff in the pull request: https://github.com/apache/spark/pull/3986#discussion_r23890734 --- Diff: ec2/spark_ec2.py --- @@ -569,6 +569,8 @@ def launch_cluster(conn, opts, cluster_name): master_nodes = master_res.instances

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72328806 @mengxr That's all I've got for now. It fixes javadoc 8 errors due to the code, but not due to the output of unidoc. --- If your project is set up for it, you can reply

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72330127 Merged into master. Thanks! I left the JIRA as open. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-4983]exception handling about adding ta...

2015-01-31 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/3986#discussion_r23891135 --- Diff: ec2/spark_ec2.py --- @@ -569,6 +569,8 @@ def launch_cluster(conn, opts, cluster_name): master_nodes = master_res.instances

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-01-31 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/4231#issuecomment-72331831 Thanks :) Looking forward to your reviews. I will work on other stuff till then. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib]...

2015-01-31 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3637#discussion_r23891545 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/DeveloperApiExample.scala --- @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib]...

2015-01-31 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3637#discussion_r23891546 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/Regressor.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Disabling Utils.chmod700 for Windows

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4299#issuecomment-72333480 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib]...

2015-01-31 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3637#discussion_r23891575 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -80,69 +50,157 @@ class LogisticRegression extends

[GitHub] spark pull request: Disabling Utils.chmod700 for Windows

2015-01-31 Thread MartinWeindel
Github user MartinWeindel commented on the pull request: https://github.com/apache/spark/pull/4299#issuecomment-72333732 Yes, it's a regression. It worked with 1.2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: Disabling Utils.chmod700 for Windows

2015-01-31 Thread MartinWeindel
GitHub user MartinWeindel opened a pull request: https://github.com/apache/spark/pull/4299 Disabling Utils.chmod700 for Windows This patch makes Spark 1.2.1rc2 work again on Windows. Without it you get following log output on creating a Spark context: INFO

[GitHub] spark pull request: [SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib]...

2015-01-31 Thread tomerk
Github user tomerk commented on a diff in the pull request: https://github.com/apache/spark/pull/3637#discussion_r23891560 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/Regressor.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib]...

2015-01-31 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3637#issuecomment-72333635 @shivaram Thanks for the comments! As far as adding a strongly typed protected API later on, we might be able to, but I don't see it as a high priority, especially

[GitHub] spark pull request: Disabling Utils.chmod700 for Windows

2015-01-31 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4299#issuecomment-72333666 (This needs a JIRA) Did this work in 1.2.0? the question will be whether it's a regression or not. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib]...

2015-01-31 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/3637#discussion_r23891605 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/Regressor.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib]...

2015-01-31 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/3637#issuecomment-72335560 I guess the more usable DataFrame does make it better. Agree that we can revisit this after we write a few more estimators and see how they look. --- If your project

[GitHub] spark pull request: Disabling Utils.chmod700 for Windows

2015-01-31 Thread zhzhan
Github user zhzhan commented on the pull request: https://github.com/apache/spark/pull/4299#issuecomment-72335760 I didn't try 1.2.1, but in 1.2.0, I didn't met such problems in windows. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72335894 [Test build #26467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26467/consoleFull) for PR 4294 at commit

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72337568 [Test build #26467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26467/consoleFull) for PR 4294 at commit

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72337572 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72337823 [Test build #26468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26468/consoleFull) for PR 4294 at commit

[GitHub] spark pull request: Disabling Utils.chmod700 for Windows

2015-01-31 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/4299#discussion_r23892404 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -255,12 +255,17 @@ private[spark] object Utils extends Logging { * @return

[GitHub] spark pull request: [SPARK-5422] Add support for sending Graphite ...

2015-01-31 Thread hammer
Github user hammer commented on the pull request: https://github.com/apache/spark/pull/4218#issuecomment-72352426 @pwendell @rxin we'd love to get this patch into 1.3 to improve the scalability of metrics collection. it's a pretty straightforward patch. Could you guys give it some

[GitHub] spark pull request: [Minor][SQL] Little refactor DataFrame related...

2015-01-31 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/4298#discussion_r23895621 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala --- @@ -139,12 +139,13 @@ class DataFrame protected[sql]( * val rdd: RDD[(Int,

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/4294#discussion_r23892984 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -145,6 +145,11 @@ abstract class PrunedFilteredScan extends

[GitHub] spark pull request: [SPARK-5422] Add support for sending Graphite ...

2015-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4218 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5413] bump metrics dependency to v3.1.0

2015-01-31 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4209#issuecomment-72355239 Actually I merged https://github.com/apache/spark/pull/4218 which contains this patch. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-5422] Add support for sending Graphite ...

2015-01-31 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4218#issuecomment-72355227 Merging in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72340638 [Test build #26468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26468/consoleFull) for PR 4294 at commit

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72346268 [Test build #26469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26469/consoleFull) for PR 4294 at commit

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72349756 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72349753 [Test build #26470 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26470/consoleFull) for PR 3976 at commit

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72340640 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3778] newAPIHadoopRDD doesn't properly ...

2015-01-31 Thread harishreedharan
Github user harishreedharan commented on a diff in the pull request: https://github.com/apache/spark/pull/4292#discussion_r23893987 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -820,7 +822,10 @@ class SparkContext(config: SparkConf) extends Logging with

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72348214 [Test build #26469 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26469/consoleFull) for PR 4294 at commit

[GitHub] spark pull request: [WIP][SPARK-5501][SQL] Write support for the d...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4294#issuecomment-72348215 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4382] Add locations parameter to Twitte...

2015-01-31 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/3246#issuecomment-72351744 @srowen Does @tdas have time to look at this? If no, maybe others can? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-5413] bump metrics dependency to v3.1.0

2015-01-31 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4209#issuecomment-72355173 @andrewor14 are you going to merge this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72348373 [Test build #26470 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26470/consoleFull) for PR 3976 at commit

[GitHub] spark pull request: [SPARK-1405] [mllib] Latent Dirichlet Allocati...

2015-01-31 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4047#issuecomment-72348800 *Update on tests* Summary: * On a small dataset (20 newsgroups), it seems to work fine (on my laptop). * On a big dataset (Wikipedia dump with close to

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread lianhuiwang
Github user lianhuiwang closed the pull request at: https://github.com/apache/spark/pull/3976 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread lianhuiwang
GitHub user lianhuiwang reopened a pull request: https://github.com/apache/spark/pull/3976 [SPARK-5173]support python application running on yarn cluster mode now when we run python application on yarn cluster mode through spark-submit, spark-submit does not support python

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-31 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4233#discussion_r23888056 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala --- @@ -68,6 +79,65 @@ class LogisticRegressionModel (

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4233#issuecomment-72313229 [Test build #26459 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26459/consoleFull) for PR 4233 at commit

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4233#issuecomment-72313232 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4233#issuecomment-72311863 [Test build #26459 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26459/consoleFull) for PR 4233 at commit

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72313729 [Test build #26460 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26460/consoleFull) for PR 4193 at commit

[GitHub] spark pull request: [Minor][SQL] Refactor DataFrame related codes

2015-01-31 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/4298 [Minor][SQL] Refactor DataFrame related codes Simplify some codes related to DataFrame. * Calling `toAttributes` instead of a `map`. * Original `createDataFrame` creates the

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72315407 [Test build #26460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26460/consoleFull) for PR 4193 at commit

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72315410 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Minor][SQL] Little refactor DataFrame related...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4298#issuecomment-72315508 [Test build #26461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26461/consoleFull) for PR 4298 at commit

[GitHub] spark pull request: [SPARK-5212][SQL] Add support of schema-less, ...

2015-01-31 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/4014#issuecomment-72315535 @rxin I have added the explanation for this feature. Would you have time to review this pr and see if it is ok to merge? Thanks! --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3976#issuecomment-72315747 [Test build #26462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26462/consoleFull) for PR 3976 at commit

[GitHub] spark pull request: [SPARK-5173]support python application running...

2015-01-31 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/3976#discussion_r23888565 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -430,6 +430,10 @@ private[spark] class ApplicationMaster(args:

[GitHub] spark pull request: SPARK-3359 [CORE] [DOCS] `sbt/sbt unidoc` does...

2015-01-31 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4193#issuecomment-72316297 [Test build #26463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26463/consoleFull) for PR 4193 at commit