[GitHub] spark pull request: [SPARK-3781] code format and little improvemen...

2014-10-13 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2734#issuecomment-58849017 Hi @shijinkui, After some discussion, we've decided that we'd like to avoid merging pull requests that make large, sweeping style changes/improvements, since

[GitHub] spark pull request: [SPARK-3873] [build] Add style checker to enfo...

2014-10-13 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2757#issuecomment-58849258 Hi @vanzin, We'd like to avoid making large refactorings for style, since these changes tend to create merge-conflicts when backporting to maintenance branches

[GitHub] spark pull request: Add echo Run streaming tests ...

2014-10-13 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2778#issuecomment-58849321 LGTM; thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3854] Scala style: require spaces befor...

2014-10-13 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2761#issuecomment-58849280 Hi @sarutak, We'd like to avoid making large refactorings for style, since these changes tend to create merge-conflicts when backporting to maintenance

[GitHub] spark pull request: Add echo Run streaming tests ...

2014-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2778 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3869] ./bin/spark-class miss Java versi...

2014-10-13 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2725#issuecomment-58849438 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-10-13 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-58849802 @zhzhan @scwf - I think this should be okay now for protobuf. We made some other changes this week updating the protobuf version to be based on protobuf 2.5 instead of

[GitHub] spark pull request: [SPARK-3899][Doc]fix wrong links in streaming ...

2014-10-13 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/2749#issuecomment-58849985 Hmm, but #implementing-and-using-a-custom-actor-based-receiver is a not valid link, sorry, did not get you, can you explain more? --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-3869] ./bin/spark-class miss Java versi...

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2725#issuecomment-58850138 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-10-13 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-58850172 Ok, thanks for that, i will also test it in https://github.com/apache/spark/pull/2685 --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-3899][Doc]fix wrong links in streaming ...

2014-10-13 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2749#issuecomment-58850273 Oh, I meant that you could have linked the page like this, so that the link jumps to the Akka-specific section:

[GitHub] spark pull request: Bug Fix: without unpersist method in RandomFor...

2014-10-13 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2775#issuecomment-58850468 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3869] ./bin/spark-class miss Java versi...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2725#issuecomment-58850691 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/354/consoleFull) for PR 2725 at commit

[GitHub] spark pull request: Bug Fix: without unpersist method in RandomFor...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2775#issuecomment-58851060 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21678/consoleFull) for PR 2775 at commit

[GitHub] spark pull request: [SPARK-3899][Doc]fix wrong links in streaming ...

2014-10-13 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/2749#issuecomment-58851126 Get it, use ```streaming-custom-receivers.html#implementing-and-using-a-custom-actor-based-receiver``` here jumps to the Akka-specific section:) --- If your project is

[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...

2014-10-13 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-58851300 @pwendell I don't see an easy way with maven shade plugin either ? Do you ?, One way is to include a fake dependency and then ask it to shade that across all

[GitHub] spark pull request: [SPARK-3899][Doc]fix wrong links in streaming ...

2014-10-13 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/2749#issuecomment-58851418 Updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-3899][Doc]fix wrong links in streaming ...

2014-10-13 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2749#issuecomment-58851490 LGTM. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3899][Doc]fix wrong links in streaming ...

2014-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2749 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3854] Scala style: require spaces befor...

2014-10-13 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2761#issuecomment-58852435 O.K. I close this PR for now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3921] Fix CoarseGrainedExecutorBackend'...

2014-10-13 Thread aarondav
GitHub user aarondav opened a pull request: https://github.com/apache/spark/pull/2779 [SPARK-3921] Fix CoarseGrainedExecutorBackend's arguments for Standalone mode The goal of this patch is to fix the swapped arguments in standalone mode, which was caused by

[GitHub] spark pull request: [SPARK-3854] Scala style: require spaces befor...

2014-10-13 Thread sarutak
Github user sarutak closed the pull request at: https://github.com/apache/spark/pull/2761 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread chouqin
GitHub user chouqin opened a pull request: https://github.com/apache/spark/pull/2780 [SPARK-3207][MLLIB]Choose splits for continuous features in DecisionTree more adaptively DecisionTree splits on continuous features by choosing an array of values from a subsample of the data.

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58853083 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58853271 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21680/consoleFull) for PR 2780 at commit

[GitHub] spark pull request: [SPARK-3921] Fix CoarseGrainedExecutorBackend'...

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2779#issuecomment-58853441 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-10-13 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-58853725 Hi @zhzhan and @scwf - I made some changes to the build to simplify it a bit. I made a PR into your branch. I tested it locally compiling for 0.12 and 0.13, but it

[GitHub] spark pull request: [SPARK-3921] Fix CoarseGrainedExecutorBackend'...

2014-10-13 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/2779#issuecomment-58853884 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 3922] Refactor spark-core to use Utils....

2014-10-13 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/2781 [Spark 3922] Refactor spark-core to use Utils.UTF_8 A global UTF8 constant is very helpful to handle encoding problems when converting between String and bytes. There are several solutions here:

[GitHub] spark pull request: [Spark 3922] Refactor spark-core to use Utils....

2014-10-13 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/2781#issuecomment-58854034 /cc @rxin, @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [Spark 3922] Refactor spark-core to use Utils....

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2781#issuecomment-58854193 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3921] Fix CoarseGrainedExecutorBackend'...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2779#issuecomment-58854400 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21682/consoleFull) for PR 2779 at commit

[GitHub] spark pull request: [Spark 3922] Refactor spark-core to use Utils....

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2781#issuecomment-58854409 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21681/consoleFull) for PR 2781 at commit

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-10-13 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-58854626 Note @scwf there are some TODO's in there that need to be addressed in your patch for JDBC. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-3453] Netty-based BlockTransferService,...

2014-10-13 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/2753#discussion_r18755797 --- Diff: core/src/main/scala/org/apache/spark/network/netty/NettyBlockFetcher.scala --- @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [Spark 3922] Refactor spark-core to use Utils....

2014-10-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2781#issuecomment-58855147 I vote for `com.google.common.base.Charsets.UTF_8` now, and `java.nio.charset.StandardCharsets.UTF_8` when Spark moves to Java 7+. No need to define this constant yet

[GitHub] spark pull request: [SPARK-3869] ./bin/spark-class miss Java versi...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2725#issuecomment-58855812 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/354/consoleFull) for PR 2725 at commit

[GitHub] spark pull request: Bug Fix: without unpersist method in RandomFor...

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2775#issuecomment-58856471 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3921] Fix CoarseGrainedExecutorBackend'...

2014-10-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2779#issuecomment-58856481 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: Bug Fix: without unpersist method in RandomFor...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2775#issuecomment-58856468 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21678/consoleFull) for PR 2775 at commit

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-13 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-58856535 @pwendell, i am resolving the conflicts, other TODO's here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1405][MLLIB] topic modeling on Graphx

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-58856594 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21683/consoleFull) for PR 2388 at commit

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58857589 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21680/consoleFull) for PR 2780 at commit

[GitHub] spark pull request: [Spark 3922] Refactor spark-core to use Utils....

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2781#issuecomment-58859149 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21681/consoleFull) for PR 2781 at commit

[GitHub] spark pull request: [Spark 3922] Refactor spark-core to use Utils....

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2781#issuecomment-58859154 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3921] Fix CoarseGrainedExecutorBackend'...

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2779#issuecomment-58859175 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [WIP] SPARK-2426: Quadratic Minimizati...

2014-10-13 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/2705#issuecomment-58862691 @chanda what's your problem formulation? min x'Hx + c'x s.t Ax = B You can write it as min x'Hx + c'x + g(z) s.t Ax = B + z g(z) here is indicator

[GitHub] spark pull request: [SPARK-1405][MLLIB] topic modeling on Graphx

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-58862844 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21683/consoleFull) for PR 2388 at commit

[GitHub] spark pull request: [SPARK-1405][MLLIB] topic modeling on Graphx

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-58862848 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [WIP] SPARK-2426: Quadratic Minimizati...

2014-10-13 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/2705#issuecomment-58863216 @Chanda breeze sparse matrix does not solve your problem since breeze does not have sparse LDL but the ECOS jar has the ldl and amd native libraries which we will use

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread chouqin
Github user chouqin commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58865080 Jekins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58865274 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21684/consoleFull) for PR 2780 at commit

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58865777 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21685/consoleFull) for PR 2780 at commit

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread chouqin
Github user chouqin commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58865951 @jkbradley, RandomForestSuite fails because original splits are better fit for the training data(for example, 899.5 is a split threshold, which is close to 900.) I think

[GitHub] spark pull request: [SPARK-3814][SQL] Bitwise does not work in H...

2014-10-13 Thread ravipesala
Github user ravipesala commented on the pull request: https://github.com/apache/spark/pull/2736#issuecomment-58872032 Thank you @scwf , I have created new PR since it has merge conflicts. It will not be neat If I rebase and push to old PR because it will show all changed files which

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58872054 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58872046 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21684/consoleFull) for PR 2780 at commit

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58872500 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58872495 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21685/consoleFull) for PR 2780 at commit

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2014-10-13 Thread viper-kun
Github user viper-kun commented on a diff in the pull request: https://github.com/apache/spark/pull/2471#discussion_r18763243 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -214,6 +224,43 @@ private[history] class

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58876511 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21686/consoleFull) for PR 2780 at commit

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2771#discussion_r18764783 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala --- @@ -99,6 +99,16 @@ private[hive] case class

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-13 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2771#issuecomment-58881732 This LGTM. Would you mind to add some tests? Probably in `ExpressionOptimizationSuite`. Thanks. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58883501 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21686/consoleFull) for PR 2780 at commit

[GitHub] spark pull request: [SPARK-3207][MLLIB]Choose splits for continuou...

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2780#issuecomment-58883509 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18765864 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/HiveFromSpark.scala --- @@ -62,6 +62,16 @@ object HiveFromSpark {

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18765873 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -114,6 +114,22 @@ case class

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766058 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -128,6 +144,13 @@ case class WriteToFile(

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766091 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala --- @@ -77,6 +77,18 @@ private[sql] trait SchemaRDDLike { }

[GitHub] spark pull request: [SPARK-3580] add 'partitions' property to PySp...

2014-10-13 Thread mattf
Github user mattf commented on the pull request: https://github.com/apache/spark/pull/2478#issuecomment-58884300 @JoshRosen @pwendell any further comment on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766361 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -504,19 +505,41 @@ private[parquet] object

[GitHub] spark pull request: [SPARK-3818] Graph coarsening

2014-10-13 Thread uncleGen
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/2679#issuecomment-58885039 @ankurdave I have some doubts, but not about this patch. In [GraphX OSDI paper](http://ankurdave.com/dl/graphx-osdi14.pdf) , I find that you have implemented a

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766443 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -504,19 +505,41 @@ private[parquet] object

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2014-10-13 Thread mattf
Github user mattf commented on the pull request: https://github.com/apache/spark/pull/2471#issuecomment-58885115 @mattf I understand what you're trying to say, but think about it in context. As I said above, the when to poll the file system code is the most trivial part of this

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766608 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -504,19 +505,41 @@ private[parquet] object

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766670 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala --- @@ -121,6 +122,48 @@ class HiveContext(sc: SparkContext) extends

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766697 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala --- @@ -121,6 +122,48 @@ class HiveContext(sc: SparkContext) extends

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766749 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -28,6 +28,7 @@ import

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18766771 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -221,4 +222,24 @@ private[hive] trait HiveStrategies {

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread ScrapCodes
GitHub user ScrapCodes opened a pull request: https://github.com/apache/spark/pull/2782 SPARK-3874, Provide stable TaskContext API You can merge this pull request into a Git repository by running: $ git pull https://github.com/ScrapCodes/spark-1 SPARK-3874/stable-tc

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-58886892 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18767300 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-58887166 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21687/consoleFull) for PR 2782 at commit

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-58887515 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21687/consoleFull) for PR 2782 at commit

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-58887519 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-5896 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18768084 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-58889380 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21688/consoleFull) for PR 2782 at commit

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18768122 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-58889799 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-58889796 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21688/consoleFull) for PR 2782 at commit

[GitHub] spark pull request: [SPARK-1405][MLLIB] topic modeling on Graphx

2014-10-13 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/2388#discussion_r18768316 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/TopicModeling.scala --- @@ -0,0 +1,682 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18768380 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18768433 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-3874, Provide stable TaskContext API

2014-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2782#issuecomment-58890576 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21689/consoleFull) for PR 2782 at commit

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18768525 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [spark-3586][streaming]Support nested director...

2014-10-13 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/2765#issuecomment-58890751 Hi @wangxiaojing ,a small suggestion, why not making this improvement more flexible by adding a parameter to control the searching depth of directories, this will be

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18768639 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18768648 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-13 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2576#discussion_r18768785 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software

  1   2   3   4   >