[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/1 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17781: [SPARK-20476] [SQL] Block users to create a table that u...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17781 **[Test build #76238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76238/testReport)** for PR 17781 at commit

[GitHub] spark pull request #17130: [SPARK-19791] [ML] Add doc and example for fpgrow...

2017-04-27 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17130#discussion_r113782786 --- Diff: docs/ml-frequent-pattern-mining.md --- @@ -0,0 +1,87 @@ +--- +layout: global +title: Frequent Pattern Mining +displayTitle:

[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/1 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76233/ Test PASSed. ---

[GitHub] spark issue #16714: [SPARK-16333][Core] Enable EventLoggingListener to log l...

2017-04-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16714 Options put the burden on the user to figure things out (do I need to set this or not?). If you want to investigate whether you can trim more data (e.g. internal metrics that just mirror stuff

[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #76233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76233/testReport)** for PR 1 at commit

[GitHub] spark issue #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqNullSafe...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17783 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76237/ Test FAILed. ---

[GitHub] spark issue #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqNullSafe...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17783 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqNullSafe...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17783 **[Test build #76237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76237/testReport)** for PR 17783 at commit

[GitHub] spark issue #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqNullSafe...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17783 **[Test build #76237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76237/testReport)** for PR 17783 at commit

[GitHub] spark issue #17787: [SPARK-20496][SS] Bug in KafkaWriter Looks at Unanalyzed...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17787 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17781: [SPARK-20476] [SQL] Block users to create a table that u...

2017-04-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17781 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17787: [SPARK-20496][SS] Bug in KafkaWriter Looks at Unanalyzed...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17787 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76236/ Test PASSed. ---

[GitHub] spark issue #17787: [SPARK-20496][SS] Bug in KafkaWriter Looks at Unanalyzed...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17787 **[Test build #76236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76236/testReport)** for PR 17787 at commit

[GitHub] spark pull request #17780: [SPARK-20487][SQL] `HiveTableScan` node is quite ...

2017-04-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17780 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17780: [SPARK-20487][SQL] `HiveTableScan` node is quite verbose...

2017-04-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17780 Thanks! Merging to master/2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/1 Thanks! Merging to master/2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17777: [SPARK-20482][SQL] Resolving Casts is too strict ...

2017-04-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16714: [SPARK-16333][Core] Enable EventLoggingListener to log l...

2017-04-27 Thread jisookim0513
Github user jisookim0513 commented on the issue: https://github.com/apache/spark/pull/16714 @vanzin @ajbozarth if you guys think having an option to skip logging internal accumulators (in my case I don't use the SQL UI) and completely getting rid of updated block statues are not

[GitHub] spark pull request #17744: [SPARK-20426] Lazy initialization of FileSegmentM...

2017-04-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17744 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqN...

2017-04-27 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17783#discussion_r113777544 --- Diff: R/pkg/R/column.R --- @@ -302,3 +301,65 @@ setMethod("otherwise", jc <- callJMethod(x@jc, "otherwise", value)

[GitHub] spark issue #17744: [SPARK-20426] Lazy initialization of FileSegmentManagedB...

2017-04-27 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/17744 We see the same issue on some of our clusters. I was planning on doing 2 things. Something like this to reduce that memory usage and then on the other side you could change the shuffle fetcher

[GitHub] spark pull request #16714: [SPARK-16333][Core] Enable EventLoggingListener t...

2017-04-27 Thread jisookim0513
Github user jisookim0513 commented on a diff in the pull request: https://github.com/apache/spark/pull/16714#discussion_r113776549 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -343,10 +376,14 @@ private[spark] object JsonProtocol {

[GitHub] spark issue #17787: [SPARK-20496][SS] Bug in KafkaWriter Looks at Unanalyzed...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17787 **[Test build #76236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76236/testReport)** for PR 17787 at commit

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76232/ Test PASSed. ---

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17191 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17191 **[Test build #76232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76232/testReport)** for PR 17191 at commit

[GitHub] spark issue #16714: [SPARK-16333][Core] Enable EventLoggingListener to log l...

2017-04-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16714 I think so. Just replying to the question. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16714: [SPARK-16333][Core] Enable EventLoggingListener to log l...

2017-04-27 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/16714 Didn't #17412 already get rid of block statuses though? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #17787: [SS] Bug in KafkaWriter Looks at Unanalyzed Plans

2017-04-27 Thread anabranch
GitHub user anabranch opened a pull request: https://github.com/apache/spark/pull/17787 [SS] Bug in KafkaWriter Looks at Unanalyzed Plans ## What changes were proposed in this pull request? This now asserts that a plan has been analyzed before reading the schema.

[GitHub] spark issue #16714: [SPARK-16333][Core] Enable EventLoggingListener to log l...

2017-04-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16714 After having played with some of this code for other reasons, at least some of the internal accumulators are needed to rebuild the SQL UI. As far as logging updated block statuses, I still

[GitHub] spark pull request #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqN...

2017-04-27 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17783#discussion_r113773816 --- Diff: R/pkg/R/column.R --- @@ -302,3 +301,65 @@ setMethod("otherwise", jc <- callJMethod(x@jc, "otherwise", value)

[GitHub] spark issue #16714: [SPARK-16333][Core] Enable EventLoggingListener to log l...

2017-04-27 Thread jisookim0513
Github user jisookim0513 commented on the issue: https://github.com/apache/spark/pull/16714 I would still like not to have internal accumulators in the event logs, as well as updated block statuses. @vanzin would you be ok with eliminating all internal accumulators and have an option

[GitHub] spark pull request #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqN...

2017-04-27 Thread zero323
Github user zero323 commented on a diff in the pull request: https://github.com/apache/spark/pull/17783#discussion_r113772682 --- Diff: R/pkg/R/functions.R --- @@ -3803,3 +3803,41 @@ setMethod("repeat_string", jc <- callJStatic("org.apache.spark.sql.functions",

[GitHub] spark issue #17768: [SPARK-20465][CORE] Throws a proper exception when any t...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17768 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76231/ Test PASSed. ---

[GitHub] spark issue #17768: [SPARK-20465][CORE] Throws a proper exception when any t...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17768 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17768: [SPARK-20465][CORE] Throws a proper exception when any t...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17768 **[Test build #76231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76231/testReport)** for PR 17768 at commit

[GitHub] spark pull request #17766: [SPARK-20421][core] Mark internal listeners as de...

2017-04-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17766 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16714: [SPARK-16333][Core] Enable EventLoggingListener to log l...

2017-04-27 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/16714 Is this still an issue or did #17412 fix this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17766: [SPARK-20421][core] Mark internal listeners as deprecate...

2017-04-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17766 Thanks. Merging to master / 2.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16609: [SPARK-8480] [CORE] [PYSPARK] [SPARKR] Add setName for D...

2017-04-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16609 The JIRA https://issues.apache.org/jira/browse/SPARK-20495 was just opened. Please feel free to submit the PR --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #17774: [SPARK-18371][Streaming] Spark Streaming backpressure ge...

2017-04-27 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/17774 LGTM pending jason's comments on tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqNullSafe...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17783 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqNullSafe...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17783 **[Test build #76235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76235/testReport)** for PR 17783 at commit

[GitHub] spark issue #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqNullSafe...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17783 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76235/ Test FAILed. ---

[GitHub] spark issue #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqNullSafe...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17783 **[Test build #76235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76235/testReport)** for PR 17783 at commit

[GitHub] spark issue #17771: [SPARK-20471]Remove AggregateBenchmark testsuite warning...

2017-04-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17771 Could you update the PR title to ``` [SPARK-20471]Remove AggregateBenchmark testsuite warning: Two level hashmap is disabled but vectorized hashmap is enabled ``` --- If your

[GitHub] spark pull request #17786: [SPARK-20483] Mesos Coarse mode may starve other ...

2017-04-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17786 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 LGTM. Merged into master and branch 2.2. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recomm...

2017-04-27 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17742#discussion_r113762883 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -276,18 +277,39 @@ object

[GitHub] spark pull request #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recomm...

2017-04-27 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17742#discussion_r113762745 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -297,23 +319,10 @@ object

[GitHub] spark pull request #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recomm...

2017-04-27 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17742#discussion_r113762344 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -277,39 +277,38 @@ object

[GitHub] spark pull request #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recomm...

2017-04-27 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17742#discussion_r113761881 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -277,39 +277,38 @@ object

[GitHub] spark pull request #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recomm...

2017-04-27 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17742#discussion_r113761692 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -277,39 +277,38 @@ object

[GitHub] spark issue #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recommendForA...

2017-04-27 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17742 @mpjlu yeah we can do the ML version in a follow up PR that is ok (I can help if needed). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #17742: [Spark-11968][ML][MLLIB]Optimize MLLIB ALS recomm...

2017-04-27 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17742#discussion_r113761236 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -277,17 +278,39 @@ object

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17786 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76234/ Test PASSed. ---

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17786 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17786 **[Test build #76234 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76234/testReport)** for PR 17786 at commit

[GitHub] spark issue #17785: [SPARK-20493][R] De-deuplicate parse logics for DDL-like...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17785 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76230/ Test PASSed. ---

[GitHub] spark issue #17785: [SPARK-20493][R] De-deuplicate parse logics for DDL-like...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17785 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17785: [SPARK-20493][R] De-deuplicate parse logics for DDL-like...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17785 **[Test build #76230 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76230/testReport)** for PR 17785 at commit

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76229/ Test FAILed. ---

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #76229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76229/testReport)** for PR 15009 at commit

[GitHub] spark issue #17785: [SPARK-20493][R] De-deuplicate parse logics for DDL-like...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17785 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17785: [SPARK-20493][R] De-deuplicate parse logics for DDL-like...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17785 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76228/ Test PASSed. ---

[GitHub] spark issue #17785: [SPARK-20493][R] De-deuplicate parse logics for DDL-like...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17785 **[Test build #76228 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76228/testReport)** for PR 17785 at commit

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17786 **[Test build #76234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76234/testReport)** for PR 17786 at commit

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 Jenkins, ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17766: [SPARK-20421][core] Mark internal listeners as deprecate...

2017-04-27 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/17766 Per my comments on the earlier email discussion thread, this seems fine to me. (I'm not totally MIA; I just got back from vacation and busy getting caught up). --- If your project is

[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/1 LGTM pending Jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqN...

2017-04-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17783#discussion_r113754078 --- Diff: R/pkg/R/column.R --- @@ -302,3 +301,65 @@ setMethod("otherwise", jc <- callJMethod(x@jc, "otherwise", value)

[GitHub] spark issue #17778: [SPARK-20494] Implement UDF array_unique in Spark with c...

2017-04-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17778 @janewangfb that's because `array_contains` is a Hive function --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #17774: [SPARK-18371][Streaming] Spark Streaming backpressure ge...

2017-04-27 Thread JasonMWhite
Github user JasonMWhite commented on the issue: https://github.com/apache/spark/pull/17774 Tests have some fairly repetitive code, but not sure if that's a problem or not. Looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #17673: [SPARK-20372] [ML] Word2Vec Continuous Bag of Words mode...

2017-04-27 Thread shubhamchopra
Github user shubhamchopra commented on the issue: https://github.com/apache/spark/pull/17673 @Krimit @MLnick @hhbyyh I am working on getting your earlier queries answered. @Krimit Thanks for looking into the code, I will try to get the code-review feedback incorporated in a

[GitHub] spark pull request #17783: [SPARK-20490][SPARKR][WIP] Add R wrappers for eqN...

2017-04-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17783#discussion_r113753986 --- Diff: R/pkg/R/functions.R --- @@ -3803,3 +3803,41 @@ setMethod("repeat_string", jc <- callJStatic("org.apache.spark.sql.functions",

[GitHub] spark issue #17773: [SPARK-20474] Fixing OnHeapColumnVector reallocation

2017-04-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17773 Yes, I think it should see `capacity`. I will create a PR tomorrow for `OffHeapColumnVector`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #17774: [SPARK-18371][Streaming] Spark Streaming backpres...

2017-04-27 Thread JasonMWhite
Github user JasonMWhite commented on a diff in the pull request: https://github.com/apache/spark/pull/17774#discussion_r113753764 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/DirectKafkaStreamSuite.scala --- @@ -617,6 +617,94 @@ class

[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/1 **[Test build #76233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76233/testReport)** for PR 1 at commit

[GitHub] spark issue #17778: Add array_unique UDF

2017-04-27 Thread janewangfb
Github user janewangfb commented on the issue: https://github.com/apache/spark/pull/17778 @srowen We have array_contains UDF. I think it is nice to have one that removes all the duplicated elements. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-27 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/1 Thanks for the review! I'll update the patch again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17766: [SPARK-20421][core] Mark internal listeners as deprecate...

2017-04-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17766 Josh seems MIA, @srowen care to take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17777: [SPARK-20482][SQL] Resolving Casts is too strict ...

2017-04-27 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r113751399 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -89,6 +89,22 @@ object Cast { case _ =>

[GitHub] spark issue #17774: [SPARK-18371][Streaming] Spark Streaming backpressure ge...

2017-04-27 Thread arzt
Github user arzt commented on the issue: https://github.com/apache/spark/pull/17774 I changed the max messages per partition to be at least 1. Agreed? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #17777: [SPARK-20482][SQL] Resolving Casts is too strict ...

2017-04-27 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r113751468 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -165,6 +181,14 @@ case class Cast(child: Expression,

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17786 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17786: [SPARK-20483] Mesos Coarse mode may starve other ...

2017-04-27 Thread dgshep
GitHub user dgshep opened a pull request: https://github.com/apache/spark/pull/17786 [SPARK-20483] Mesos Coarse mode may starve other Mesos frameworks Set max cores to a multiple of the smallestExecutorSize ## What changes were proposed in this pull request? Set

[GitHub] spark pull request #12004: [SPARK-7481] [build] Add spark-cloud module to pu...

2017-04-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/12004#discussion_r113750071 --- Diff: pom.xml --- @@ -1145,6 +1150,70 @@ + + --- End diff -- I'm

[GitHub] spark pull request #12004: [SPARK-7481] [build] Add spark-cloud module to pu...

2017-04-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/12004#discussion_r113749971 --- Diff: pom.xml --- @@ -621,6 +621,11 @@ ${fasterxml.jackson.version} +

[GitHub] spark pull request #12004: [SPARK-7481] [build] Add spark-cloud module to pu...

2017-04-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/12004#discussion_r113749781 --- Diff: cloud/pom.xml --- @@ -0,0 +1,117 @@ + + +http://maven.apache.org/POM/4.0.0; +

[GitHub] spark pull request #12004: [SPARK-7481] [build] Add spark-cloud module to pu...

2017-04-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/12004#discussion_r113749812 --- Diff: docs/storage-openstack-swift.md --- @@ -19,41 +20,32 @@ Although not mandatory, it is recommended to configure the proxy server of Swift

[GitHub] spark pull request #12004: [SPARK-7481] [build] Add spark-cloud module to pu...

2017-04-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/12004#discussion_r113749428 --- Diff: cloud/pom.xml --- @@ -0,0 +1,158 @@ + + +http://maven.apache.org/POM/4.0.0;

[GitHub] spark issue #17774: [SPARK-18371][Streaming] Spark Streaming backpressure ge...

2017-04-27 Thread JasonMWhite
Github user JasonMWhite commented on the issue: https://github.com/apache/spark/pull/17774 I think @koeninger's suggestion is valid. `effectiveRateLimitPerPartition` is the upper bound on the number of messages per partition per second, and `maxMessagesPerPartition` sets an upper

[GitHub] spark issue #17774: [SPARK-18371][Streaming] Spark Streaming backpressure ge...

2017-04-27 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/17774 Have you read the function def clamp? Rate limit of 1 should not imply an attempt to grab 1 message even if it doesn't exist. On Apr 27, 2017 11:01, "Sebastian Arzt"

[GitHub] spark issue #17191: [SPARK-14471][SQL] Aliases in SELECT could be used in GR...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17191 **[Test build #76232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76232/testReport)** for PR 17191 at commit

[GitHub] spark issue #17768: [SPARK-20465][CORE] Throws a proper exception when any t...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17768 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17768: [SPARK-20465][CORE] Throws a proper exception when any t...

2017-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17768 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76225/ Test FAILed. ---

[GitHub] spark issue #17768: [SPARK-20465][CORE] Throws a proper exception when any t...

2017-04-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17768 **[Test build #76225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76225/testReport)** for PR 17768 at commit

<    1   2   3   4   5   >