[jira] [Created] (SPARK-4828) sum and avg over empty table should return null

Adrian Wang created SPARK-4828: -- Summary: sum and avg over empty table should return null Key: SPARK-4828 URL: https://issues.apache.org/jira/browse/SPARK-4828 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4828) sum and avg over empty table should return null

[ https://issues.apache.org/jira/browse/SPARK-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242388#comment-14242388 ] Apache Spark commented on SPARK-4828: - User 'adrian-wang' has created a pull request

[jira] [Created] (SPARK-4829) eliminate expressions calculation in count expression

Adrian Wang created SPARK-4829: -- Summary: eliminate expressions calculation in count expression Key: SPARK-4829 URL: https://issues.apache.org/jira/browse/SPARK-4829 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242402#comment-14242402 ] Sean Owen commented on SPARK-4817: -- [~tianyi] I agree that it would be nice to add a

[jira] [Commented] (SPARK-4829) eliminate expressions calculation in count expression

[ https://issues.apache.org/jira/browse/SPARK-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242413#comment-14242413 ] Apache Spark commented on SPARK-4829: - User 'adrian-wang' has created a pull request

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242436#comment-14242436 ] Paulo Motta commented on SPARK-2984: We're also facing a similar issue when using S3N,

[jira] [Commented] (SPARK-4156) Add expectation maximization for Gaussian mixture models to MLLib clustering

[ https://issues.apache.org/jira/browse/SPARK-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242486#comment-14242486 ] Meethu Mathew commented on SPARK-4156: -- [~tgaloppo] The current version of the code

[jira] [Created] (SPARK-4830) Spark Java Application : java.lang.ClassNotFoundException

Mykhaylo Telizhyn created SPARK-4830: Summary: Spark Java Application : java.lang.ClassNotFoundException Key: SPARK-4830 URL: https://issues.apache.org/jira/browse/SPARK-4830 Project: Spark

[jira] [Commented] (SPARK-4156) Add expectation maximization for Gaussian mixture models to MLLib clustering

[ https://issues.apache.org/jira/browse/SPARK-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242494#comment-14242494 ] Travis Galoppo commented on SPARK-4156: --- [~MeethuMathew] This would be great! If

[jira] [Updated] (SPARK-4830) Spark Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Description: We have Spark Streaming application that consumes messages from RabbitMQ and

[jira] [Commented] (SPARK-4526) Gradient should be added batch computing interface

[ https://issues.apache.org/jira/browse/SPARK-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242504#comment-14242504 ] Apache Spark commented on SPARK-4526: - User 'witgo' has created a pull request for

[jira] [Updated] (SPARK-4830) Spark Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Description: h4. Application Overview: We have Spark Streaming application that

[jira] [Updated] (SPARK-4830) Spark Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Description: h4. Application Overview: We have Spark Streaming application that

[jira] [Updated] (SPARK-4830) Spark Streaming Java Application : java.lang.ClassNotFoundException

[ https://issues.apache.org/jira/browse/SPARK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mykhaylo Telizhyn updated SPARK-4830: - Summary: Spark Streaming Java Application : java.lang.ClassNotFoundException (was: Spark

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242547#comment-14242547 ] koert kuipers commented on SPARK-3655: -- i updated the pullreq to use Iterables

[jira] [Resolved] (SPARK-4806) Update Streaming Programming Guide for Spark 1.2

[ https://issues.apache.org/jira/browse/SPARK-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4806. -- Resolution: Done Target Version/s: 1.2.0, 1.3.0 (was: 1.2.0) Update Streaming

[jira] [Commented] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242611#comment-14242611 ] Cheng Lian commented on SPARK-4814: --- This assertion failure seems to be related to

[jira] [Commented] (SPARK-4740) Netty's network throughput is about 1/2 of NIO's in spark-perf sortByKey

[ https://issues.apache.org/jira/browse/SPARK-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242656#comment-14242656 ] Zhang, Liye commented on SPARK-4740: Hi [~adav], I missed there is another patch from

[jira] [Created] (SPARK-4831) Current directory always on classpath with spark-submit

Daniel Darabos created SPARK-4831: - Summary: Current directory always on classpath with spark-submit Key: SPARK-4831 URL: https://issues.apache.org/jira/browse/SPARK-4831 Project: Spark

[jira] [Commented] (SPARK-2980) Python support for chi-squared test

[ https://issues.apache.org/jira/browse/SPARK-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242718#comment-14242718 ] Apache Spark commented on SPARK-2980: - User 'jbencook' has created a pull request for

[jira] [Commented] (SPARK-4831) Current directory always on classpath with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242778#comment-14242778 ] Sean Owen commented on SPARK-4831: -- Hm, so I made a quick test, where I put a class

[jira] [Reopened] (SPARK-2892) Socket Receiver does not stop when streaming context is stopped

[ https://issues.apache.org/jira/browse/SPARK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-2892: -- OK. The issues may have a common cause but that can be deferred until the other JIRA is resolved. If it

[jira] [Resolved] (SPARK-3677) pom.xml and SparkBuild.scala are wrong : Scalastyle is never applyed to the sources under yarn/common

[ https://issues.apache.org/jira/browse/SPARK-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3677. -- Resolution: Not a Problem Target Version/s: (was: 1.1.2, 1.2.1) Obsoleted by the

[jira] [Resolved] (SPARK-3918) Forget Unpersist in RandomForest.scala(train Method)

[ https://issues.apache.org/jira/browse/SPARK-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3918. -- Resolution: Fixed Great, looks like this was in fact fixed for 1.2 then. Forget Unpersist in

[jira] [Updated] (SPARK-3918) Forget Unpersist in RandomForest.scala(train Method)

[ https://issues.apache.org/jira/browse/SPARK-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3918: - Target Version/s: 1.2.0 (was: 1.1.0) Affects Version/s: (was: 1.2.0)

[jira] [Resolved] (SPARK-4458) Skip compilation of tests classes when using make-distribution

[ https://issues.apache.org/jira/browse/SPARK-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4458. -- Resolution: Won't Fix Given PR discussion, looks like a WontFix. Skip compilation of tests classes

[jira] [Commented] (SPARK-4675) Find similar products and similar users in MatrixFactorizationModel

[ https://issues.apache.org/jira/browse/SPARK-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242808#comment-14242808 ] Sean Owen commented on SPARK-4675: -- The lower dimensional space is of course smaller.

[jira] [Commented] (SPARK-4779) PySpark Shuffle Fails Looking for Files that Don't Exist when low on Memory

[ https://issues.apache.org/jira/browse/SPARK-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242831#comment-14242831 ] Ilya Ganelin commented on SPARK-4779: - I've seen this issue on Scala as well. This

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242913#comment-14242913 ] Ilya Ganelin commented on SPARK-3533: - I am looking into a solution for this. Add

[jira] [Commented] (SPARK-4728) Add exponential, log normal, and gamma distributions to data generator to MLlib

[ https://issues.apache.org/jira/browse/SPARK-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242915#comment-14242915 ] Apache Spark commented on SPARK-4728: - User 'rnowling' has created a pull request for

[jira] [Commented] (SPARK-4728) Add exponential, log normal, and gamma distributions to data generator to MLlib

[ https://issues.apache.org/jira/browse/SPARK-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242924#comment-14242924 ] RJ Nowling commented on SPARK-4728: --- I posted a PR for this issue:

[jira] [Issue Comment Deleted] (SPARK-4728) Add exponential, log normal, and gamma distributions to data generator to MLlib

[ https://issues.apache.org/jira/browse/SPARK-4728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RJ Nowling updated SPARK-4728: -- Comment: was deleted (was: I posted a PR for this issue: https://github.com/apache/spark/pull/3680)

[jira] [Commented] (SPARK-4675) Find similar products and similar users in MatrixFactorizationModel

[ https://issues.apache.org/jira/browse/SPARK-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243026#comment-14243026 ] Debasish Das commented on SPARK-4675: - Is there a metric like MAP / AUC kind of

[jira] [Commented] (SPARK-4455) Exclude dependency on hbase-annotations module

[ https://issues.apache.org/jira/browse/SPARK-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243044#comment-14243044 ] Sean Owen commented on SPARK-4455: -- FWIW this change also unfortunately causes compiler

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243048#comment-14243048 ] Debasish Das commented on SPARK-4823: - [~srowen] did you implement map-reduce row

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243055#comment-14243055 ] Sean Owen commented on SPARK-4823: -- I don't think MapReduce matters here. You can compute

[jira] [Commented] (SPARK-1412) Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243072#comment-14243072 ] Sean Owen commented on SPARK-1412: -- The PRs for SPARK-2253 and SPARK-1412 were abandoned.

[jira] [Commented] (SPARK-4831) Current directory always on classpath with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243074#comment-14243074 ] Daniel Darabos commented on SPARK-4831: --- bq. Is it perhaps finding and exploded

[jira] [Commented] (SPARK-1412) Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243077#comment-14243077 ] Reynold Xin commented on SPARK-1412: I think we should still do it - it's just that

[jira] [Updated] (SPARK-1412) Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1412: --- Fix Version/s: (was: 1.2.0) Disable partial aggregation automatically when reduction factor is

[jira] [Updated] (SPARK-1412) [SQL] Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1412: --- Summary: [SQL] Disable partial aggregation automatically when reduction factor is low (was: Disable

[jira] [Resolved] (SPARK-1627) Support external aggregation in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1627. -- Resolution: Won't Fix Fix Version/s: (was: 1.2.0) The discussion in

[jira] [Updated] (SPARK-2253) [Core] Disable partial aggregation automatically when reduction factor is low

[ https://issues.apache.org/jira/browse/SPARK-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2253: --- Summary: [Core] Disable partial aggregation automatically when reduction factor is low (was: Disable

[jira] [Resolved] (SPARK-1581) Allow One Flume Avro RPC Server for Each Worker rather than Just One Worker

[ https://issues.apache.org/jira/browse/SPARK-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1581. -- Resolution: Won't Fix No follow-up from OP explaining the change, and so the PR was closed already.

[jira] [Resolved] (SPARK-1559) Add conf dir to CLASSPATH in compute-classpath.sh dependent on whether SPARK_CONF_DIR is set

[ https://issues.apache.org/jira/browse/SPARK-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1559. -- Resolution: Duplicate The PR discussion suggests it was duplicated by the PR for SPARK-2058. Add conf

[jira] [Commented] (SPARK-1532) provide option for more restrictive firewall rule in ec2/spark_ec2.py

[ https://issues.apache.org/jira/browse/SPARK-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243086#comment-14243086 ] Sean Owen commented on SPARK-1532: -- [~foundart] Is this abandoned? your second PR was

[jira] [Resolved] (SPARK-1771) CoarseGrainedSchedulerBackend is not resilient to Akka restarts

[ https://issues.apache.org/jira/browse/SPARK-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1771. -- Resolution: Won't Fix The PR says this was abandoned in favor of SPARK-4004

[jira] [Resolved] (SPARK-1700) PythonRDD leaks socket descriptors during cancellation

[ https://issues.apache.org/jira/browse/SPARK-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1700. -- Resolution: Fixed Fix Version/s: 1.0.0 The PR was https://github.com/apache/spark/pull/623 and

[jira] [Resolved] (SPARK-1888) enhance MEMORY_AND_DISK mode by dropping blocks in parallel

[ https://issues.apache.org/jira/browse/SPARK-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1888. -- Resolution: Duplicate From the end of the PR discussion, it looks like this was continued in

[jira] [Commented] (SPARK-4746) integration tests should be separated from faster unit tests

[ https://issues.apache.org/jira/browse/SPARK-4746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243101#comment-14243101 ] Ryan Williams commented on SPARK-4746: -- I don't have any experience with test tags,

[jira] [Commented] (SPARK-1880) Eliminate unnecessary job executions.

[ https://issues.apache.org/jira/browse/SPARK-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243103#comment-14243103 ] Sean Owen commented on SPARK-1880: -- Is this now a WontFix? The PR refers to this being

[jira] [Closed] (SPARK-1880) Eliminate unnecessary job executions.

[ https://issues.apache.org/jira/browse/SPARK-1880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-1880. -- Resolution: Won't Fix Eliminate unnecessary job executions. -

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243109#comment-14243109 ] Sean Owen commented on SPARK-2016: -- Is this and SPARK-2017 now subsumed by SPARK-3644?

[jira] [Resolved] (SPARK-2227) Support dfs command

[ https://issues.apache.org/jira/browse/SPARK-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2227. -- Resolution: Fixed Fix Version/s: 1.1.0 Looks like this was in fact merged in

[jira] [Resolved] (SPARK-2201) Improve FlumeInputDStream's stability and make it scalable

[ https://issues.apache.org/jira/browse/SPARK-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2201. -- Resolution: Won't Fix I hope I understood this right, but the PR discussion seemed to end with

[jira] [Resolved] (SPARK-2193) Improve tasks‘ preferred locality by sorting tasks partial ordering

[ https://issues.apache.org/jira/browse/SPARK-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2193. -- Resolution: Won't Fix Last word appears to be that this was obviated by SPARK-2294 and

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243133#comment-14243133 ] Reza Zadeh commented on SPARK-4823: --- Given that we're talking about RowMatrices,

[jira] [Resolved] (SPARK-2127) Use application specific folders to dump metrics via CsvSink

[ https://issues.apache.org/jira/browse/SPARK-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2127. -- Resolution: Duplicate The PR that closes SPARK-3377 also closed the PR for this JIRA, and it looks

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243147#comment-14243147 ] Reynold Xin commented on SPARK-2016: cc [~andrewor14] can you comment on this? rdd

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243149#comment-14243149 ] Debasish Das commented on SPARK-2426: - [~mengxr] as per our discussion,

[jira] [Resolved] (SPARK-2381) streaming receiver crashed,but seems nothing happened

[ https://issues.apache.org/jira/browse/SPARK-2381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2381. -- Resolution: Won't Fix PR comments didn't receive follow-up changes either, so per comments here, looks

[jira] [Commented] (SPARK-2016) rdd in-memory storage UI becomes unresponsive when the number of RDD partitions is large

[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243153#comment-14243153 ] Andrew Or commented on SPARK-2016: -- This was filed before SPARK-2316

[jira] [Resolved] (SPARK-2368) Improve io.netty related handlers and clients in network.netty

[ https://issues.apache.org/jira/browse/SPARK-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2368. -- Resolution: Won't Fix PR discussion says that the OP abandoned this change as it was covered by other

[jira] [Resolved] (SPARK-2402) DiskBlockObjectWriter should update the initial position when reusing this object

[ https://issues.apache.org/jira/browse/SPARK-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2402. -- Resolution: Won't Fix There was disagreement about whether to merge this change, but looks like it was

[jira] [Commented] (SPARK-4823) rowSimilarities

[ https://issues.apache.org/jira/browse/SPARK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243207#comment-14243207 ] Debasish Das commented on SPARK-4823: - Even for matrix factorization userFactors are

[jira] [Resolved] (SPARK-2542) Exit Code Class should be renamed and placed package properly

[ https://issues.apache.org/jira/browse/SPARK-2542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2542. -- Resolution: Won't Fix PR discussion says this is WontFix. Exit Code Class should be renamed and

[jira] [Resolved] (SPARK-2671) BlockObjectWriter should create parent directory when the directory doesn't exist

[ https://issues.apache.org/jira/browse/SPARK-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2671. -- Resolution: Won't Fix This is another where the PR discussion indicates this is WontFix.

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243221#comment-14243221 ] Sean Owen commented on SPARK-2604: -- PR comments suggest this was fixed by SPARK-2140?

[jira] [Commented] (SPARK-2770) Rename spark-ganglia-lgpl to ganglia-lgpl

[ https://issues.apache.org/jira/browse/SPARK-2770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243244#comment-14243244 ] Sean Owen commented on SPARK-2770: -- Is this still active? the PR was attempted but got

[jira] [Commented] (SPARK-2750) Add Https support for Web UI

[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243250#comment-14243250 ] Sean Owen commented on SPARK-2750: -- Shall this be rolled into SPARK-3883? both have an

[jira] [Commented] (SPARK-3247) Improved support for external data sources

[ https://issues.apache.org/jira/browse/SPARK-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243253#comment-14243253 ] Matei Zaharia commented on SPARK-3247: -- For those looking to learn about the

[jira] [Resolved] (SPARK-2715) ExternalAppendOnlyMap adds max limit of times and max limit of disk bytes written for spilling

[ https://issues.apache.org/jira/browse/SPARK-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2715. -- Resolution: Won't Fix PR discussion says it is a WontFix. ExternalAppendOnlyMap adds max limit of

[jira] [Resolved] (SPARK-2710) Build SchemaRDD from a JdbcRDD with MetaData (no hard-coded case class)

[ https://issues.apache.org/jira/browse/SPARK-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2710. -- Resolution: Won't Fix PR discussion says that this should become an external library, given the new

[jira] [Resolved] (SPARK-2872) Fix conflict between code and doc in YarnClientSchedulerBackend

[ https://issues.apache.org/jira/browse/SPARK-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2872. -- Resolution: Won't Fix Looks like this was obsoleted by subsequent changes to how YARN parses

[jira] [Resolved] (SPARK-2947) DAGScheduler resubmit the stage into an infinite loop

[ https://issues.apache.org/jira/browse/SPARK-2947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2947. -- Resolution: Duplicate Fix Version/s: (was: 1.0.3) (was: 1.2.0)

[jira] [Resolved] (SPARK-3099) Staging Directory is never deleted when we run job with YARN Client Mode

[ https://issues.apache.org/jira/browse/SPARK-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3099. -- Resolution: Won't Fix PR discussion says this was obsoleted by SPARK-2933. Staging Directory is never

[jira] [Resolved] (SPARK-3038) delete history server logs when there are too many logs

[ https://issues.apache.org/jira/browse/SPARK-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3038. -- Resolution: Won't Fix Fix Version/s: (was: 1.2.0) PR says this is WontFix. delete history

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243398#comment-14243398 ] Valeriy Avanesov commented on SPARK-2426: - what's the normalization constraint ?

[jira] [Resolved] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3124. -- Resolution: Fixed I'm a little unclear on the outcome, but in master, running {{mvn -Phive

[jira] [Commented] (SPARK-3358) PySpark worker fork()ing performance regression in m3.* / PVM instances

[ https://issues.apache.org/jira/browse/SPARK-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243454#comment-14243454 ] Sean Owen commented on SPARK-3358: -- Is this then resolved by one of

[jira] [Resolved] (SPARK-3352) Rename Flume Polling stream to Pull Based stream

[ https://issues.apache.org/jira/browse/SPARK-3352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3352. -- Resolution: Won't Fix According to the PR, this is WontFix. Rename Flume Polling stream to Pull Based

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243456#comment-14243456 ] Debasish Das commented on SPARK-2426: - [~akopich] I got good MAP results on

[jira] [Resolved] (SPARK-3229) spark.shuffle.safetyFraction and spark.storage.safetyFraction is not documented

[ https://issues.apache.org/jira/browse/SPARK-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3229. -- Resolution: Won't Fix Another one that says it's WontFix in the PR. spark.shuffle.safetyFraction and

[jira] [Resolved] (SPARK-3201) Yarn Client do not support the -X java opts

[ https://issues.apache.org/jira/browse/SPARK-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3201. -- Resolution: Won't Fix Looks like it was abandoned by the OP, possibly in favor of SPARK-1953 or

[jira] [Resolved] (SPARK-3548) Display cache hit ratio on WebUI

[ https://issues.apache.org/jira/browse/SPARK-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3548. -- Resolution: Won't Fix I think this particular suggestion is WontFix since the idea of a hit ratio was

[jira] [Resolved] (SPARK-3433) Mima false-positives with @DeveloperAPI and @Experimental annotations

[ https://issues.apache.org/jira/browse/SPARK-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3433. -- Resolution: Fixed Fix Version/s: 1.2.0 The PR was https://github.com/apache/spark/pull/2285 and

[jira] [Resolved] (SPARK-3719) Spark UI: complete/failed stages is better to show the total number of stages

[ https://issues.apache.org/jira/browse/SPARK-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3719. -- Resolution: Duplicate Target Version/s: (was: 1.1.1, 1.2.0) Apparently resolved by the very

[jira] [Resolved] (SPARK-3712) add a new UpdateDStream to update a rdd dynamically

[ https://issues.apache.org/jira/browse/SPARK-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3712. -- Resolution: Won't Fix Target Version/s: (was: 1.1.1, 1.2.0) Withdrawn in the PR by the

[jira] [Resolved] (SPARK-3689) FileLogger should create new instance of FileSystem regardless of it's scheme

[ https://issues.apache.org/jira/browse/SPARK-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3689. -- Resolution: Not a Problem Target Version/s: (was: 1.1.1, 1.2.0) The PR says it's not a

[jira] [Resolved] (SPARK-3663) Document SPARK_LOG_DIR and SPARK_PID_DIR

[ https://issues.apache.org/jira/browse/SPARK-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3663. -- Resolution: Fixed Fix Version/s: 1.3.0 The PR was merged, though looks like not for 1.2:

[jira] [Resolved] (SPARK-3636) It is not friendly to interrupt a Job when user passes different storageLevels to a RDD

[ https://issues.apache.org/jira/browse/SPARK-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3636. -- Resolution: Won't Fix Target Version/s: (was: 1.1.1) PR discussion says this is WontFix

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243683#comment-14243683 ] 宿荣全 commented on SPARK-4817: [~srowen] I‘m sorry that didn't describe the problem clearly. If

[jira] [Resolved] (SPARK-4713) SchemaRDD.unpersist() should not raise exception if it is not cached.

[ https://issues.apache.org/jira/browse/SPARK-4713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4713. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3572

[jira] [Resolved] (SPARK-4662) Whitelist more Hive unittest

[ https://issues.apache.org/jira/browse/SPARK-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4662. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3522

[jira] [Resolved] (SPARK-4639) Pass maxIterations in as a parameter in Analyzer

[ https://issues.apache.org/jira/browse/SPARK-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4639. - Resolution: Fixed Issue resolved by pull request 3499

[jira] [Resolved] (SPARK-4293) Make Cast be able to handle complex types.

[ https://issues.apache.org/jira/browse/SPARK-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4293. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3150

[jira] [Resolved] (SPARK-4828) sum and avg over empty table should return null

[ https://issues.apache.org/jira/browse/SPARK-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4828. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3675

[jira] [Resolved] (SPARK-4825) CTAS fails to resolve when created using saveAsTable

[ https://issues.apache.org/jira/browse/SPARK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4825. - Resolution: Fixed Fix Version/s: 1.2.1 Target Version/s: 1.2.1 (was:

[jira] [Resolved] (SPARK-4742) The name of Parquet File generated by AppendingParquetOutputFormat should be zero padded

[ https://issues.apache.org/jira/browse/SPARK-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4742. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3602

[jira] [Resolved] (SPARK-4829) eliminate expressions calculation in count expression

[ https://issues.apache.org/jira/browse/SPARK-4829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4829. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3676

  1   2   >