[jira] [Updated] (SPARK-3359) `sbt/sbt unidoc` doesn't work with Java 8

[ https://issues.apache.org/jira/browse/SPARK-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3359: - Assignee: Sean Owen `sbt/sbt unidoc` doesn't work with Java 8

[jira] [Updated] (SPARK-4049) Storage web UI fraction cached shows as 100%

[ https://issues.apache.org/jira/browse/SPARK-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4049: -- Priority: Minor (was: Major) Storage web UI fraction cached shows as 100%

[jira] [Created] (SPARK-4090) Memory leak in snappy-java 1.1.1.4/5

Davies Liu created SPARK-4090: - Summary: Memory leak in snappy-java 1.1.1.4/5 Key: SPARK-4090 URL: https://issues.apache.org/jira/browse/SPARK-4090 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4090) Memory leak in snappy-java 1.1.1.4/5

[ https://issues.apache.org/jira/browse/SPARK-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-4090: -- Attachment: screenshot-12.png Memory leak in snappy-java 1.1.1.4/5

[jira] [Commented] (SPARK-4090) Memory leak in snappy-java 1.1.1.4/5

[ https://issues.apache.org/jira/browse/SPARK-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184423#comment-14184423 ] Josh Rosen commented on SPARK-4090: --- I rolled back earlier today, so the build should be

[jira] [Resolved] (SPARK-4090) Memory leak in snappy-java 1.1.1.4/5

[ https://issues.apache.org/jira/browse/SPARK-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-4090. --- Resolution: Fixed Memory leak in snappy-java 1.1.1.4/5

[jira] [Commented] (SPARK-1473) Feature selection for high dimensional datasets

[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184453#comment-14184453 ] sam commented on SPARK-1473: [~gbr...@cs.man.ac.uk] Thanks for taking the time to respond to

[jira] [Commented] (SPARK-1473) Feature selection for high dimensional datasets

[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184457#comment-14184457 ] David Martinez Rego commented on SPARK-1473: Dear Sam, Thank you for the

[jira] [Created] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

Cheng Lian created SPARK-4091: - Summary: Occasionally spark.local.dir can be deleted twice and causes test failure Key: SPARK-4091 URL: https://issues.apache.org/jira/browse/SPARK-4091 Project: Spark

[jira] [Updated] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

[ https://issues.apache.org/jira/browse/SPARK-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4091: -- Description: By persisting an arbitrary RDD with storage level {{MEMORY_AND_DISK}}, Spark may

[jira] [Commented] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

[ https://issues.apache.org/jira/browse/SPARK-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184560#comment-14184560 ] Apache Spark commented on SPARK-4091: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-4090) Memory leak in snappy-java 1.1.1.4/5

[ https://issues.apache.org/jira/browse/SPARK-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184562#comment-14184562 ] Davies Liu commented on SPARK-4090: --- [~joshrosen] 1.1.1.6 is released. Memory leak in

[jira] [Updated] (SPARK-4064) NioBlockTransferService should deal with empty messages correctly

[ https://issues.apache.org/jira/browse/SPARK-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4064: --- Summary: NioBlockTransferService should deal with empty messages correctly (was: If we

[jira] [Commented] (SPARK-3655) Secondary sort

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184565#comment-14184565 ] Patrick Wendell commented on SPARK-3655: Okay, sounds good. Secondary sort

[jira] [Updated] (SPARK-4056) Upgrade snappy-java to 1.1.1.5

[ https://issues.apache.org/jira/browse/SPARK-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4056: --- Component/s: Spark Core Upgrade snappy-java to 1.1.1.5 --

[jira] [Commented] (SPARK-4056) Upgrade snappy-java to 1.1.1.5

[ https://issues.apache.org/jira/browse/SPARK-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184570#comment-14184570 ] Josh Rosen commented on SPARK-4056: --- We reverted the 1.1.5 upgrade after discovering

[jira] [Updated] (SPARK-4085) Job will fail if a shuffle file that's read locally gets deleted

[ https://issues.apache.org/jira/browse/SPARK-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4085: --- Component/s: Spark Core Job will fail if a shuffle file that's read locally gets deleted

[jira] [Updated] (SPARK-2760) Caching tables from multiple databases does not work

[ https://issues.apache.org/jira/browse/SPARK-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2760: --- Component/s: SQL Caching tables from multiple databases does not work

[jira] [Commented] (SPARK-3655) Support sorting of values in addition to keys (i.e. secondary sort)

[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184575#comment-14184575 ] koert kuipers commented on SPARK-3655: -- can you assign to me? i will have 2 pullreq

[jira] [Updated] (SPARK-3917) Compress data before network transfer

[ https://issues.apache.org/jira/browse/SPARK-3917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3917: --- Priority: Major (was: Critical) Compress data before network transfer

[jira] [Commented] (SPARK-2532) Fix issues with consolidated shuffle

[ https://issues.apache.org/jira/browse/SPARK-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184576#comment-14184576 ] Patrick Wendell commented on SPARK-2532: Hey [~matei] - you created some sub-tasks

[jira] [Comment Edited] (SPARK-3962) Mark spark dependency as provided in external libraries

[ https://issues.apache.org/jira/browse/SPARK-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184580#comment-14184580 ] Patrick Wendell edited comment on SPARK-3962 at 10/26/14 6:11 PM:

[jira] [Updated] (SPARK-3962) Mark spark dependency as provided in external libraries

[ https://issues.apache.org/jira/browse/SPARK-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3962: --- Assignee: Prashant Sharma Mark spark dependency as provided in external libraries

[jira] [Commented] (SPARK-3962) Mark spark dependency as provided in external libraries

[ https://issues.apache.org/jira/browse/SPARK-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184580#comment-14184580 ] Patrick Wendell commented on SPARK-3962: [~prashant_] can you take a crack at

[jira] [Resolved] (SPARK-1758) failing test org.apache.spark.JavaAPISuite.wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-1758. --- Resolution: Cannot Reproduce Resolving this as Cannot Reproduce for now, since I haven't observed

[jira] [Resolved] (SPARK-2527) incorrect persistence level shown in Spark UI after repersisting

[ https://issues.apache.org/jira/browse/SPARK-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2527. --- Resolution: Cannot Reproduce Assignee: Josh Rosen I think that this was fixed in either 1.1 or

[jira] [Updated] (SPARK-2527) incorrect persistence level shown in Spark UI after repersisting

[ https://issues.apache.org/jira/browse/SPARK-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2527: -- Fix Version/s: 1.2.0 incorrect persistence level shown in Spark UI after repersisting

[jira] [Commented] (SPARK-2105) SparkUI doesn't remove active stages that failed

[ https://issues.apache.org/jira/browse/SPARK-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184613#comment-14184613 ] Josh Rosen commented on SPARK-2105: --- I tried and failed to reproduce this:

[jira] [Resolved] (SPARK-3274) Spark Streaming Java API reports java.lang.ClassCastException when calling collectAsMap on JavaPairDStream

[ https://issues.apache.org/jira/browse/SPARK-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3274. --- Resolution: Invalid Spark Streaming Java API reports java.lang.ClassCastException when calling

[jira] [Commented] (SPARK-2105) SparkUI doesn't remove active stages that failed

[ https://issues.apache.org/jira/browse/SPARK-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184620#comment-14184620 ] Andrew Or commented on SPARK-2105: -- Hey Josh I think this was fixed by this commit:

[jira] [Commented] (SPARK-2811) update algebird to 0.8.1

[ https://issues.apache.org/jira/browse/SPARK-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184627#comment-14184627 ] Apache Spark commented on SPARK-2811: - User 'adampingel' has created a pull request

[jira] [Commented] (SPARK-2811) update algebird to 0.8.1

[ https://issues.apache.org/jira/browse/SPARK-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184635#comment-14184635 ] Adam Pingel commented on SPARK-2811: This seemed like an easy first way to contribute

[jira] [Created] (SPARK-4092) Input metrics don't work for coalesce()'d RDD's

Patrick Wendell created SPARK-4092: -- Summary: Input metrics don't work for coalesce()'d RDD's Key: SPARK-4092 URL: https://issues.apache.org/jira/browse/SPARK-4092 Project: Spark Issue

[jira] [Commented] (SPARK-3960) We can apply unary minus only to literal.

[ https://issues.apache.org/jira/browse/SPARK-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184649#comment-14184649 ] Apache Spark commented on SPARK-3960: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-799) Windows versions of the deploy scripts

[ https://issues.apache.org/jira/browse/SPARK-799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184663#comment-14184663 ] Andrew Tweddle commented on SPARK-799: -- Powershell is the modern Microsoft shell for

[jira] [Resolved] (SPARK-3925) Do not consider the ordering of qualifiers during comparison

[ https://issues.apache.org/jira/browse/SPARK-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3925. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2783

[jira] [Commented] (SPARK-3266) JavaDoubleRDD doesn't contain max()

[ https://issues.apache.org/jira/browse/SPARK-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184688#comment-14184688 ] Apache Spark commented on SPARK-3266: - User 'JoshRosen' has created a pull request for

[jira] [Resolved] (SPARK-3537) Statistics for cached RDDs

[ https://issues.apache.org/jira/browse/SPARK-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3537. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2860

[jira] [Commented] (SPARK-3266) JavaDoubleRDD doesn't contain max()

[ https://issues.apache.org/jira/browse/SPARK-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184696#comment-14184696 ] Josh Rosen commented on SPARK-3266: --- I've opened a new pull request which tries to work

[jira] [Resolved] (SPARK-3997) scalastyle should output the error location

[ https://issues.apache.org/jira/browse/SPARK-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3997. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2846

[jira] [Resolved] (SPARK-3953) Confusable variable name.

[ https://issues.apache.org/jira/browse/SPARK-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3953. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2807

[jira] [Resolved] (SPARK-4052) Use scala.collection.Map for pattern matching instead of using Predef.Map (it is scala.collection.immutable.Map)

[ https://issues.apache.org/jira/browse/SPARK-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4052. - Resolution: Fixed Fix Version/s: 1.2.0 Use scala.collection.Map for pattern

[jira] [Resolved] (SPARK-4068) NPE in jsonRDD schema inference

[ https://issues.apache.org/jira/browse/SPARK-4068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4068. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2918

[jira] [Commented] (SPARK-3266) JavaDoubleRDD doesn't contain max()

[ https://issues.apache.org/jira/browse/SPARK-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184704#comment-14184704 ] Patrick Wendell commented on SPARK-3266: I think it sort of depends how many

[jira] [Resolved] (SPARK-3483) Special chars in column names

[ https://issues.apache.org/jira/browse/SPARK-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3483. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2927

[jira] [Resolved] (SPARK-3959) SqlParser fails to parse literal -9223372036854775808 (Long.MinValue).

[ https://issues.apache.org/jira/browse/SPARK-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3959. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2816

[jira] [Resolved] (SPARK-4061) We cannot use EOL character in the operand of LIKE predicate.

[ https://issues.apache.org/jira/browse/SPARK-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4061. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2908

[jira] [Resolved] (SPARK-4042) append columns ids and names before broadcast

[ https://issues.apache.org/jira/browse/SPARK-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4042. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2885

[jira] [Resolved] (SPARK-2760) Caching tables from multiple databases does not work

[ https://issues.apache.org/jira/browse/SPARK-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2760. - Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Michael Armbrust This

[jira] [Updated] (SPARK-3970) Remove duplicate removal of local dirs

[ https://issues.apache.org/jira/browse/SPARK-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3970: - Affects Version/s: 1.1.0 Remove duplicate removal of local dirs --

[jira] [Updated] (SPARK-3970) Remove duplicate removal of local dirs

[ https://issues.apache.org/jira/browse/SPARK-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3970: - Assignee: Liang-Chi Hsieh Remove duplicate removal of local dirs --

[jira] [Closed] (SPARK-3970) Remove duplicate removal of local dirs

[ https://issues.apache.org/jira/browse/SPARK-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3970. Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 Remove duplicate removal

[jira] [Created] (SPARK-4093) Simplify the unwrap/wrap between HiveUDFs

Cheng Hao created SPARK-4093: Summary: Simplify the unwrap/wrap between HiveUDFs Key: SPARK-4093 URL: https://issues.apache.org/jira/browse/SPARK-4093 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

[ https://issues.apache.org/jira/browse/SPARK-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184741#comment-14184741 ] Cheng Lian commented on SPARK-4091: --- Yes, thanks [~joshrosen], closing this.

[jira] [Closed] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

[ https://issues.apache.org/jira/browse/SPARK-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian closed SPARK-4091. - Resolution: Duplicate Occasionally spark.local.dir can be deleted twice and causes test failure

[jira] [Commented] (SPARK-3838) Python code example for Word2Vec in user guide

[ https://issues.apache.org/jira/browse/SPARK-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184750#comment-14184750 ] Anant Daksh Asthana commented on SPARK-3838: Pull request for resolution can

[jira] [Commented] (SPARK-3838) Python code example for Word2Vec in user guide

[ https://issues.apache.org/jira/browse/SPARK-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184749#comment-14184749 ] Apache Spark commented on SPARK-3838: - User 'anantasty' has created a pull request for

[jira] [Commented] (SPARK-2396) Spark EC2 scripts fail when trying to log in to EC2 instances

[ https://issues.apache.org/jira/browse/SPARK-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184803#comment-14184803 ] Anant Daksh Asthana commented on SPARK-2396: Seems like a python issue on your

[jira] [Comment Edited] (SPARK-3988) Public API for DateType support

[ https://issues.apache.org/jira/browse/SPARK-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14180933#comment-14180933 ] Adrian Wang edited comment on SPARK-3988 at 10/27/14 4:28 AM: --

[jira] [Commented] (SPARK-2336) Approximate k-NN Models for MLLib

[ https://issues.apache.org/jira/browse/SPARK-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184848#comment-14184848 ] Ashutosh Trivedi commented on SPARK-2336: - Is anybody already working on it ? I