[jira] [Commented] (SPARK-3995) [PYSPARK] PySpark's sample methods do not work with NumPy 1.9

2014-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179657#comment-14179657 ] Apache Spark commented on SPARK-3995: - User 'freeman-lab' has created a pull request f

[jira] [Commented] (SPARK-4037) NPE in JDBC server when calling SET

2014-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179651#comment-14179651 ] Apache Spark commented on SPARK-4037: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-2926) Add MR-style (merge-sort) SortShuffleReader for sort-based shuffle

2014-10-21 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179631#comment-14179631 ] Sandy Ryza commented on SPARK-2926: --- [~rxin] did you ever get a chance to try this out?

[jira] [Commented] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179625#comment-14179625 ] Josh Rosen commented on SPARK-4006: --- Thanks for the bug report + patch! I'd like to see

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4006: -- Target Version/s: 1.1.1, 1.2.0 (was: 1.2.0) > Spark Driver crashes whenever an Executor is registered t

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4006: -- Affects Version/s: 1.2.0 > Spark Driver crashes whenever an Executor is registered twice > -

[jira] [Updated] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4006: -- Description: This is a huge robustness issue for us (Taboola), in mission critical , time sensitive (re

[jira] [Commented] (SPARK-4044) Thriftserver fails to start when JAVA_HOME points to JRE instead of JDK

2014-10-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179620#comment-14179620 ] Sean Owen commented on SPARK-4044: -- How about using {{unzip -l}} to probe the contents of

[jira] [Resolved] (SPARK-4018) RDD.reduce failing with java.lang.ClassCastException: org.apache.spark.SparkContext$$anonfun$26 cannot be cast to scala.Function2

2014-10-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4018. -- Resolution: Not a Problem > RDD.reduce failing with java.lang.ClassCastException: > org.apache.spark.Sp

[jira] [Commented] (SPARK-4037) NPE in JDBC server when calling SET

2014-10-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179598#comment-14179598 ] Cheng Lian commented on SPARK-4037: --- I think we can safely remove the global singleton S

[jira] [Created] (SPARK-4044) Thriftserver fails to start when JAVA_HOME points to JRE instead of JDK

2014-10-21 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-4044: - Summary: Thriftserver fails to start when JAVA_HOME points to JRE instead of JDK Key: SPARK-4044 URL: https://issues.apache.org/jira/browse/SPARK-4044 Project: Spark

[jira] [Comment Edited] (SPARK-3987) NNLS generates incorrect result

2014-10-21 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179585#comment-14179585 ] Shuo Xiang edited comment on SPARK-3987 at 10/22/14 5:09 AM: -

[jira] [Comment Edited] (SPARK-3987) NNLS generates incorrect result

2014-10-21 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179585#comment-14179585 ] Shuo Xiang edited comment on SPARK-3987 at 10/22/14 5:08 AM: -

[jira] [Commented] (SPARK-3987) NNLS generates incorrect result

2014-10-21 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179585#comment-14179585 ] Shuo Xiang commented on SPARK-3987: --- [~debasish83] By using a finer config (change 1e-6

[jira] [Updated] (SPARK-4042) append columns ids and names before broadcast

2014-10-21 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-4042: --- Description: appended columns ids and names will not broadcast because we append them after create table read

[jira] [Updated] (SPARK-4042) append columns ids and names before broadcast

2014-10-21 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-4042: --- Description: appended columns ids and names will not broadcast because we append them after create table read

[jira] [Updated] (SPARK-4042) append columns ids and names before broadcast

2014-10-21 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-4042: --- Description: appended columns ids and names will not broadcast because we append them after creating table re

[jira] [Updated] (SPARK-3987) NNLS generates incorrect result

2014-10-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3987: - Assignee: Shuo Xiang > NNLS generates incorrect result > --- > >

[jira] [Updated] (SPARK-4042) append columns ids and names before broadcast

2014-10-21 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-4042: --- Description: appended columns ids and names will not broadcast because we append them after create table reade

[jira] [Resolved] (SPARK-1813) Add a utility to SparkConf that makes using Kryo really easy

2014-10-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1813. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Sandy Ryza Fixed in https:/

[jira] [Issue Comment Deleted] (SPARK-3987) NNLS generates incorrect result

2014-10-21 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-3987: -- Comment: was deleted (was: [~debasish83] I'm wondering if P2 and q2 are the **ata** and **atb** matrice

[jira] [Updated] (SPARK-4043) Add a flag for stopping threads of cancelled tasks if Thread.interrupt doesn't kill them

2014-10-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4043: --- Component/s: Spark Core > Add a flag for stopping threads of cancelled tasks if Thread.interru

[jira] [Updated] (SPARK-4040) calling count() on RDD's emitted from a DStream blocks forEachRDD progress.

2014-10-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4040: --- Component/s: Streaming > calling count() on RDD's emitted from a DStream blocks forEachRDD pro

[jira] [Comment Edited] (SPARK-3987) NNLS generates incorrect result

2014-10-21 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179316#comment-14179316 ] Shuo Xiang edited comment on SPARK-3987 at 10/22/14 4:32 AM: -

[jira] [Updated] (SPARK-4033) Integer overflow when SparkPi is called with more than 25000 slices

2014-10-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4033: --- Summary: Integer overflow when SparkPi is called with more than 25000 slices (was: Input of t

[jira] [Updated] (SPARK-4038) Outlier Detection Algorithm for MLlib

2014-10-21 Thread Ashutosh Trivedi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Trivedi updated SPARK-4038: Description: The aim of this JIRA is to discuss about which parallel outlier detection algo

[jira] [Commented] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

2014-10-21 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179510#comment-14179510 ] Jongyoul Lee commented on SPARK-3955: - [~srowen], yes, This is about dependency leakag

[jira] [Created] (SPARK-4043) Add a flag for stopping threads of cancelled tasks if Thread.interrupt doesn't kill them

2014-10-21 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-4043: Summary: Add a flag for stopping threads of cancelled tasks if Thread.interrupt doesn't kill them Key: SPARK-4043 URL: https://issues.apache.org/jira/browse/SPARK-4043

[jira] [Updated] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

2014-10-21 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-4036: --- Description: Conditional random fields (CRFs) are a class of statistical modelling method often appli

[jira] [Commented] (SPARK-4042) append columns ids and names before broadcast

2014-10-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179469#comment-14179469 ] Yin Huai commented on SPARK-4042: - Can you add an explanation about the problem in Descrip

[jira] [Updated] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

2014-10-21 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-4036: --- Summary: Add Conditional Random Fields (CRF) algorithm to Spark MLlib (was: Conditional Random Fields

[jira] [Updated] (SPARK-4042) append columns ids and names before broadcast

2014-10-21 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-4042: Fix Version/s: (was: 1.1.1) > append columns ids and names before broadcast > --

[jira] [Commented] (SPARK-4018) RDD.reduce failing with java.lang.ClassCastException: org.apache.spark.SparkContext$$anonfun$26 cannot be cast to scala.Function2

2014-10-21 Thread Haithem Turki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179461#comment-14179461 ] Haithem Turki commented on SPARK-4018: -- Hey Sean, you're totally right. I was using S

[jira] [Updated] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3426: -- Description: We have the following configs: {code} spark.shuffle.compress spark.shuffle.spill.compress

[jira] [Commented] (SPARK-4002) JavaKafkaStreamSuite.testKafkaStream fails on OSX

2014-10-21 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179445#comment-14179445 ] Saisai Shao commented on SPARK-4002: Hi Ryan, I've tested using Maven with your hadoop

[jira] [Updated] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3426: -- Affects Version/s: 1.2.0 > Sort-based shuffle compression behavior is inconsistent > ---

[jira] [Issue Comment Deleted] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2014-10-21 Thread Tal Sliwowicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tal Sliwowicz updated SPARK-4006: - Comment: was deleted (was: Another pull request - this time on master - https://github.com/apache

[jira] [Commented] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179355#comment-14179355 ] Josh Rosen commented on SPARK-3426: --- Based on the discussion in that PR, it sounds folks

[jira] [Commented] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2014-10-21 Thread Tal Sliwowicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179351#comment-14179351 ] Tal Sliwowicz commented on SPARK-4006: -- Another pull request - this time on master -

[jira] [Commented] (SPARK-4006) Spark Driver crashes whenever an Executor is registered twice

2014-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179347#comment-14179347 ] Apache Spark commented on SPARK-4006: - User 'tsliwowicz' has created a pull request fo

[jira] [Commented] (SPARK-4021) Issues observed after upgrading Jenkins to JDK7u71

2014-10-21 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179346#comment-14179346 ] shane knapp commented on SPARK-4021: ok, i can believe that it doesn't have anything t

[jira] [Commented] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179336#comment-14179336 ] Josh Rosen commented on SPARK-3426: --- I've edited this issue to list the actual exception

[jira] [Updated] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3426: -- Description: We have the following configs: {code} spark.shuffle.compress spark.shuffle.spill.compress

[jira] [Assigned] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-3426: - Assignee: Josh Rosen (was: Andrew Or) > Sort-based shuffle compression behavior is inconsistent

[jira] [Commented] (SPARK-3987) NNLS generates incorrect result

2014-10-21 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179316#comment-14179316 ] Shuo Xiang commented on SPARK-3987: --- [~debasish83] Just wondering did you use q2 or -q2

[jira] [Resolved] (SPARK-3517) mapPartitions is not correct clearing up the closure

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3517. --- Resolution: Incomplete Resolving this as "Incomplete" for now, since witgo was unable to reproduce th

[jira] [Commented] (SPARK-4037) NPE in JDBC server when calling SET

2014-10-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179305#comment-14179305 ] Cheng Lian commented on SPARK-4037: --- This is a regression of SPARK-2814, added in SPARK-

[jira] [Commented] (SPARK-2201) Improve FlumeInputDStream's stability and make it scalable

2014-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179292#comment-14179292 ] Apache Spark commented on SPARK-2201: - User 'joyyoj' has created a pull request for th

[jira] [Commented] (SPARK-4021) Issues observed after upgrading Jenkins to JDK7u71

2014-10-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179283#comment-14179283 ] Sean Owen commented on SPARK-4021: -- [~shaneknapp] I don't think this can have anything to

[jira] [Closed] (SPARK-3819) Jenkins should compile Spark against multiple versions of Hadoop

2014-10-21 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah closed SPARK-3819. - Resolution: Won't Fix Not much activity for awhile. Doesn't seem that important anyways, the cases when w

[jira] [Closed] (SPARK-4021) Issues observed after upgrading Jenkins to JDK7u71

2014-10-21 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp closed SPARK-4021. -- Resolution: Not a Problem > Issues observed after upgrading Jenkins to JDK7u71 > ---

[jira] [Commented] (SPARK-4021) Issues observed after upgrading Jenkins to JDK7u71

2014-10-21 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179272#comment-14179272 ] shane knapp commented on SPARK-4021: take a look at this: https://issues.jenkins-ci.or

[jira] [Resolved] (SPARK-3568) Add metrics for ranking algorithms

2014-10-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3568. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2667 [https://githu

[jira] [Commented] (SPARK-3655) Secondary sort

2014-10-21 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179252#comment-14179252 ] koert kuipers commented on SPARK-3655: -- hey matei, i was referring to the partition b

[jira] [Commented] (SPARK-3740) Use a compressed bitmap to track zero sized blocks in HighlyCompressedMapStatus

2014-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179176#comment-14179176 ] Apache Spark commented on SPARK-3740: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-4021) Issues observed after upgrading Jenkins to JDK7u71

2014-10-21 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179143#comment-14179143 ] shane knapp commented on SPARK-4021: oh yeah, in steps 0 and 2 (from above), JAVA_HOME

[jira] [Commented] (SPARK-4042) append columns ids and names before broadcast

2014-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179133#comment-14179133 ] Apache Spark commented on SPARK-4042: - User 'scwf' has created a pull request for this

[jira] [Created] (SPARK-4042) append columns ids and names before broadcast

2014-10-21 Thread wangfei (JIRA)
wangfei created SPARK-4042: -- Summary: append columns ids and names before broadcast Key: SPARK-4042 URL: https://issues.apache.org/jira/browse/SPARK-4042 Project: Spark Issue Type: Bug Com

[jira] [Resolved] (SPARK-3770) The userFeatures RDD from MatrixFactorizationModel isn't accessible from the python bindings

2014-10-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3770. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2636 [https://githu

[jira] [Updated] (SPARK-3770) The userFeatures RDD from MatrixFactorizationModel isn't accessible from the python bindings

2014-10-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3770: - Assignee: Michelangelo D'Agostino > The userFeatures RDD from MatrixFactorizationModel isn't acces

[jira] [Commented] (SPARK-4041) convert attributes names in table scan lowercase when compare with relation attributes

2014-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179122#comment-14179122 ] Apache Spark commented on SPARK-4041: - User 'scwf' has created a pull request for this

[jira] [Created] (SPARK-4041) convert attributes names in table scan lowercase when compare with relation attributes

2014-10-21 Thread wangfei (JIRA)
wangfei created SPARK-4041: -- Summary: convert attributes names in table scan lowercase when compare with relation attributes Key: SPARK-4041 URL: https://issues.apache.org/jira/browse/SPARK-4041 Project: Spa

[jira] [Updated] (SPARK-4040) calling count() on RDD's emitted from a DStream blocks forEachRDD progress.

2014-10-21 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-4040: Description: Please note that Im somewhat new to spark streaming's API, and am not a spark expert - so I've

[jira] [Updated] (SPARK-4040) calling count() on RDD's emitted from a DStream blocks forEachRDD progress.

2014-10-21 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-4040: Description: CC [~rnowling] [~willbenton] It appears that in a DStream context, a call to {{MappedRDD.c

[jira] [Updated] (SPARK-4040) calling count() on RDD's emitted from a DStream blocks forEachRDD progress.

2014-10-21 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-4040: Description: CC [~rnowling] [~willbenton] It appears that in a DStream context, a call to {{MappedRDD.c

[jira] [Created] (SPARK-4040) calling count() on RDD's emitted from a DStream blocks forEachRDD progress.

2014-10-21 Thread jay vyas (JIRA)
jay vyas created SPARK-4040: --- Summary: calling count() on RDD's emitted from a DStream blocks forEachRDD progress. Key: SPARK-4040 URL: https://issues.apache.org/jira/browse/SPARK-4040 Project: Spark

[jira] [Commented] (SPARK-4021) Issues observed after upgrading Jenkins to JDK7u71

2014-10-21 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179072#comment-14179072 ] shane knapp commented on SPARK-4021: ok, so i was able to recreate what happened yeste

[jira] [Updated] (SPARK-4026) Write ahead log management

2014-10-21 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-4026: - Summary: Write ahead log management (was: Write ahead log to synchronously write received data to

[jira] [Updated] (SPARK-3994) countByKey / countByValue do not go through Aggregator

2014-10-21 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3994: - Affects Version/s: 1.1.0 > countByKey / countByValue do not go through Aggregator > --

[jira] [Updated] (SPARK-3994) countByKey / countByValue do not go through Aggregator

2014-10-21 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3994: - Affects Version/s: (was: 1.1.0) 1.0.0 > countByKey / countByValue do not go thr

[jira] [Closed] (SPARK-3994) countByKey / countByValue do not go through Aggregator

2014-10-21 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3994. Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 > countByKey / countByValue

[jira] [Commented] (SPARK-4026) Write ahead log to synchronously write received data to HDFS and recover on driver failure

2014-10-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178967#comment-14178967 ] Apache Spark commented on SPARK-4026: - User 'tdas' has created a pull request for this

[jira] [Created] (SPARK-4039) KMeans support HashingTF vectors

2014-10-21 Thread Antoine Amend (JIRA)
Antoine Amend created SPARK-4039: Summary: KMeans support HashingTF vectors Key: SPARK-4039 URL: https://issues.apache.org/jira/browse/SPARK-4039 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-4038) Outlier Detection Algorithm for MLlib

2014-10-21 Thread Ashutosh Trivedi (JIRA)
Ashutosh Trivedi created SPARK-4038: --- Summary: Outlier Detection Algorithm for MLlib Key: SPARK-4038 URL: https://issues.apache.org/jira/browse/SPARK-4038 Project: Spark Issue Type: New Fea

[jira] [Comment Edited] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

2014-10-21 Thread Pat Ferrel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178879#comment-14178879 ] Pat Ferrel edited comment on SPARK-2292 at 10/21/14 7:21 PM: -

[jira] [Commented] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

2014-10-21 Thread Pat Ferrel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178879#comment-14178879 ] Pat Ferrel commented on SPARK-2292: --- If this is related to SPARK-2075 the answer is: If

[jira] [Commented] (SPARK-3466) Limit size of results that a driver collects for each action

2014-10-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178857#comment-14178857 ] Patrick Wendell commented on SPARK-3466: I spoke with Matt today and I'm re-assign

[jira] [Updated] (SPARK-3466) Limit size of results that a driver collects for each action

2014-10-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3466: --- Assignee: Davies Liu (was: Matthew Cheah) > Limit size of results that a driver collects for

[jira] [Commented] (SPARK-2075) Anonymous classes are missing from Spark distribution

2014-10-21 Thread Pat Ferrel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178852#comment-14178852 ] Pat Ferrel commented on SPARK-2075: --- OK solved. The WAG worked. Instead of 'mvn package

[jira] [Resolved] (SPARK-4020) Failed executor not properly removed if it has not run tasks

2014-10-21 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-4020. --- Resolution: Fixed > Failed executor not properly removed if it has not run tasks > ---

[jira] [Updated] (SPARK-4020) Failed executor not properly removed if it has not run tasks

2014-10-21 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-4020: -- Fix Version/s: 1.2.0 Fixed by https://github.com/apache/spark/commit/61ca7742d21dd66f5a7b3bb826

[jira] [Commented] (SPARK-3254) Streaming K-Means

2014-10-21 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178825#comment-14178825 ] Reza Zadeh commented on SPARK-3254: --- Will this make it to 1.2? > Streaming K-Means > --

[jira] [Assigned] (SPARK-3740) Use a compressed bitmap to track zero sized blocks in HighlyCompressedMapStatus

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-3740: - Assignee: Josh Rosen (was: Liquan Pei) > Use a compressed bitmap to track zero sized blocks in

[jira] [Commented] (SPARK-3740) Use a compressed bitmap to track zero sized blocks in HighlyCompressedMapStatus

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178807#comment-14178807 ] Josh Rosen commented on SPARK-3740: --- This isn't just an optimization; it's required for

[jira] [Commented] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

2014-10-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178798#comment-14178798 ] Joseph K. Bradley commented on SPARK-3717: -- Hi Sumanth, it would be great to get

[jira] [Created] (SPARK-4037) NPE in JDBC server when calling SET

2014-10-21 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-4037: --- Summary: NPE in JDBC server when calling SET Key: SPARK-4037 URL: https://issues.apache.org/jira/browse/SPARK-4037 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2075) Anonymous classes are missing from Spark distribution

2014-10-21 Thread Pat Ferrel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178720#comment-14178720 ] Pat Ferrel commented on SPARK-2075: --- trying mvn install instead of the documented mvn pa

[jira] [Resolved] (SPARK-4035) Wrong format specifier in BlockerManager.scala

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4035. --- Resolution: Fixed Fix Version/s: (was: 1.1.1) Issue resolved by pull request 2875 [https://

[jira] [Commented] (SPARK-2075) Anonymous classes are missing from Spark distribution

2014-10-21 Thread Pat Ferrel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178688#comment-14178688 ] Pat Ferrel commented on SPARK-2075: --- Oops, right. But the function name is being constru

[jira] [Resolved] (SPARK-4015) Documentation in the streaming context references non-existent function

2014-10-21 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4015. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2861 [https://github.com/

[jira] [Resolved] (SPARK-4023) PySpark's stat.Statistics is broken

2014-10-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-4023. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2870 [https://githu

[jira] [Commented] (SPARK-3359) `sbt/sbt unidoc` doesn't work with Java 8

2014-10-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178583#comment-14178583 ] Sean Owen commented on SPARK-3359: -- Sure, but it can be fixed right now too. I tried to f

[jira] [Commented] (SPARK-3299) [SQL] Public API in SQLContext to list tables

2014-10-21 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178572#comment-14178572 ] Bill Bejeck commented on SPARK-3299: This change is a little more involved as it requi

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-21 Thread Dag Liodden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178554#comment-14178554 ] Dag Liodden commented on SPARK-3174: Hey guys, glad to see some progress on this! I

[jira] [Commented] (SPARK-4030) `destroy` method in Broadcast should be public

2014-10-21 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178532#comment-14178532 ] Shivaram Venkataraman commented on SPARK-4030: -- Yes - there is a bunch of log

[jira] [Created] (SPARK-4036) Conditional Random Fields (CRF) atop of spark in MLlib

2014-10-21 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-4036: -- Summary: Conditional Random Fields (CRF) atop of spark in MLlib Key: SPARK-4036 URL: https://issues.apache.org/jira/browse/SPARK-4036 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2075) Anonymous classes are missing from Spark distribution

2014-10-21 Thread Iulian Dragos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178350#comment-14178350 ] Iulian Dragos commented on SPARK-2075: -- The name of the missing class is {{org.apach

[jira] [Commented] (SPARK-3359) `sbt/sbt unidoc` doesn't work with Java 8

2014-10-21 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178316#comment-14178316 ] Prashant Sharma commented on SPARK-3359: I guess this will happen along with scala

[jira] [Commented] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

2014-10-21 Thread Jeroen van Wilgenburg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178313#comment-14178313 ] Jeroen van Wilgenburg commented on SPARK-3955: -- Is is indeed a dependency lea

[jira] [Commented] (SPARK-3955) Different versions between jackson-mapper-asl and jackson-core-asl

2014-10-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178294#comment-14178294 ] Sean Owen commented on SPARK-3955: -- I think the issue is matching the version used by Had

  1   2   >