[jira] [Created] (SPARK-1458) Add programmatic way to determine Spark version

2014-04-09 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-1458: --- Summary: Add programmatic way to determine Spark version Key: SPARK-1458 URL: https://issues.apache.org/jira/browse/SPARK-1458 Project: Spark Issue

[jira] [Updated] (SPARK-2008) Enhance spark-ec2 to be able to add and remove slaves to an existing cluster

2014-06-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2008: Priority: Minor (was: Major) Enhance spark-ec2 to be able to add and remove slaves to an

[jira] [Commented] (SPARK-2065) Have spark-ec2 set EC2 instance names

2014-06-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14020893#comment-14020893 ] Nicholas Chammas commented on SPARK-2065: - Sure, I'd love to. Have spark-ec2 set

[jira] [Commented] (SPARK-2065) Have spark-ec2 set EC2 instance names

2014-06-10 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14027199#comment-14027199 ] Nicholas Chammas commented on SPARK-2065: - I've submitted [PR

[jira] [Created] (SPARK-2115) Stage kill link is too close to stage details link

2014-06-11 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2115: --- Summary: Stage kill link is too close to stage details link Key: SPARK-2115 URL: https://issues.apache.org/jira/browse/SPARK-2115 Project: Spark Issue

[jira] [Updated] (SPARK-2115) Stage kill link is too close to stage details link

2014-06-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2115: Attachment: Accident-prone kill link.png Safe and user-friendly kill

[jira] [Created] (SPARK-2130) Clarify PySpark docs for RDD.getStorageLevel

2014-06-12 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2130: --- Summary: Clarify PySpark docs for RDD.getStorageLevel Key: SPARK-2130 URL: https://issues.apache.org/jira/browse/SPARK-2130 Project: Spark Issue Type:

[jira] [Created] (SPARK-2141) Add sc.getPersistentRDDs() to PySpark

2014-06-13 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2141: --- Summary: Add sc.getPersistentRDDs() to PySpark Key: SPARK-2141 URL: https://issues.apache.org/jira/browse/SPARK-2141 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-2141) Add sc.getPersistentRDDs() to PySpark

2014-06-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2141: Description: PySpark does not appear to have {{sc.getPersistentRDDs()}}. (was: PySpark

[jira] [Commented] (SPARK-1652) Fixes and improvements for spark-submit/configs

2014-06-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14039565#comment-14039565 ] Nicholas Chammas commented on SPARK-1652: - [~pwendell], is there currently a

[jira] [Created] (SPARK-2376) List values inside nested JSON objects cause java.lang.IllegalArgumentException

2014-07-05 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2376: --- Summary: List values inside nested JSON objects cause java.lang.IllegalArgumentException Key: SPARK-2376 URL: https://issues.apache.org/jira/browse/SPARK-2376

[jira] [Updated] (SPARK-2376) Selecting list values inside nested JSON objects raises java.lang.IllegalArgumentException

2014-07-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2376: Summary: Selecting list values inside nested JSON objects raises

[jira] [Updated] (SPARK-2376) Selecting list values inside nested JSON objects raises java.lang.IllegalArgumentException

2014-07-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2376: Description: Repro script for PySpark, deployed via {{spark-ec2}} at git revision

[jira] [Created] (SPARK-2377) Create a Python API for Spark Streaming

2014-07-05 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2377: --- Summary: Create a Python API for Spark Streaming Key: SPARK-2377 URL: https://issues.apache.org/jira/browse/SPARK-2377 Project: Spark Issue Type: New

[jira] [Created] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

2014-07-07 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2394: --- Summary: Make it easier to read LZO-compressed files from EC2 clusters Key: SPARK-2394 URL: https://issues.apache.org/jira/browse/SPARK-2394 Project: Spark

[jira] [Reopened] (SPARK-1458) Add programmatic way to determine Spark version

2014-07-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas reopened SPARK-1458: - Reopening this ticket. It seems that, while the Scala shell exposes the Spark version via

[jira] [Created] (SPARK-2463) Creating multiple StreamingContexts from shell generates duplicate Streaming tabs in UI

2014-07-12 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2463: --- Summary: Creating multiple StreamingContexts from shell generates duplicate Streaming tabs in UI Key: SPARK-2463 URL: https://issues.apache.org/jira/browse/SPARK-2463

[jira] [Commented] (SPARK-524) spark integration issue with Cloudera hadoop

2014-07-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060145#comment-14060145 ] Nicholas Chammas commented on SPARK-524: +1 for cleanup of issues that likely have

[jira] [Commented] (SPARK-2463) Creating multiple StreamingContexts from shell generates duplicate Streaming tabs in UI

2014-07-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062458#comment-14062458 ] Nicholas Chammas commented on SPARK-2463: - Ah, my apologies about the Trivial

[jira] [Created] (SPARK-2528) spark-ec2 security group permissions are too open

2014-07-16 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2528: --- Summary: spark-ec2 security group permissions are too open Key: SPARK-2528 URL: https://issues.apache.org/jira/browse/SPARK-2528 Project: Spark Issue

[jira] [Commented] (SPARK-2463) Creating multiple StreamingContexts from shell generates duplicate Streaming tabs in UI

2014-07-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064478#comment-14064478 ] Nicholas Chammas commented on SPARK-2463: - {quote} Yeah, as a first step, we

[jira] [Commented] (SPARK-2463) Creating multiple StreamingContexts from shell generates duplicate Streaming tabs in UI

2014-07-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064501#comment-14064501 ] Nicholas Chammas commented on SPARK-2463: - Ah OK, that's good to hear! Sorry,

[jira] [Updated] (SPARK-2528) spark-ec2 security group permissions are too open

2014-07-17 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2528: Description: {{spark-ec2}} configures EC2 security groups with ports [open to the world |

[jira] [Created] (SPARK-2560) Create Spark SQL syntax reference

2014-07-17 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2560: --- Summary: Create Spark SQL syntax reference Key: SPARK-2560 URL: https://issues.apache.org/jira/browse/SPARK-2560 Project: Spark Issue Type:

[jira] [Updated] (SPARK-2560) Create Spark SQL syntax reference

2014-07-17 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2560: Description: Does Spark SQL support {{LEN()}}? How about {{LIMIT}}? And what about {{MY

[jira] [Commented] (SPARK-1458) Expose sc.version in PySpark

2014-07-17 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065693#comment-14065693 ] Nicholas Chammas commented on SPARK-1458: - Perhaps that could also be some kind of

[jira] [Created] (SPARK-2602) sbt/sbt test steals window focus on OS X

2014-07-20 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2602: --- Summary: sbt/sbt test steals window focus on OS X Key: SPARK-2602 URL: https://issues.apache.org/jira/browse/SPARK-2602 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2602) sbt/sbt test steals window focus on OS X

2014-07-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068050#comment-14068050 ] Nicholas Chammas commented on SPARK-2602: - Ah, I'm on Java 6. Looking at [this

[jira] [Comment Edited] (SPARK-2602) sbt/sbt test steals window focus on OS X

2014-07-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068050#comment-14068050 ] Nicholas Chammas edited comment on SPARK-2602 at 7/20/14 9:24 PM:

[jira] [Commented] (SPARK-2602) sbt/sbt test steals window focus on OS X

2014-07-21 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069404#comment-14069404 ] Nicholas Chammas commented on SPARK-2602: - I upgraded to Java 7 and that resolved

[jira] [Created] (SPARK-2627) Check for PEP 8 compliance on all Python code in the Jenkins CI cycle

2014-07-22 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2627: --- Summary: Check for PEP 8 compliance on all Python code in the Jenkins CI cycle Key: SPARK-2627 URL: https://issues.apache.org/jira/browse/SPARK-2627 Project:

[jira] [Commented] (SPARK-2407) Implement SQL SUBSTR() directly in Catalyst

2014-07-25 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075122#comment-14075122 ] Nicholas Chammas commented on SPARK-2407: - Minor point: Shouldn't the issue type

[jira] [Commented] (SPARK-2154) Worker goes down.

2014-07-25 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075123#comment-14075123 ] Nicholas Chammas commented on SPARK-2154: - Can we edit the title of this issue to

[jira] [Commented] (SPARK-1458) Expose sc.version in PySpark

2014-07-25 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075137#comment-14075137 ] Nicholas Chammas commented on SPARK-1458: - Derp, I guess we

[jira] [Created] (SPARK-2697) Source Scala and Python shell banners from a single place

2014-07-25 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-2697: --- Summary: Source Scala and Python shell banners from a single place Key: SPARK-2697 URL: https://issues.apache.org/jira/browse/SPARK-2697 Project: Spark

[jira] [Updated] (SPARK-2797) SchemaRDDs don't support unpersist()

2014-08-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2797: Description: Looks like something simple got missed in the Java layer? {code} from

[jira] [Commented] (SPARK-1981) Add AWS Kinesis streaming support

2014-08-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084025#comment-14084025 ] Nicholas Chammas commented on SPARK-1981: - Word. Thanks for the clarification!

[jira] [Commented] (SPARK-2010) Support for nested data in PySpark SQL

2014-08-04 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085743#comment-14085743 ] Nicholas Chammas commented on SPARK-2010: - I just tried this on {{master}}:

[jira] [Commented] (SPARK-2010) Support for nested data in PySpark SQL

2014-08-04 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085807#comment-14085807 ] Nicholas Chammas commented on SPARK-2010: - Hmm, [it doesn't look like it's in the

[jira] [Comment Edited] (SPARK-2010) Support for nested data in PySpark SQL

2014-08-04 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085807#comment-14085807 ] Nicholas Chammas edited comment on SPARK-2010 at 8/5/14 5:06 AM:

[jira] [Commented] (SPARK-2627) Check for PEP 8 compliance on all Python code in the Jenkins CI cycle

2014-08-06 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14088167#comment-14088167 ] Nicholas Chammas commented on SPARK-2627: - [~rxin] - Shall I post the updates to

[jira] [Commented] (SPARK-2627) Check for PEP 8 compliance on all Python code in the Jenkins CI cycle

2014-08-06 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14088201#comment-14088201 ] Nicholas Chammas commented on SPARK-2627: - h4. [Contributing to

[jira] [Reopened] (SPARK-2627) Check for PEP 8 compliance on all Python code in the Jenkins CI cycle

2014-08-06 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas reopened SPARK-2627: - Reopening just to make sure that updating the Contributing guide stays on the radar.

[jira] [Created] (SPARK-3044) Create RSS feed for Spark News

2014-08-14 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3044: --- Summary: Create RSS feed for Spark News Key: SPARK-3044 URL: https://issues.apache.org/jira/browse/SPARK-3044 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099049#comment-14099049 ] Nicholas Chammas commented on SPARK-922: Josh, at the end of your updated script do

[jira] [Created] (SPARK-3076) Gracefully report build timeouts in Jenkins

2014-08-15 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3076: --- Summary: Gracefully report build timeouts in Jenkins Key: SPARK-3076 URL: https://issues.apache.org/jira/browse/SPARK-3076 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3044) Create RSS feed for Spark News

2014-08-25 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109677#comment-14109677 ] Nicholas Chammas commented on SPARK-3044: - Hi Michael, I don't know if the site

[jira] [Comment Edited] (SPARK-3044) Create RSS feed for Spark News

2014-08-26 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109677#comment-14109677 ] Nicholas Chammas edited comment on SPARK-3044 at 8/26/14 7:30 PM:

[jira] [Commented] (SPARK-3044) Create RSS feed for Spark News

2014-08-27 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113061#comment-14113061 ] Nicholas Chammas commented on SPARK-3044: - Hehe, I believe many people migrated

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14115636#comment-14115636 ] Nicholas Chammas commented on SPARK-922: FYI, I believe the line to install numpy

[jira] [Commented] (SPARK-3094) Support run pyspark in PyPy

2014-08-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116026#comment-14116026 ] Nicholas Chammas commented on SPARK-3094: - This is super cool. If [PyPy grows to

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116868#comment-14116868 ] Nicholas Chammas commented on SPARK-2870: - [~marmbrus], [~davies], [~yhuai] - We

[jira] [Created] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-: --- Summary: Large number of partitions causes OOM Key: SPARK- URL: https://issues.apache.org/jira/browse/SPARK- Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116895#comment-14116895 ] Nicholas Chammas commented on SPARK-: - Note: I have not yet confirmed that

[jira] [Updated] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-: Description: Here’s a repro for PySpark: {code} a = sc.parallelize([Nick, John, Bob]) a =

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116978#comment-14116978 ] Nicholas Chammas commented on SPARK-: - For the record, I got the OOM in a

[jira] [Comment Edited] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14116978#comment-14116978 ] Nicholas Chammas edited comment on SPARK- at 9/1/14 2:09 AM:

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-09-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117853#comment-14117853 ] Nicholas Chammas commented on SPARK-: - It looks like the default number of

[jira] [Comment Edited] (SPARK-3333) Large number of partitions causes OOM

2014-09-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117853#comment-14117853 ] Nicholas Chammas edited comment on SPARK- at 9/2/14 3:13 AM:

[jira] [Commented] (SPARK-1701) Inconsistent naming: slice or partition

2014-09-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117917#comment-14117917 ] Nicholas Chammas commented on SPARK-1701: - OK, so it sounds like action is: *

[jira] [Commented] (SPARK-1701) Inconsistent naming: slice or partition

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14118282#comment-14118282 ] Nicholas Chammas commented on SPARK-1701: - Oh absolutely; sorry, didn't mean to

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14118735#comment-14118735 ] Nicholas Chammas commented on SPARK-: - {quote} If we can't narrow it down in

[jira] [Created] (SPARK-3354) Add LENGTH and DATALENGTH functions to Spark SQL

2014-09-02 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3354: --- Summary: Add LENGTH and DATALENGTH functions to Spark SQL Key: SPARK-3354 URL: https://issues.apache.org/jira/browse/SPARK-3354 Project: Spark Issue

[jira] [Commented] (SPARK-3176) Implement 'POWER', 'ABS and 'LAST' for sql

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14118796#comment-14118796 ] Nicholas Chammas commented on SPARK-3176: - Reposting a [comment I made on the

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119119#comment-14119119 ] Nicholas Chammas commented on SPARK-: - Just to double check my results, I

[jira] [Updated] (SPARK-3333) Large number of partitions causes OOM

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-: Attachment: nick-1.0.2.driver.log.zip nick-1.1.0-rc3.driver.log.zip Here

[jira] [Commented] (SPARK-3358) PySpark worker fork()ing performance regression in m3.* / PVM instances

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119141#comment-14119141 ] Nicholas Chammas commented on SPARK-3358: - Josh, do you think this is related to

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119177#comment-14119177 ] Nicholas Chammas commented on SPARK-: - So I've repeated the tests with the

[jira] [Commented] (SPARK-2627) Check for PEP 8 compliance on all Python code in the Jenkins CI cycle

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119226#comment-14119226 ] Nicholas Chammas commented on SPARK-2627: - Note: We should cover the EC2 script

[jira] [Created] (SPARK-3361) Expand PEP 8 checks to include EC2 script and Python examples

2014-09-02 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3361: --- Summary: Expand PEP 8 checks to include EC2 script and Python examples Key: SPARK-3361 URL: https://issues.apache.org/jira/browse/SPARK-3361 Project: Spark

[jira] [Commented] (SPARK-2627) Check for PEP 8 compliance on all Python code in the Jenkins CI cycle

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119239#comment-14119239 ] Nicholas Chammas commented on SPARK-2627: - Okie doke: [SPARK-3361] Check for PEP

[jira] [Commented] (SPARK-3358) PySpark worker fork()ing performance regression in m3.* / PVM instances

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119276#comment-14119276 ] Nicholas Chammas commented on SPARK-3358: - Nit: Isn't it [PV and not

[jira] [Created] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-09-03 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3398: --- Summary: Have spark-ec2 intelligently wait for specific cluster states Key: SPARK-3398 URL: https://issues.apache.org/jira/browse/SPARK-3398 Project: Spark

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-09-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120957#comment-14120957 ] Nicholas Chammas commented on SPARK-3398: - Hey [~joshrosen], does this seem like a

[jira] [Updated] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-09-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3398: Description: {{spark-ec2}} currently has retry logic for when it tries to install stuff on

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator-Iterable is inconsistent with Scala's Iterator-Iterator

2014-09-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123203#comment-14123203 ] Nicholas Chammas commented on SPARK-3369: - {quote} The API change is unlikely to

[jira] [Created] (SPARK-3431) Parallelize execution of tests

2014-09-07 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3431: --- Summary: Parallelize execution of tests Key: SPARK-3431 URL: https://issues.apache.org/jira/browse/SPARK-3431 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3432) Fix logging of unit test execution time

2014-09-07 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3432: --- Summary: Fix logging of unit test execution time Key: SPARK-3432 URL: https://issues.apache.org/jira/browse/SPARK-3432 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2560) Create Spark SQL syntax reference

2014-09-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130464#comment-14130464 ] Nicholas Chammas commented on SPARK-2560: - [~marmbrus] - Would that be [this

[jira] [Commented] (SPARK-2560) Create Spark SQL syntax reference

2014-09-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130580#comment-14130580 ] Nicholas Chammas commented on SPARK-2560: - I guess that's good for starters, but I

[jira] [Commented] (SPARK-2045) Sort-based shuffle implementation

2014-09-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130974#comment-14130974 ] Nicholas Chammas commented on SPARK-2045: - The [1.1.0 release

[jira] [Created] (SPARK-3499) Create Spark-based distcp utility

2014-09-11 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3499: --- Summary: Create Spark-based distcp utility Key: SPARK-3499 URL: https://issues.apache.org/jira/browse/SPARK-3499 Project: Spark Issue Type: Wish

[jira] [Commented] (SPARK-3499) Create Spark-based distcp utility

2014-09-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131054#comment-14131054 ] Nicholas Chammas commented on SPARK-3499: - I'm not sure if this type of request

[jira] [Comment Edited] (SPARK-3499) Create Spark-based distcp utility

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131054#comment-14131054 ] Nicholas Chammas edited comment on SPARK-3499 at 9/12/14 2:27 PM:

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131688#comment-14131688 ] Nicholas Chammas commented on SPARK-3500: - [~davies] - Shouldn't the target

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131752#comment-14131752 ] Nicholas Chammas commented on SPARK-3500: - Hmm, you _could_ perhaps consider this

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131757#comment-14131757 ] Nicholas Chammas commented on SPARK-3500: - Btw, this seems like the same type of

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131933#comment-14131933 ] Nicholas Chammas commented on SPARK-3500: - [~davies] - PySpark doesn't seem to

[jira] [Created] (SPARK-3519) PySpark RDDs are missing the distinct(n) method

2014-09-13 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3519: --- Summary: PySpark RDDs are missing the distinct(n) method Key: SPARK-3519 URL: https://issues.apache.org/jira/browse/SPARK-3519 Project: Spark Issue

[jira] [Commented] (SPARK-3519) PySpark RDDs are missing the distinct(n) method

2014-09-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132833#comment-14132833 ] Nicholas Chammas commented on SPARK-3519: - [~joshrosen] [~davies]: Here is a

[jira] [Created] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-14 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3522: --- Summary: Make spark-ec2 verbosity configurable Key: SPARK-3522 URL: https://issues.apache.org/jira/browse/SPARK-3522 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3528) Reading data from file:/// should be called NODE_LOCAL not PROCESS_LOCAL

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133927#comment-14133927 ] Nicholas Chammas commented on SPARK-3528: - [~aash] - How about for data read from

[jira] [Commented] (SPARK-3526) Docs section on data locality

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133935#comment-14133935 ] Nicholas Chammas commented on SPARK-3526: - FYI: Looks like the valid localities

[jira] [Updated] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3533: Affects Version/s: 1.1.0 Add saveAsTextFileByKey() method to RDDs

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134800#comment-14134800 ] Nicholas Chammas commented on SPARK-922: [~joshrosen] By the way, as part of this

[jira] [Commented] (SPARK-3534) Avoid running MLlib and Streaming tests when testing SQL PRs

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134853#comment-14134853 ] Nicholas Chammas commented on SPARK-3534: - [~marmbrus] - If you just want to run

[jira] [Commented] (SPARK-1455) Determine which test suites to run based on code changes

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134856#comment-14134856 ] Nicholas Chammas commented on SPARK-1455: - I think we could reuse some of [the

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134877#comment-14134877 ] Nicholas Chammas commented on SPARK-3533: - CC [~davies] and [~pwendell]. Add

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135597#comment-14135597 ] Nicholas Chammas commented on SPARK-3533: - [~kzhang] - I noticed you authored

[jira] [Commented] (SPARK-2022) Spark 1.0.0 is failing if mesos.coarse set to true

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135663#comment-14135663 ] Nicholas Chammas commented on SPARK-2022: - Pinging [~pwendell] about closing this

  1   2   3   4   5   6   7   8   9   10   >