[jira] [Commented] (SPARK-4021) Kinesis code can cause compile failures with newer JDK's

2014-10-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177720#comment-14177720 ] Nicholas Chammas commented on SPARK-4021: - cc [~cfregly]. You may be interested in

[jira] [Commented] (SPARK-3928) Support wildcard matches on Parquet files

2014-10-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169580#comment-14169580 ] Nicholas Chammas commented on SPARK-3928: - cc [~marmbrus] > Support wildcard matc

[jira] [Created] (SPARK-3928) Support wildcard matches on Parquet files

2014-10-13 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3928: --- Summary: Support wildcard matches on Parquet files Key: SPARK-3928 URL: https://issues.apache.org/jira/browse/SPARK-3928 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-3849) Automate remaining Spark Code Style Guide rules

2014-10-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3849: Description: Style problems continue to take up a large amount of review time, mostly becau

[jira] [Comment Edited] (SPARK-922) Update Spark AMI to Python 2.7

2014-10-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169331#comment-14169331 ] Nicholas Chammas edited comment on SPARK-922 at 10/13/14 2:19 PM: ---

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-10-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169331#comment-14169331 ] Nicholas Chammas commented on SPARK-922: [~joshrosen] - Do you mean [this script|h

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-10-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168459#comment-14168459 ] Nicholas Chammas commented on SPARK-922: [~joshrosen] Are you open to having this r

[jira] [Updated] (SPARK-3849) Automate remaining Spark Code Style Guide rules

2014-10-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3849: Summary: Automate remaining Spark Code Style Guide rules (was: Automate remaining Scala sty

[jira] [Updated] (SPARK-3849) Automate remaining Scala style rules

2014-10-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3849: Description: Style problems continue to take up a large amount of review time, mostly becau

[jira] [Commented] (SPARK-3376) Memory-based shuffle strategy to reduce overhead of disk I/O

2014-10-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165359#comment-14165359 ] Nicholas Chammas commented on SPARK-3376: - [~matei], [~rxin], [~pwendell]: This is

[jira] [Updated] (SPARK-3850) Scala style: disallow trailing spaces

2014-10-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3850: Description: [Ted Yu on the dev list|http://mail-archives.apache.org/mod_mbox/spark-dev/2014

[jira] [Updated] (SPARK-3850) Scala style: disallow trailing spaces

2014-10-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3850: Summary: Scala style: disallow trailing spaces (was: Scala style: Disallow trailing spaces)

[jira] [Created] (SPARK-3850) Scala style: Disallow trailing spaces

2014-10-08 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3850: --- Summary: Scala style: Disallow trailing spaces Key: SPARK-3850 URL: https://issues.apache.org/jira/browse/SPARK-3850 Project: Spark Issue Type: Sub-tas

[jira] [Created] (SPARK-3849) Automate remaining Scala style rules

2014-10-08 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3849: --- Summary: Automate remaining Scala style rules Key: SPARK-3849 URL: https://issues.apache.org/jira/browse/SPARK-3849 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163618#comment-14163618 ] Nicholas Chammas commented on SPARK-3561: - {quote} Obviously this does not work in

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162663#comment-14162663 ] Nicholas Chammas commented on SPARK-3821: - [~shivaram] / [~pwendell]: # In a Spark

[jira] [Updated] (SPARK-3479) Have Jenkins show which category of tests failed in his GitHub messages

2014-10-06 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3479: Summary: Have Jenkins show which category of tests failed in his GitHub messages (was: Have

[jira] [Commented] (SPARK-3314) Script creation of AMIs

2014-10-06 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160901#comment-14160901 ] Nicholas Chammas commented on SPARK-3314: - Sounds good. I've created [SPARK-3821]

[jira] [Created] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2014-10-06 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3821: --- Summary: Develop an automated way of creating Spark images (AMI, Docker, and others) Key: SPARK-3821 URL: https://issues.apache.org/jira/browse/SPARK-3821 Proje

[jira] [Commented] (SPARK-3314) Script creation of AMIs

2014-10-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158777#comment-14158777 ] Nicholas Chammas commented on SPARK-3314: - Hey [~holdenk], I think this is a great

[jira] [Commented] (SPARK-3105) Calling cache() after RDDs are pipelined has no effect in PySpark

2014-10-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156936#comment-14156936 ] Nicholas Chammas commented on SPARK-3105: - I think it's definitely important for t

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2014-10-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14156904#comment-14156904 ] Nicholas Chammas commented on SPARK-2870: - [~marmbrus] - A related feature that I

[jira] [Commented] (SPARK-2247) Data frame (or Pandas) like API for structured data

2014-09-30 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154264#comment-14154264 ] Nicholas Chammas commented on SPARK-2247: - Is [Adato's work on "Distributed DataF

[jira] [Commented] (SPARK-2008) Enhance spark-ec2 to be able to add and remove slaves to an existing cluster

2014-09-30 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153723#comment-14153723 ] Nicholas Chammas commented on SPARK-2008: - [~shivaram] [~pwendell] - Is this a goo

[jira] [Updated] (SPARK-2463) Creating then stopping StreamingContext multiple times from shell generates duplicate Streaming tabs in UI

2014-09-30 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-2463: Summary: Creating then stopping StreamingContext multiple times from shell generates duplica

[jira] [Commented] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151787#comment-14151787 ] Nicholas Chammas commented on SPARK-3522: - Always logging to a file sounds like a

[jira] [Updated] (SPARK-3479) Have Jenkins show which tests failed in his GitHub messages

2014-09-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3479: Description: A nice thing to do for starters would be to report which category of tests fail

[jira] [Updated] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3522: Description: When launching a cluster, {{spark-ec2}} spits out a lot of stuff that feels li

[jira] [Commented] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151158#comment-14151158 ] Nicholas Chammas commented on SPARK-3522: - [~pwendell] [~shivaram] - Does this sou

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143744#comment-14143744 ] Nicholas Chammas commented on SPARK-3431: - I see. I'll try to look into it then. I

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143720#comment-14143720 ] Nicholas Chammas commented on SPARK-2870: - [~marmbrus] - API-wise, how are you thi

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143707#comment-14143707 ] Nicholas Chammas commented on SPARK-3431: - {quote} Do you know how maven / sbt plu

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143656#comment-14143656 ] Nicholas Chammas commented on SPARK-3431: - [~joshrosen] I can take a crack at this

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14143308#comment-14143308 ] Nicholas Chammas commented on SPARK-3533: - [~pwendell] / [~davies] - Is there any

[jira] [Commented] (SPARK-1455) Determine which test suites to run based on code changes

2014-09-18 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139751#comment-14139751 ] Nicholas Chammas commented on SPARK-1455: - There is still some work to be done to

[jira] [Commented] (SPARK-2463) Creating multiple StreamingContexts from shell generates duplicate Streaming tabs in UI

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136432#comment-14136432 ] Nicholas Chammas commented on SPARK-2463: - [~joshrosen] - Though the most recent c

[jira] [Commented] (SPARK-1455) Determine which test suites to run based on code changes

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136418#comment-14136418 ] Nicholas Chammas commented on SPARK-1455: - Which approach do y'all prefer? # Have

[jira] [Commented] (SPARK-2022) Spark 1.0.0 is failing if mesos.coarse set to true

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135663#comment-14135663 ] Nicholas Chammas commented on SPARK-2022: - Pinging [~pwendell] about closing this

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135597#comment-14135597 ] Nicholas Chammas commented on SPARK-3533: - [~kzhang] - I noticed you authored thes

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134877#comment-14134877 ] Nicholas Chammas commented on SPARK-3533: - CC [~davies] and [~pwendell]. > Add sa

[jira] [Commented] (SPARK-1455) Determine which test suites to run based on code changes

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134856#comment-14134856 ] Nicholas Chammas commented on SPARK-1455: - I think we could reuse some of [the log

[jira] [Commented] (SPARK-3534) Avoid running MLlib and Streaming tests when testing SQL PRs

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134853#comment-14134853 ] Nicholas Chammas commented on SPARK-3534: - [~marmbrus] - If you just want to run a

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134800#comment-14134800 ] Nicholas Chammas commented on SPARK-922: [~joshrosen] By the way, as part of this w

[jira] [Updated] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3533: Affects Version/s: 1.1.0 > Add saveAsTextFileByKey() method to RDDs > --

[jira] [Created] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2014-09-15 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3533: --- Summary: Add saveAsTextFileByKey() method to RDDs Key: SPARK-3533 URL: https://issues.apache.org/jira/browse/SPARK-3533 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-3526) Docs section on data locality

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133935#comment-14133935 ] Nicholas Chammas commented on SPARK-3526: - FYI: Looks like the valid localities ar

[jira] [Commented] (SPARK-3528) Reading data from file:/// should be called NODE_LOCAL not PROCESS_LOCAL

2014-09-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133927#comment-14133927 ] Nicholas Chammas commented on SPARK-3528: - [~aash] - How about for data read from

[jira] [Created] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-14 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3522: --- Summary: Make spark-ec2 verbosity configurable Key: SPARK-3522 URL: https://issues.apache.org/jira/browse/SPARK-3522 Project: Spark Issue Type: Improve

[jira] [Commented] (SPARK-3519) PySpark RDDs are missing the distinct(n) method

2014-09-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132833#comment-14132833 ] Nicholas Chammas commented on SPARK-3519: - [~joshrosen] & [~davies]: Here is a tic

[jira] [Created] (SPARK-3519) PySpark RDDs are missing the distinct(n) method

2014-09-13 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3519: --- Summary: PySpark RDDs are missing the distinct(n) method Key: SPARK-3519 URL: https://issues.apache.org/jira/browse/SPARK-3519 Project: Spark Issue Typ

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131933#comment-14131933 ] Nicholas Chammas commented on SPARK-3500: - [~davies] - PySpark doesn't seem to sup

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131757#comment-14131757 ] Nicholas Chammas commented on SPARK-3500: - Btw, this seems like the same type of p

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131752#comment-14131752 ] Nicholas Chammas commented on SPARK-3500: - Hmm, you _could_ perhaps consider this

[jira] [Commented] (SPARK-3500) SchemaRDD from jsonRDD() has not coalesce() method

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131688#comment-14131688 ] Nicholas Chammas commented on SPARK-3500: - [~davies] - Shouldn't the target versio

[jira] [Comment Edited] (SPARK-3499) Create Spark-based distcp utility

2014-09-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131054#comment-14131054 ] Nicholas Chammas edited comment on SPARK-3499 at 9/12/14 2:27 PM: --

[jira] [Commented] (SPARK-3499) Create Spark-based distcp utility

2014-09-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131054#comment-14131054 ] Nicholas Chammas commented on SPARK-3499: - I'm not sure if this type of request sh

[jira] [Created] (SPARK-3499) Create Spark-based distcp utility

2014-09-11 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3499: --- Summary: Create Spark-based distcp utility Key: SPARK-3499 URL: https://issues.apache.org/jira/browse/SPARK-3499 Project: Spark Issue Type: Wish

[jira] [Commented] (SPARK-2045) Sort-based shuffle implementation

2014-09-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130974#comment-14130974 ] Nicholas Chammas commented on SPARK-2045: - The [1.1.0 release notes|http://spark.

[jira] [Commented] (SPARK-2560) Create Spark SQL syntax reference

2014-09-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130580#comment-14130580 ] Nicholas Chammas commented on SPARK-2560: - I guess that's good for starters, but I

[jira] [Commented] (SPARK-2560) Create Spark SQL syntax reference

2014-09-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130464#comment-14130464 ] Nicholas Chammas commented on SPARK-2560: - [~marmbrus] - Would that be [this sect

[jira] [Updated] (SPARK-3479) Have Jenkins show which tests failed in his GitHub messages

2014-09-10 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3479: Summary: Have Jenkins show which tests failed in his GitHub messages (was: Have Jenkins sho

[jira] [Created] (SPARK-3479) Have Jenkins show which tests failed in GitHub message

2014-09-10 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3479: --- Summary: Have Jenkins show which tests failed in GitHub message Key: SPARK-3479 URL: https://issues.apache.org/jira/browse/SPARK-3479 Project: Spark Is

[jira] [Created] (SPARK-3432) Fix logging of unit test execution time

2014-09-07 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3432: --- Summary: Fix logging of unit test execution time Key: SPARK-3432 URL: https://issues.apache.org/jira/browse/SPARK-3432 Project: Spark Issue Type: Sub-t

[jira] [Created] (SPARK-3431) Parallelize execution of tests

2014-09-07 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3431: --- Summary: Parallelize execution of tests Key: SPARK-3431 URL: https://issues.apache.org/jira/browse/SPARK-3431 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

2014-09-05 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123203#comment-14123203 ] Nicholas Chammas commented on SPARK-3369: - {quote} The API change is unlikely to h

[jira] [Commented] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-09-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120957#comment-14120957 ] Nicholas Chammas commented on SPARK-3398: - Hey [~joshrosen], does this seem like a

[jira] [Updated] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-09-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-3398: Description: {{spark-ec2}} currently has retry logic for when it tries to install stuff on a

[jira] [Created] (SPARK-3398) Have spark-ec2 intelligently wait for specific cluster states

2014-09-03 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3398: --- Summary: Have spark-ec2 intelligently wait for specific cluster states Key: SPARK-3398 URL: https://issues.apache.org/jira/browse/SPARK-3398 Project: Spark

[jira] [Commented] (SPARK-3358) PySpark worker fork()ing performance regression in m3.* / PVM instances

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119276#comment-14119276 ] Nicholas Chammas commented on SPARK-3358: - Nit: Isn't it [PV and not PVM|http://d

[jira] [Commented] (SPARK-2627) Check for PEP 8 compliance on all Python code in the Jenkins CI cycle

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119239#comment-14119239 ] Nicholas Chammas commented on SPARK-2627: - Okie doke: [SPARK-3361] > Check for PE

[jira] [Created] (SPARK-3361) Expand PEP 8 checks to include EC2 script and Python examples

2014-09-02 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3361: --- Summary: Expand PEP 8 checks to include EC2 script and Python examples Key: SPARK-3361 URL: https://issues.apache.org/jira/browse/SPARK-3361 Project: Spark

[jira] [Commented] (SPARK-2627) Check for PEP 8 compliance on all Python code in the Jenkins CI cycle

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119226#comment-14119226 ] Nicholas Chammas commented on SPARK-2627: - Note: We should cover the EC2 script an

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119177#comment-14119177 ] Nicholas Chammas commented on SPARK-: - So I've repeated the tests with the exa

[jira] [Commented] (SPARK-3358) PySpark worker fork()ing performance regression in m3.* / PVM instances

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119141#comment-14119141 ] Nicholas Chammas commented on SPARK-3358: - Josh, do you think this is related to t

[jira] [Updated] (SPARK-3333) Large number of partitions causes OOM

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-: Attachment: nick-1.0.2.driver.log.zip nick-1.1.0-rc3.driver.log.zip Here are

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119119#comment-14119119 ] Nicholas Chammas commented on SPARK-: - Just to double check my results, I re-r

[jira] [Commented] (SPARK-3176) Implement 'POWER', 'ABS and 'LAST' for sql

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118796#comment-14118796 ] Nicholas Chammas commented on SPARK-3176: - Reposting a [comment I made on the PR|

[jira] [Commented] (SPARK-3354) Add LENGTH and DATALENGTH functions to Spark SQL

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118771#comment-14118771 ] Nicholas Chammas commented on SPARK-3354: - [~marmbrus] - This one's for your radar

[jira] [Created] (SPARK-3354) Add LENGTH and DATALENGTH functions to Spark SQL

2014-09-02 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3354: --- Summary: Add LENGTH and DATALENGTH functions to Spark SQL Key: SPARK-3354 URL: https://issues.apache.org/jira/browse/SPARK-3354 Project: Spark Issue Ty

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118735#comment-14118735 ] Nicholas Chammas commented on SPARK-: - {quote} If we can't narrow it down in t

[jira] [Commented] (SPARK-1701) Inconsistent naming: "slice" or "partition"

2014-09-02 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118282#comment-14118282 ] Nicholas Chammas commented on SPARK-1701: - Oh absolutely; sorry, didn't mean to mi

[jira] [Commented] (SPARK-1701) Inconsistent naming: "slice" or "partition"

2014-09-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117917#comment-14117917 ] Nicholas Chammas commented on SPARK-1701: - OK, so it sounds like action is: * Repl

[jira] [Commented] (SPARK-1701) Inconsistent naming: "slice" or "partition"

2014-09-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117883#comment-14117883 ] Nicholas Chammas commented on SPARK-1701: - [~pwendell] and [~rxin], I'm pinging yo

[jira] [Commented] (SPARK-1701) Inconsistent naming: "slice" or "partition"

2014-09-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117880#comment-14117880 ] Nicholas Chammas commented on SPARK-1701: - In addition to "slice" and "partition",

[jira] [Comment Edited] (SPARK-3333) Large number of partitions causes OOM

2014-09-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117853#comment-14117853 ] Nicholas Chammas edited comment on SPARK- at 9/2/14 3:13 AM: ---

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-09-01 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117853#comment-14117853 ] Nicholas Chammas commented on SPARK-: - It looks like the default number of re

[jira] [Comment Edited] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14116978#comment-14116978 ] Nicholas Chammas edited comment on SPARK- at 9/1/14 2:09 AM: ---

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14116978#comment-14116978 ] Nicholas Chammas commented on SPARK-: - For the record, I got the OOM in a rela

[jira] [Updated] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-: Description: Here’s a repro for PySpark: {code} a = sc.parallelize(["Nick", "John", "Bob"])

[jira] [Commented] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14116895#comment-14116895 ] Nicholas Chammas commented on SPARK-: - Note: I have not yet confirmed that 1.1

[jira] [Created] (SPARK-3333) Large number of partitions causes OOM

2014-08-31 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-: --- Summary: Large number of partitions causes OOM Key: SPARK- URL: https://issues.apache.org/jira/browse/SPARK- Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2014-08-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14116868#comment-14116868 ] Nicholas Chammas commented on SPARK-2870: - [~marmbrus], [~davies], [~yhuai] - We d

[jira] [Commented] (SPARK-3094) Support run pyspark in PyPy

2014-08-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14116026#comment-14116026 ] Nicholas Chammas commented on SPARK-3094: - This is super cool. If [PyPy grows to s

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115636#comment-14115636 ] Nicholas Chammas commented on SPARK-922: FYI, I believe the line to install numpy o

[jira] [Commented] (SPARK-3044) Create RSS feed for Spark News

2014-08-27 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113061#comment-14113061 ] Nicholas Chammas commented on SPARK-3044: - Hehe, I believe many people migrated ov

[jira] [Comment Edited] (SPARK-3044) Create RSS feed for Spark News

2014-08-26 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109677#comment-14109677 ] Nicholas Chammas edited comment on SPARK-3044 at 8/26/14 7:30 PM: --

[jira] [Commented] (SPARK-3044) Create RSS feed for Spark News

2014-08-25 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109677#comment-14109677 ] Nicholas Chammas commented on SPARK-3044: - Hi Michael, I don't know if the site i

[jira] [Created] (SPARK-3076) Gracefully report build timeouts in Jenkins

2014-08-15 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3076: --- Summary: Gracefully report build timeouts in Jenkins Key: SPARK-3076 URL: https://issues.apache.org/jira/browse/SPARK-3076 Project: Spark Issue Type: S

[jira] [Commented] (SPARK-922) Update Spark AMI to Python 2.7

2014-08-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14099049#comment-14099049 ] Nicholas Chammas commented on SPARK-922: Josh, at the end of your updated script do

[jira] [Created] (SPARK-3044) Create RSS feed for Spark News

2014-08-14 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-3044: --- Summary: Create RSS feed for Spark News Key: SPARK-3044 URL: https://issues.apache.org/jira/browse/SPARK-3044 Project: Spark Issue Type: Documentation

<    6   7   8   9   10   11   12   >