[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2015-12-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053977#comment-15053977 ] Nicholas Chammas commented on SPARK-2870: - > Do you think its OK to close this issue? I haven't

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2015-12-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053131#comment-15053131 ] Nicholas Chammas commented on SPARK-2870: - Go for it. I don't think anyone else is. > Thorough

[jira] [Created] (SPARK-12107) Update spark-ec2 versions

2015-12-02 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-12107: Summary: Update spark-ec2 versions Key: SPARK-12107 URL: https://issues.apache.org/jira/browse/SPARK-12107 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

2015-11-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022735#comment-15022735 ] Nicholas Chammas edited comment on SPARK- at 11/23/15 8:06 PM: ---

[jira] [Commented] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

2015-11-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022735#comment-15022735 ] Nicholas Chammas commented on SPARK-: - [~sandyr] - Hmm, so are you saying that, generally

[jira] [Commented] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

2015-11-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022957#comment-15022957 ] Nicholas Chammas commented on SPARK-: - If you are referring to my comment, note that I am

[jira] [Commented] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

2015-11-21 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020729#comment-15020729 ] Nicholas Chammas commented on SPARK-11903: -- Also, we could just leave the option in there and

[jira] [Created] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

2015-11-21 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-11903: Summary: Deprecate make-distribution.sh --skip-java-test Key: SPARK-11903 URL: https://issues.apache.org/jira/browse/SPARK-11903 Project: Spark

[jira] [Commented] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

2015-11-21 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020725#comment-15020725 ] Nicholas Chammas commented on SPARK-11903: -- cc [~pwendell] and [~srowen] - Y'all probably know

[jira] [Commented] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

2015-11-21 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020728#comment-15020728 ] Nicholas Chammas commented on SPARK-11903: -- Oh, could you elaborate a bit? From what I

[jira] [Updated] (SPARK-11903) Deprecate make-distribution.sh --skip-java-test

2015-11-21 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-11903: - Description: The {{\-\-skip-java-test}} option to {{make-distribution.sh}} [does not

[jira] [Commented] (SPARK-9999) Dataset API on top of Catalyst/DataFrame

2015-11-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15019214#comment-15019214 ] Nicholas Chammas commented on SPARK-: - Arriving a little late to this discussion. Quick

[jira] [Commented] (SPARK-11744) bin/pyspark --version doesn't return version and exit

2015-11-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005572#comment-15005572 ] Nicholas Chammas commented on SPARK-11744: -- Not sure who would be the best person to comment on

[jira] [Updated] (SPARK-11744) bin/pyspark --version doesn't return version and exit

2015-11-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-11744: - Description: {{bin/pyspark \-\-help}} offers a {{\-\-version}} option: {code} $

[jira] [Created] (SPARK-11744) bin/pyspark --version doesn't return version and exit

2015-11-14 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-11744: Summary: bin/pyspark --version doesn't return version and exit Key: SPARK-11744 URL: https://issues.apache.org/jira/browse/SPARK-11744 Project: Spark

[jira] [Updated] (SPARK-11744) bin/pyspark --version doesn't return version and exit

2015-11-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-11744: - Description: {{bin/pyspark \-\-help}} offers a {{\-\-version}} option: {code} $

[jira] [Commented] (SPARK-3342) m3 instances don't get local SSDs

2015-10-26 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974660#comment-14974660 ] Nicholas Chammas commented on SPARK-3342: - FWIW, that statement on M3 instances is [no longer

[jira] [Commented] (SPARK-10002) SSH problem during Setup of Spark(1.3.0) cluster on EC2

2015-10-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969814#comment-14969814 ] Nicholas Chammas commented on SPARK-10002: -- [~deepalib] - Is {{--private-ips}} the solution, as

[jira] [Commented] (SPARK-2622) Add Jenkins build numbers to SparkQA messages

2015-09-17 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14803169#comment-14803169 ] Nicholas Chammas commented on SPARK-2622: - [~mxm] - I noticed you have been posting this kind of

[jira] [Commented] (SPARK-2622) Add Jenkins build numbers to SparkQA messages

2015-09-17 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804559#comment-14804559 ] Nicholas Chammas commented on SPARK-2622: - No worries. Thanks for quickly finding and resolving

[jira] [Commented] (SPARK-4216) Eliminate duplicate Jenkins GitHub posts from AMPLab

2015-09-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791016#comment-14791016 ] Nicholas Chammas commented on SPARK-4216: - Thanks Josh! > Eliminate duplicate Jenkins GitHub

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

2015-09-08 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735323#comment-14735323 ] Nicholas Chammas commented on SPARK-3369: - Sean said: {quote} I don't think there's a "why" –

[jira] [Commented] (SPARK-10191) spark-ec2 cannot stop running cluster

2015-08-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710093#comment-14710093 ] Nicholas Chammas commented on SPARK-10191: -- Can you fill in the description here

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2015-08-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705446#comment-14705446 ] Nicholas Chammas commented on SPARK-3533: - {quote} Nicholas Chammas Have you been

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2015-08-20 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705182#comment-14705182 ] Nicholas Chammas commented on SPARK-3533: - No need to open a separate ticket if

[jira] [Commented] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

2015-08-17 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699613#comment-14699613 ] Nicholas Chammas commented on SPARK-3533: - [~silasdavis] - If you already have a

[jira] [Commented] (SPARK-8960) Style cleanup of spark_ec2.py

2015-07-10 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14622542#comment-14622542 ] Nicholas Chammas commented on SPARK-8960: - Style cleanup is OK, but should be

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14605822#comment-14605822 ] Nicholas Chammas commented on SPARK-8670: - Not sure. Does Scala offer the same

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606178#comment-14606178 ] Nicholas Chammas commented on SPARK-8670: - FYI: `df.stats.age` works neither on

[jira] [Comment Edited] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606328#comment-14606328 ] Nicholas Chammas edited comment on SPARK-8670 at 6/29/15 9:01 PM:

[jira] [Resolved] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-8670. - Resolution: Invalid Nested columns can't be referenced (but they can be selected)

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606328#comment-14606328 ] Nicholas Chammas commented on SPARK-8670: - After a discussion with [~davies], it

[jira] [Created] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-26 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-8670: --- Summary: Nested columns can't be referenced (but they can be selected) Key: SPARK-8670 URL: https://issues.apache.org/jira/browse/SPARK-8670 Project: Spark

[jira] [Updated] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-26 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-8670: Description: This is strange and looks like a regression from 1.3. {code} import json

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-26 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603572#comment-14603572 ] Nicholas Chammas commented on SPARK-8670: - cc [~rxin], [~davies] Nested columns

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

2015-06-26 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603658#comment-14603658 ] Nicholas Chammas commented on SPARK-8670: - I thought, per the discussion on

[jira] [Resolved] (SPARK-6220) Allow extended EC2 options to be passed through spark-ec2

2015-06-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-6220. - Resolution: Won't Fix Resolving this issue as won't fix since it is of low importance and

[jira] [Updated] (SPARK-8576) Add spark-ec2 options to assign launched instances into IAM roles and to set instance-initiated shutdown behavior

2015-06-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-8576: Summary: Add spark-ec2 options to assign launched instances into IAM roles and to set

[jira] [Created] (SPARK-8576) Add spark-ec2 options to assigned launched instances into IAM roles and to set instance-initiated shutdown behavior

2015-06-23 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-8576: --- Summary: Add spark-ec2 options to assigned launched instances into IAM roles and to set instance-initiated shutdown behavior Key: SPARK-8576 URL:

[jira] [Commented] (SPARK-8417) spark-class has illegal statement

2015-06-18 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14591988#comment-14591988 ] Nicholas Chammas commented on SPARK-8417: - I'm not sure what I'm looking at. Can

[jira] [Commented] (SPARK-8429) Add ability to set additional tags

2015-06-18 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14592010#comment-14592010 ] Nicholas Chammas commented on SPARK-8429: - What is your use case for this feature?

[jira] [Commented] (SPARK-6220) Allow extended EC2 options to be passed through spark-ec2

2015-06-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14586334#comment-14586334 ] Nicholas Chammas commented on SPARK-6220: - please forgive my greenness No need.

[jira] [Created] (SPARK-8316) Upgrade Maven to 3.3.3

2015-06-11 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-8316: --- Summary: Upgrade Maven to 3.3.3 Key: SPARK-8316 URL: https://issues.apache.org/jira/browse/SPARK-8316 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-5398) Support the eu-central-1 region for spark-ec2

2015-06-04 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573138#comment-14573138 ] Nicholas Chammas commented on SPARK-5398: - I don't have the credentials to do

[jira] [Commented] (SPARK-5398) Support the eu-central-1 region for spark-ec2

2015-06-04 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573140#comment-14573140 ] Nicholas Chammas commented on SPARK-5398: - I don't have the credentials to do

[jira] [Issue Comment Deleted] (SPARK-5398) Support the eu-central-1 region for spark-ec2

2015-06-04 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5398: Comment: was deleted (was: I don't have the credentials to do that, unfortunately. Maybe

[jira] [Commented] (SPARK-7900) Reduce number of tagging calls in spark-ec2

2015-06-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571460#comment-14571460 ] Nicholas Chammas commented on SPARK-7900: - I'm marking this as a duplicate of

[jira] [Resolved] (SPARK-7900) Reduce number of tagging calls in spark-ec2

2015-06-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-7900. - Resolution: Duplicate Reduce number of tagging calls in spark-ec2

[jira] [Commented] (SPARK-4983) Add sleep() before tagging EC2 instances to allow instance metadata to propagate

2015-06-03 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571467#comment-14571467 ] Nicholas Chammas commented on SPARK-4983: - Per the discussion on [SPARK-7900], I

[jira] [Updated] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

2015-05-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5189: Description: As of 1.2.0, we launch Spark clusters on EC2 by setting up the master first,

[jira] [Updated] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

2015-05-31 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5189: Description: As of 1.2.0, we launch Spark clusters on EC2 by setting up the master first,

[jira] [Commented] (SPARK-7900) Reduce number of tagging calls in spark-ec2

2015-05-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563208#comment-14563208 ] Nicholas Chammas commented on SPARK-7900: - The name tags are optional, but we can

[jira] [Commented] (SPARK-7900) Reduce number of tagging calls in spark-ec2

2015-05-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563053#comment-14563053 ] Nicholas Chammas commented on SPARK-7900: - An alternative approach would be to

[jira] [Created] (SPARK-7900) Reduce number of tagging calls in spark-ec2

2015-05-27 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-7900: --- Summary: Reduce number of tagging calls in spark-ec2 Key: SPARK-7900 URL: https://issues.apache.org/jira/browse/SPARK-7900 Project: Spark Issue Type:

[jira] [Commented] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

2015-05-22 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14556860#comment-14556860 ] Nicholas Chammas commented on SPARK-7505: - cc [~davies] - I think the most

[jira] [Commented] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

2015-05-21 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14555482#comment-14555482 ] Nicholas Chammas commented on SPARK-7507: - Since {{Row}} seems most analogous to a

[jira] [Commented] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

2015-05-21 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14554550#comment-14554550 ] Nicholas Chammas commented on SPARK-7507: - Related: A Stack Overflow question

[jira] [Commented] (SPARK-7640) Private VPC with default Spark AMI breaks yum

2015-05-19 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551795#comment-14551795 ] Nicholas Chammas commented on SPARK-7640: - [~brdwrd] - According to [this doc on

[jira] [Commented] (SPARK-7640) Private VPC with default Spark AMI breaks yum

2015-05-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544169#comment-14544169 ] Nicholas Chammas commented on SPARK-7640: - {quote} Switch everything to support

[jira] [Commented] (SPARK-7640) Private VPC with default Spark AMI breaks yum

2015-05-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544288#comment-14544288 ] Nicholas Chammas commented on SPARK-7640: - If there is no way around this (like,

[jira] [Comment Edited] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

2015-05-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542701#comment-14542701 ] Nicholas Chammas edited comment on SPARK-7606 at 5/13/15 8:57 PM:

[jira] [Commented] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

2015-05-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542701#comment-14542701 ] Nicholas Chammas commented on SPARK-7606: - Just looked into this. If we are using

[jira] [Created] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

2015-05-13 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-7606: --- Summary: Document all PySpark SQL/DataFrame public methods with @since tag Key: SPARK-7606 URL: https://issues.apache.org/jira/browse/SPARK-7606 Project: Spark

[jira] [Commented] (SPARK-7606) Document all PySpark SQL/DataFrame public methods with @since tag

2015-05-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542176#comment-14542176 ] Nicholas Chammas commented on SPARK-7606: - (I just cloned SPARK-7588.) Dunno what

[jira] [Commented] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

2015-05-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542175#comment-14542175 ] Nicholas Chammas commented on SPARK-7507: - cc [~davies] On a related note to our

[jira] [Updated] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

2015-05-13 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7505: Description: The PySpark docs for DataFrame need the following fixes and improvements: #

[jira] [Commented] (SPARK-7548) Add explode expression

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538937#comment-14538937 ] Nicholas Chammas commented on SPARK-7548: - To provide a motivating example for the

[jira] [Updated] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7509: Target Version/s: 1.4.0 I'm targeting this for 1.4.0, though that's optimistic given that

[jira] [Commented] (SPARK-7549) Support aggregating over nested fields

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538965#comment-14538965 ] Nicholas Chammas commented on SPARK-7549: - To provide a motivating example for the

[jira] [Commented] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538978#comment-14538978 ] Nicholas Chammas commented on SPARK-7509: - Oh, well nevermind then. :) Add drop

[jira] [Commented] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538832#comment-14538832 ] Nicholas Chammas commented on SPARK-7133: - [SPARK-7544} Implement struct, array,

[jira] [Comment Edited] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538832#comment-14538832 ] Nicholas Chammas edited comment on SPARK-7133 at 5/11/15 11:02 PM:

[jira] [Created] (SPARK-7544) pyspark.sql.types.Row should implement __getitem__

2015-05-11 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-7544: --- Summary: pyspark.sql.types.Row should implement __getitem__ Key: SPARK-7544 URL: https://issues.apache.org/jira/browse/SPARK-7544 Project: Spark Issue

[jira] [Commented] (SPARK-7544) pyspark.sql.types.Row should implement __getitem__

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538830#comment-14538830 ] Nicholas Chammas commented on SPARK-7544: - cc [~rxin], [~davies]

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539255#comment-14539255 ] Nicholas Chammas commented on SPARK-2870: - Another use case for this feature is

[jira] [Commented] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538272#comment-14538272 ] Nicholas Chammas commented on SPARK-7133: - [~rxin] - Should we also implement

[jira] [Commented] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538283#comment-14538283 ] Nicholas Chammas commented on SPARK-7507: - On a related note, perhaps we should

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2015-05-10 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537324#comment-14537324 ] Nicholas Chammas commented on SPARK-2870: - cc [~rxin] - Are we still targeting

[jira] [Commented] (SPARK-7280) Add a method for dropping a column in Java/Scala

2015-05-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536885#comment-14536885 ] Nicholas Chammas commented on SPARK-7280: - OK, good to know. Add a method for

[jira] [Commented] (SPARK-7280) Add a method for dropping a column

2015-05-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536733#comment-14536733 ] Nicholas Chammas commented on SPARK-7280: - I don't think we should mark this as

[jira] [Reopened] (SPARK-7280) Add a method for dropping a column

2015-05-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas reopened SPARK-7280: - Add a method for dropping a column --

[jira] [Commented] (SPARK-7035) Drop __getattr__ on pyspark.sql.DataFrame

2015-05-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536751#comment-14536751 ] Nicholas Chammas commented on SPARK-7035: - {quote} Zen of Python states there

[jira] [Updated] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

2015-05-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7505: Component/s: PySpark Update PySpark DataFrame docs: encourage __getitem__, mark as

[jira] [Created] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

2015-05-09 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-7505: --- Summary: Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc. Key: SPARK-7505 URL: https://issues.apache.org/jira/browse/SPARK-7505

[jira] [Created] (SPARK-7507) pyspark.sql.types.StructType and Row should implement __iter__()

2015-05-09 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-7507: --- Summary: pyspark.sql.types.StructType and Row should implement __iter__() Key: SPARK-7507 URL: https://issues.apache.org/jira/browse/SPARK-7507 Project: Spark

[jira] [Created] (SPARK-7506) pyspark.sql.types.StructType.fromJson() is incorrectly named

2015-05-09 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-7506: --- Summary: pyspark.sql.types.StructType.fromJson() is incorrectly named Key: SPARK-7506 URL: https://issues.apache.org/jira/browse/SPARK-7506 Project: Spark

[jira] [Updated] (SPARK-7505) Update PySpark DataFrame docs: encourage __getitem__, mark as experimental, etc.

2015-05-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7505: Component/s: SQL Update PySpark DataFrame docs: encourage __getitem__, mark as

[jira] [Commented] (SPARK-7280) Add a method for dropping a column in Java/Scala

2015-05-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536835#comment-14536835 ] Nicholas Chammas commented on SPARK-7280: - SGTM. Add a method for dropping a

[jira] [Commented] (SPARK-7280) Add a method for dropping a column in Java/Scala

2015-05-09 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536841#comment-14536841 ] Nicholas Chammas commented on SPARK-7280: - So do we still need to add this method

[jira] [Commented] (SPARK-3928) Support wildcard matches on Parquet files

2015-05-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533277#comment-14533277 ] Nicholas Chammas commented on SPARK-3928: - {quote} Comma separated lists: were

[jira] [Commented] (SPARK-3928) Support wildcard matches on Parquet files

2015-05-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532918#comment-14532918 ] Nicholas Chammas commented on SPARK-3928: - [~yanakad] - Do you see this same issue

[jira] [Reopened] (SPARK-3928) Support wildcard matches on Parquet files

2015-05-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas reopened SPARK-3928: - Reopening this issue per the recent comments. Support wildcard matches on Parquet files

[jira] [Commented] (SPARK-3928) Support wildcard matches on Parquet files

2015-05-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532922#comment-14532922 ] Nicholas Chammas commented on SPARK-3928: - it seems they don't intend to change

[jira] [Commented] (SPARK-3928) Support wildcard matches on Parquet files

2015-05-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532978#comment-14532978 ] Nicholas Chammas commented on SPARK-3928: - Pinging [~lian cheng], who removed the

[jira] [Updated] (SPARK-7442) Spark 1.3.1 / Hadoop 2.6 prebuilt pacakge has a broken S3 filesystem access

2015-05-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7442: Summary: Spark 1.3.1 / Hadoop 2.6 prebuilt pacakge has a broken S3 filesystem access (was:

[jira] [Updated] (SPARK-7442) Spark 1.3.1 / Hadoop 2.6 prebuilt pacakge has broken S3 filesystem access

2015-05-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7442: Summary: Spark 1.3.1 / Hadoop 2.6 prebuilt pacakge has broken S3 filesystem access (was:

[jira] [Commented] (SPARK-7442) Spark 1.3.1 / Hadoop 2.6 prebuilt pacakge has broken S3 filesystem access

2015-05-07 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14533029#comment-14533029 ] Nicholas Chammas commented on SPARK-7442: - Apparently this is an issue with Hadoop

[jira] [Commented] (SPARK-5189) Reorganize EC2 scripts so that nodes can be provisioned independent of Spark master

2015-04-28 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517301#comment-14517301 ] Nicholas Chammas commented on SPARK-5189: - Yeah, as Sean said you can just start

[jira] [Commented] (SPARK-6900) spark ec2 script enters infinite loop when run-instance fails

2015-04-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14511283#comment-14511283 ] Nicholas Chammas commented on SPARK-6900: - That is correct. So again the solution

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with Launch More Like This

2015-04-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512233#comment-14512233 ] Nicholas Chammas commented on SPARK-3213: - Thanks for the background, Joseph and

<    1   2   3   4   5   6   7   8   9   10   >