[jira] [Created] (SPARK-4709) Spark SQL support for Parquet with timestamp type field

2014-12-02 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-4709: --- Summary: Spark SQL support for Parquet with timestamp type field Key: SPARK-4709 URL: https://issues.apache.org/jira/browse/SPARK-4709 Project: Spark Issue

[jira] [Updated] (SPARK-4709) Spark SQL support error reading Parquet with timestamp type field

2014-12-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-4709: Summary: Spark SQL support error reading Parquet with timestamp type field (was: Spark SQL support

[jira] [Updated] (SPARK-6407) Streaming ALS for Collaborative Filtering

2015-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-6407: Description: Like MLLib's ALS implementation for recommendation, and applying to streaming. Similar

[jira] [Created] (SPARK-6407) Streaming ALS for Collaborative Filtering

2015-03-18 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-6407: --- Summary: Streaming ALS for Collaborative Filtering Key: SPARK-6407 URL: https://issues.apache.org/jira/browse/SPARK-6407 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+

2015-07-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609673#comment-14609673 ] Felix Cheung commented on SPARK-8724: - For instance,

[jira] [Commented] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+

2015-07-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14609666#comment-14609666 ] Felix Cheung commented on SPARK-8724: - Doesn't SparkR need to be installed in R for

[jira] [Commented] (SPARK-9325) Support `collect` on DataFrame columns

2015-08-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707136#comment-14707136 ] Felix Cheung commented on SPARK-9325: - I'll take a shot Support `collect` on

[jira] [Commented] (SPARK-9325) Support `collect` on DataFrame columns

2015-08-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707075#comment-14707075 ] Felix Cheung commented on SPARK-9325: - So we should add collect and head to Column?

[jira] [Commented] (SPARK-9319) Add support for setting column names, types

2015-08-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707077#comment-14707077 ] Felix Cheung commented on SPARK-9319: - [~falaki]Would you be working on this? Add

[jira] [Commented] (SPARK-9803) Add transform and subset to DataFrame

2015-08-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708536#comment-14708536 ] Felix Cheung commented on SPARK-9803: - subset would be a synonym for `[`

[jira] [Commented] (SPARK-8041) Consistently pass SparkR library directory to SparkR application

2015-06-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606996#comment-14606996 ] Felix Cheung commented on SPARK-8041: - I've done some work to automatically get the

[jira] [Commented] (SPARK-8277) SparkR createDataFrame is slow

2015-06-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14607190#comment-14607190 ] Felix Cheung commented on SPARK-8277: - what would be a better approach? Would it work

[jira] [Commented] (SPARK-6836) RDD APIs missing in SparkR

2015-06-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606993#comment-14606993 ] Felix Cheung commented on SPARK-6836: - +1 on randomSplit - probably need that for

[jira] [Created] (SPARK-8724) Need documentation on how to deploy or use SparkR in Spark 1.4.0+

2015-06-29 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-8724: --- Summary: Need documentation on how to deploy or use SparkR in Spark 1.4.0+ Key: SPARK-8724 URL: https://issues.apache.org/jira/browse/SPARK-8724 Project: Spark

[jira] [Commented] (SPARK-1517) Publish nightly snapshots of documentation, maven artifacts, and binary builds

2015-06-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14606961#comment-14606961 ] Felix Cheung commented on SPARK-1517: - Does this handle SparkR doc build (Which

[jira] [Created] (SPARK-11255) R Test build should run on R 3.1.1

2015-10-22 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11255: Summary: R Test build should run on R 3.1.1 Key: SPARK-11255 URL: https://issues.apache.org/jira/browse/SPARK-11255 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11231) join returns schema with duplicated and ambiguous join columns

2015-10-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967564#comment-14967564 ] Felix Cheung commented on SPARK-11231: -- This should be addressed in the new merge implementation in

[jira] [Commented] (SPARK-10500) sparkr.zip cannot be created if $SPARK_HOME/R/lib is unwritable

2015-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973444#comment-14973444 ] Felix Cheung commented on SPARK-10500: -- [~sunrui]suggestion 08/Sept makes sense. Would you like to

[jira] [Created] (SPARK-11340) Support setting driver properties when starting Spark from R programmatically or from RStudio

2015-10-26 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11340: Summary: Support setting driver properties when starting Spark from R programmatically or from RStudio Key: SPARK-11340 URL: https://issues.apache.org/jira/browse/SPARK-11340

[jira] [Created] (SPARK-11407) Add documentation on using SparkR from RStudio

2015-10-29 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11407: Summary: Add documentation on using SparkR from RStudio Key: SPARK-11407 URL: https://issues.apache.org/jira/browse/SPARK-11407 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11407) Add documentation on using SparkR from RStudio

2015-10-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981318#comment-14981318 ] Felix Cheung commented on SPARK-11407: -- [~shivaram]what other content would you like to be added in

[jira] [Created] (SPARK-11409) Enable url link in R doc for Persist

2015-10-29 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11409: Summary: Enable url link in R doc for Persist Key: SPARK-11409 URL: https://issues.apache.org/jira/browse/SPARK-11409 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11167) Incorrect type resolution on heterogeneous data structures

2015-10-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981398#comment-14981398 ] Felix Cheung commented on SPARK-11167: -- Another error from PR 8984 I can have this > e <-

[jira] [Created] (SPARK-11294) Improve R doc for read.df, write.df, saveAsTable

2015-10-23 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11294: Summary: Improve R doc for read.df, write.df, saveAsTable Key: SPARK-11294 URL: https://issues.apache.org/jira/browse/SPARK-11294 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11167) Incorrect type resolution on heterogeneous data structures

2015-10-22 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970304#comment-14970304 ] Felix Cheung commented on SPARK-11167: -- Shouldn't it try to infer the tightest type that could

[jira] [Commented] (SPARK-11339) Document the list of functions in R base package that are masked by functions with same name in SparkR

2015-11-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14995810#comment-14995810 ] Felix Cheung commented on SPARK-11339: -- [~sunrui] [~shivaram] Is this the list? {code} Attaching

[jira] [Commented] (SPARK-11698) Add option to ignore kafka messages that are out of limit rate

2015-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003301#comment-15003301 ] Felix Cheung commented on SPARK-11698: -- Should we ignore message? Doesn't that mean possible data

[jira] [Commented] (SPARK-10863) Method coltypes() to return the R column types of a DataFrame

2015-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003548#comment-15003548 ] Felix Cheung commented on SPARK-10863: -- [~olarayej] I"m a little confused - I thought from

[jira] [Commented] (SPARK-10863) Method coltypes() to return the R column types of a DataFrame

2015-11-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15004442#comment-15004442 ] Felix Cheung commented on SPARK-10863: -- I understand but I think this is problematic in several

[jira] [Commented] (SPARK-11756) SparkR can not output help information for SparkR:::summary correctly

2015-11-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007481#comment-15007481 ] Felix Cheung commented on SPARK-11756: -- [~yanboliang] Could you please clarify what is the issue?

[jira] [Updated] (SPARK-11756) SparkR can not output help information for SparkR:::summary correctly

2015-11-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-11756: - Component/s: SparkR > SparkR can not output help information for SparkR:::summary correctly >

[jira] [Updated] (SPARK-11755) SparkR can not output help information for SparkR:::predict

2015-11-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-11755: - Component/s: SparkR > SparkR can not output help information for SparkR:::predict >

[jira] [Commented] (SPARK-11756) SparkR can not output help information for SparkR:::summary correctly

2015-11-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007792#comment-15007792 ] Felix Cheung commented on SPARK-11756: -- Fix use of aliases and changes uses of @rdname and @seealso

[jira] [Commented] (SPARK-11587) SparkR can not use summary.glm from base R

2015-11-09 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997750#comment-14997750 ] Felix Cheung commented on SPARK-11587: -- [~shivaram] here's the list SPARK-11339 (which includes

[jira] [Commented] (SPARK-11468) Add R API for stddev/variance

2015-11-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990180#comment-14990180 ] Felix Cheung commented on SPARK-11468: -- I'd like to take this if no one has started yet > Add R API

[jira] [Created] (SPARK-11567) Add Python API for corr in agg

2015-11-06 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11567: Summary: Add Python API for corr in agg Key: SPARK-11567 URL: https://issues.apache.org/jira/browse/SPARK-11567 Project: Spark Issue Type: Task Affects

[jira] [Updated] (SPARK-11567) Add Python API for corr aggregate function

2015-11-06 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-11567: - Summary: Add Python API for corr aggregate function (was: Add Python API for corr in agg) >

[jira] [Commented] (SPARK-11263) lintr Throws Warnings on Commented Code in Documentation

2015-11-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987964#comment-14987964 ] Felix Cheung commented on SPARK-11263: -- [~shivaram] what should we do with this? Should we convert

[jira] [Commented] (SPARK-11492) Ignore commented code warnings in SparkR

2015-11-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988949#comment-14988949 ] Felix Cheung commented on SPARK-11492: -- please see SPARK-11263 > Ignore commented code warnings in

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2015-10-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961371#comment-14961371 ] Felix Cheung commented on SPARK-11153: -- so the corrupted stats data would still be a problem when

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2015-10-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961374#comment-14961374 ] Felix Cheung commented on SPARK-11153: -- re-read what you said, I think it makes sense. I assume it

[jira] [Commented] (SPARK-10971) sparkR: RRunner should allow setting path to Rscript

2015-10-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949167#comment-14949167 ] Felix Cheung commented on SPARK-10971: -- I think he is suggesting the path to R/Rscript to be

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949185#comment-14949185 ] Felix Cheung commented on SPARK-10903: -- [~sunrui] Agreed. I'd like to propose adding .Deprecated to

[jira] [Created] (SPARK-11137) Make StreamingContext.stop() exception-safe

2015-10-15 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11137: Summary: Make StreamingContext.stop() exception-safe Key: SPARK-11137 URL: https://issues.apache.org/jira/browse/SPARK-11137 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-11139) Make SparkContext.stop() exception-safe

2015-10-15 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11139: Summary: Make SparkContext.stop() exception-safe Key: SPARK-11139 URL: https://issues.apache.org/jira/browse/SPARK-11139 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-11199) Improve R context management story and add getOrCreate

2015-10-19 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11199: Summary: Improve R context management story and add getOrCreate Key: SPARK-11199 URL: https://issues.apache.org/jira/browse/SPARK-11199 Project: Spark Issue

[jira] [Comment Edited] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964326#comment-14964326 ] Felix Cheung edited comment on SPARK-10903 at 10/20/15 12:46 AM: - Looked

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964326#comment-14964326 ] Felix Cheung commented on SPARK-10903: -- Looked into and tested with a few approaches. Tried to

[jira] [Updated] (SPARK-11199) Improve R context management story and add getOrCreate

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-11199: - Component/s: (was: R) SparkR > Improve R context management story and add

[jira] [Commented] (SPARK-11199) Improve R context management story and add getOrCreate

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964332#comment-14964332 ] Felix Cheung commented on SPARK-11199: -- I would work on this after SPARK-10903. [~falaki] [~davies]

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964333#comment-14964333 ] Felix Cheung commented on SPARK-10903: -- Created SPARK-11199 to track context initialization work. >

[jira] [Comment Edited] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964507#comment-14964507 ] Felix Cheung edited comment on SPARK-10903 at 10/20/15 4:40 AM: Same with

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964507#comment-14964507 ] Felix Cheung commented on SPARK-10903: -- Same with this {code} createDataFrame <- function(x, ...) {

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964551#comment-14964551 ] Felix Cheung commented on SPARK-10903: -- And when revert the code in SparkR package it's working as

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964570#comment-14964570 ] Felix Cheung commented on SPARK-10903: -- Something to do with the namespace {code} >

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964531#comment-14964531 ] Felix Cheung commented on SPARK-10903: -- That's puzzling. Any tips on how to debug this? > Make

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964543#comment-14964543 ] Felix Cheung commented on SPARK-10903: -- It says 'no methods found'. btw, I think I know why? It

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964504#comment-14964504 ] Felix Cheung commented on SPARK-10903: -- I was sure I tried that but double-checking now, this didn't

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964642#comment-14964642 ] Felix Cheung commented on SPARK-10903: -- Working from SparkR package version in this commit:

[jira] [Created] (SPARK-11221) Fix R doc for DataFrame function `lit`

2015-10-20 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11221: Summary: Fix R doc for DataFrame function `lit` Key: SPARK-11221 URL: https://issues.apache.org/jira/browse/SPARK-11221 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11199) Improve R context management story and add getOrCreate

2015-10-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966206#comment-14966206 ] Felix Cheung commented on SPARK-11199: -- [~falaki]Nice - I have a slightly different scope in mind -

[jira] [Commented] (SPARK-11139) Make SparkContext.stop() exception-safe

2015-10-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961112#comment-14961112 ] Felix Cheung commented on SPARK-11139: -- Cool, let's take care of SparkContext and StreamingContext

[jira] [Commented] (SPARK-11199) Improve R context management story and add getOrCreate

2015-10-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966218#comment-14966218 ] Felix Cheung commented on SPARK-11199: -- Also, we should consider converging to a single sqlContext

[jira] [Commented] (SPARK-10903) Simplify SQLContext method signatures and use a singleton

2015-10-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966222#comment-14966222 ] Felix Cheung commented on SPARK-10903: -- Thanks for testing this out and helping to diagnose the

[jira] [Updated] (SPARK-10903) Simplify SQLContext method signatures and use a singleton

2015-10-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-10903: - Summary: Simplify SQLContext method signatures and use a singleton (was: Make sqlContext global

[jira] [Commented] (SPARK-9325) Support `collect` on DataFrame columns

2015-10-06 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945446#comment-14945446 ] Felix Cheung commented on SPARK-9325: - My comment was on collect(df$Age). As I've stated,

[jira] [Commented] (SPARK-9443) Expose sampleByKey in SparkR

2015-10-06 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945593#comment-14945593 ] Felix Cheung commented on SPARK-9443: - is this sampleBy

[jira] [Commented] (SPARK-9443) Expose sampleByKey in SparkR

2015-10-06 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945943#comment-14945943 ] Felix Cheung commented on SPARK-9443: - I'd like to take this once we are done with corr, cov, and

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949654#comment-14949654 ] Felix Cheung commented on SPARK-10903: -- +1 perhaps sparkR.init() should create sqlContext and/or

[jira] [Commented] (SPARK-10996) Implement sampleBy() in DataFrameStatFunctions

2015-10-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949107#comment-14949107 ] Felix Cheung commented on SPARK-10996: -- this is a dup of SPARK-9443 > Implement sampleBy() in

[jira] [Updated] (SPARK-11800) Broken link

2015-11-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-11800: - Description: (was: https://zeppelin.incubator.apache.org/docs/manual/dynamicform.html)

[jira] [Closed] (SPARK-11800) Broken link to Spark in Dynamic Form doc

2015-11-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung closed SPARK-11800. Resolution: Invalid > Broken link to Spark in Dynamic Form doc >

[jira] [Created] (SPARK-11800) Broken link to Spark in Dynamic Form doc

2015-11-17 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11800: Summary: Broken link to Spark in Dynamic Form doc Key: SPARK-11800 URL: https://issues.apache.org/jira/browse/SPARK-11800 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-11339) Fix and document the list of functions in R base package that are masked by functions with same name in SparkR

2015-11-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-11339: - Summary: Fix and document the list of functions in R base package that are masked by functions

[jira] [Created] (SPARK-11715) Support corr for Column Aggregration

2015-11-12 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-11715: Summary: Support corr for Column Aggregration Key: SPARK-11715 URL: https://issues.apache.org/jira/browse/SPARK-11715 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-11715) R support corr for Column Aggregration

2015-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-11715: - Summary: R support corr for Column Aggregration (was: Support corr for Column Aggregration) >

[jira] [Commented] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717297#comment-14717297 ] Felix Cheung commented on SPARK-9316: -

[jira] [Created] (SPARK-10322) Column %in% function is not working

2015-08-27 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-10322: Summary: Column %in% function is not working Key: SPARK-10322 URL: https://issues.apache.org/jira/browse/SPARK-10322 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717297#comment-14717297 ] Felix Cheung edited comment on SPARK-9316 at 8/27/15 6:58 PM: --

[jira] [Updated] (SPARK-10322) Column %in% function is not exported

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-10322: - Summary: Column %in% function is not exported (was: Column %in% function is not working)

[jira] [Commented] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717328#comment-14717328 ] Felix Cheung commented on SPARK-9316: - As for this, subsetdf - df[age in (19,

[jira] [Commented] (SPARK-10322) Column %in% function is not exported

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717305#comment-14717305 ] Felix Cheung commented on SPARK-10322: --

[jira] [Resolved] (SPARK-10322) Column %in% function is not exported

2015-08-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-10322. -- Resolution: Duplicate Looks like this was fixed last night. Column %in% function is not

[jira] [Commented] (SPARK-9316) Add support for filtering using `[` (synonym for filter / select)

2015-08-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14718267#comment-14718267 ] Felix Cheung commented on SPARK-9316: - The string form is not listed in the code

[jira] [Updated] (SPARK-10346) SparkR mutate and transform should replace column with same name to match R data.frame behavior

2015-08-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-10346: - Description: Spark doesn't seem to replace existing column with the name in mutate (ie.

[jira] [Updated] (SPARK-10346) SparkR mutate and transform should replace column with same name to match R data.frame behavior

2015-08-30 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-10346: - Component/s: SparkR SparkR mutate and transform should replace column with same name to match R

[jira] [Commented] (SPARK-9325) Support `collect` on DataFrame columns

2015-09-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14742860#comment-14742860 ] Felix Cheung commented on SPARK-9325: - This turns out to be not straightforward. Since Column captures

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940231#comment-14940231 ] Felix Cheung commented on SPARK-10903: -- [~shivaram]what do you think? If ok I would love to take

[jira] [Commented] (SPARK-10904) select(df, c("col1", "col2")) fails

2015-10-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940239#comment-14940239 ] Felix Cheung commented on SPARK-10904: -- Do you have the repo steps/code? > select(df, c("col1",

[jira] [Commented] (SPARK-10894) Add 'drop' support for DataFrame's subset function

2015-10-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940252#comment-14940252 ] Felix Cheung commented on SPARK-10894: -- To Shivaram's point, I think it is intentional that

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940229#comment-14940229 ] Felix Cheung commented on SPARK-10903: -- +1 could/should we have a version of this that checks

[jira] [Commented] (SPARK-10904) select(df, c("col1", "col2")) fails

2015-10-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940416#comment-14940416 ] Felix Cheung commented on SPARK-10904: -- I added that line of comment on "df$age" I could clarify

[jira] [Commented] (SPARK-10904) select(df, c("col1", "col2")) fails

2015-10-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940429#comment-14940429 ] Felix Cheung commented on SPARK-10904: -- `head(df[,c("Sepal_Width", "Sepal_Length")])` this works, I

[jira] [Commented] (SPARK-10903) Make sqlContext global

2015-10-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940694#comment-14940694 ] Felix Cheung commented on SPARK-10903: -- toDF is doing checks in preference order btw, what functions

[jira] [Commented] (SPARK-9319) Add support for setting column names, types

2015-09-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934377#comment-14934377 ] Felix Cheung commented on SPARK-9319: - [~falaki]? > Add support for setting column names, types >

[jira] [Created] (SPARK-10346) SparkR mutate and transform should replace column with same name to match R data.frame behavior

2015-08-28 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-10346: Summary: SparkR mutate and transform should replace column with same name to match R data.frame behavior Key: SPARK-10346 URL: https://issues.apache.org/jira/browse/SPARK-10346

[jira] [Commented] (SPARK-11886) R function name conflicts with base or stats package ones

2015-12-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034700#comment-15034700 ] Felix Cheung commented on SPARK-11886: -- {code} > library(dplyr) > library(SparkR,

[jira] [Updated] (SPARK-12019) SparkR.init does not support character vector for sparkJars and sparkPackages

2015-12-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-12019: - Summary: SparkR.init does not support character vector for sparkJars and sparkPackages (was:

[jira] [Commented] (SPARK-12019) SparkR.init does not support character vector for sparkJars and sparkPackages

2015-12-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037221#comment-15037221 ] Felix Cheung commented on SPARK-12019: -- per PR comment, we are going to update the code to support

[jira] [Commented] (SPARK-12071) Programming guide should explain NULL in JVM translate to NA in R

2015-12-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037249#comment-15037249 ] Felix Cheung commented on SPARK-12071: -- Also see discussion in

  1   2   3   4   5   6   7   8   9   10   >