[jira] [Resolved] (SPARK-20257) Fix test for directory created to work when running as R CMD check

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20257. -- Resolution: Not A Problem Assignee: Felix Cheung We have disabled this test on CRAN, so

[jira] [Resolved] (SPARK-20613) Double quotes in Windows batch script

2017-05-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20613. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0

[jira] [Commented] (SPARK-20613) Double quotes in Windows batch script

2017-05-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998474#comment-15998474 ] Felix Cheung commented on SPARK-20613: -- [~shivaram]could you add jarretmeyer to contributor list in

[jira] [Resolved] (SPARK-20550) R wrappers for Dataset.alias

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20550. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-20520) R streaming tests failed on Windows

2017-05-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999252#comment-15999252 ] Felix Cheung commented on SPARK-20520: -- waiting for the next RC to try with fix for SPARK-20571 > R

[jira] [Resolved] (SPARK-20614) Use the same log4j configuration with Jenkins in AppVeyor

2017-05-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20614. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.3.0

[jira] [Created] (SPARK-20625) SQL test running on Windows hadoop command warning

2017-05-06 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20625: Summary: SQL test running on Windows hadoop command warning Key: SPARK-20625 URL: https://issues.apache.org/jira/browse/SPARK-20625 Project: Spark Issue

[jira] [Created] (SPARK-20626) Fix SparkR test on Windows with timestamp time zone

2017-05-06 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20626: Summary: Fix SparkR test on Windows with timestamp time zone Key: SPARK-20626 URL: https://issues.apache.org/jira/browse/SPARK-20626 Project: Spark Issue

[jira] [Resolved] (SPARK-20571) Flaky SparkR StructuredStreaming tests

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20571. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0 Target

[jira] [Resolved] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide, R example

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20015. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0 Target

[jira] [Resolved] (SPARK-20543) R should skip long running or non-essential tests when running on CRAN

2017-05-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20543. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0 Target

[jira] [Updated] (SPARK-20889) SparkR grouped documentation for Column methods

2017-05-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20889: - Labels: documentation (was: ) > SparkR grouped documentation for Column methods >

[jira] [Updated] (SPARK-20889) SparkR grouped documentation for Column methods

2017-05-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20889: - Issue Type: Documentation (was: Improvement) > SparkR grouped documentation for Column methods

[jira] [Resolved] (SPARK-20736) PySpark StringIndexer supports StringOrderType

2017-05-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20736. -- Resolution: Fixed Assignee: Wayne Zhang Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-21232) New built-in SQL function - Data_Type

2017-06-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065972#comment-16065972 ] Felix Cheung commented on SPARK-21232: -- I don't see this in Scala - are you proposing we add this?

[jira] [Commented] (SPARK-21802) Make sparkR MLP summary() expose probability column

2017-09-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169575#comment-16169575 ] Felix Cheung commented on SPARK-21802: -- yes if this is from the prediction (with rawPrediction etc)

[jira] [Commented] (SPARK-18131) Support returning Vector/Dense Vector from backend

2017-09-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169579#comment-16169579 ] Felix Cheung commented on SPARK-18131: -- bump. I think this is a real big problem - results from

[jira] [Commented] (SPARK-21802) Make sparkR MLP summary() expose probability column

2017-09-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169487#comment-16169487 ] Felix Cheung commented on SPARK-21802: -- Can you clarify where you see it? I just ran against the

[jira] [Updated] (SPARK-22202) Release tgz content differences for python and R

2017-10-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22202: - Description: As a follow up to SPARK-22167, currently we are running different profiles/steps

[jira] [Created] (SPARK-22281) Handle R method breaking signature changes

2017-10-15 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-22281: Summary: Handle R method breaking signature changes Key: SPARK-22281 URL: https://issues.apache.org/jira/browse/SPARK-22281 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22281) Handle R method breaking signature changes

2017-10-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205313#comment-16205313 ] Felix Cheung commented on SPARK-22281: -- And here for all r-devel WARN

[jira] [Updated] (SPARK-22281) Handle R method breaking signature changes

2017-10-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22281: - Description: cAs discussed here

[jira] [Commented] (SPARK-17608) Long type has incorrect serialization/deserialization

2017-10-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205379#comment-16205379 ] Felix Cheung commented on SPARK-17608: -- any taker on this? > Long type has incorrect

[jira] [Updated] (SPARK-22202) Release tgz content differences for python and R

2017-10-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22202: - Priority: Minor (was: Major) > Release tgz content differences for python and R >

[jira] [Commented] (SPARK-22202) Release tgz content differences for python and R

2017-10-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194141#comment-16194141 ] Felix Cheung commented on SPARK-22202: -- [~holden.ka...@gmail.com] actually, I think for R we would

[jira] [Commented] (SPARK-22202) Release tgz content differences for python and R

2017-10-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193239#comment-16193239 ] Felix Cheung commented on SPARK-22202: -- [~holden.ka...@gmail.com] would you be concerned with the

[jira] [Commented] (SPARK-22202) Release tgz content differences for python and R

2017-10-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193238#comment-16193238 ] Felix Cheung commented on SPARK-22202: -- Yes, exactly. > Release tgz content differences for python

[jira] [Updated] (SPARK-21128) Running R tests multiple times failed due to pre-exiting "spark-warehouse" / "metastore_db"

2017-09-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21128: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0) Fix Version/s: 2.2.1 > Running R tests

[jira] [Assigned] (SPARK-21801) SparkR unit test randomly fail on trees

2017-08-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-21801: Assignee: Felix Cheung > SparkR unit test randomly fail on trees >

[jira] [Resolved] (SPARK-21801) SparkR unit test randomly fail on trees

2017-08-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21801. -- Resolution: Fixed Fix Version/s: 2.3.0 > SparkR unit test randomly fail on trees >

[jira] [Reopened] (SPARK-20684) expose createGlobalTempView and dropGlobalTempView in SparkR

2017-09-10 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reopened SPARK-20684: -- > expose createGlobalTempView and dropGlobalTempView in SparkR >

[jira] [Commented] (SPARK-20684) expose createOrReplaceGlobalTempView/createGlobalTempView and dropGlobalTempView in SparkR

2017-09-10 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160420#comment-16160420 ] Felix Cheung commented on SPARK-20684: -- I"m making this primary JIRA for tracking this issue and

[jira] [Updated] (SPARK-20684) expose createOrReplaceGlobalTempView/createGlobalTempView and dropGlobalTempView in SparkR

2017-09-10 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20684: - Summary: expose createOrReplaceGlobalTempView/createGlobalTempView and dropGlobalTempView in

[jira] [Commented] (SPARK-22167) Spark Packaging w/R distro issues

2017-09-30 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187156#comment-16187156 ] Felix Cheung commented on SPARK-22167: -- I think I'd propose a change on this part of the release

[jira] [Commented] (SPARK-22063) Upgrade lintr to latest commit sha1 ID

2017-10-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187574#comment-16187574 ] Felix Cheung commented on SPARK-22063: -- surely, I think we could even start with something simple

[jira] [Comment Edited] (SPARK-22063) Upgrade lintr to latest commit sha1 ID

2017-10-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187574#comment-16187574 ] Felix Cheung edited comment on SPARK-22063 at 10/2/17 9:16 AM: --- surely, I

[jira] [Commented] (SPARK-22167) Spark Packaging w/R distro issues

2017-10-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189954#comment-16189954 ] Felix Cheung commented on SPARK-22167: -- There are likely 2 stages to this. More pressing might be

[jira] [Commented] (SPARK-15799) Release SparkR on CRAN

2017-09-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180261#comment-16180261 ] Felix Cheung commented on SPARK-15799: -- I commented on the PR. I don't think there is any code

[jira] [Resolved] (SPARK-21584) Update R method for summary to call new implementation

2017-08-22 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21584. -- Resolution: Fixed Fix Version/s: 2.3.0 Target Version/s: 2.3.0 > Update R

[jira] [Created] (SPARK-21805) disable R vignettes code on Windows

2017-08-22 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-21805: Summary: disable R vignettes code on Windows Key: SPARK-21805 URL: https://issues.apache.org/jira/browse/SPARK-21805 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21801) SparkR unit test randomly fail on trees

2017-08-22 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136370#comment-16136370 ] Felix Cheung commented on SPARK-21801: -- I've seen it a couple of times. Looking at the test now i

[jira] [Commented] (SPARK-21801) SparkR unit test randomly fail on trees

2017-08-22 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136445#comment-16136445 ] Felix Cheung commented on SPARK-21801: -- https://github.com/apache/spark/pull/19018 > SparkR unit

[jira] [Resolved] (SPARK-21805) disable R vignettes code on Windows

2017-08-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21805. -- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.3.0

[jira] [Comment Edited] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2017-09-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152973#comment-16152973 ] Felix Cheung edited comment on SPARK-21727 at 9/4/17 11:08 PM: --- precisely.

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2017-09-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152973#comment-16152973 ] Felix Cheung commented on SPARK-21727: -- precisely. as far as I can tell, everything should "just

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2017-09-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151437#comment-16151437 ] Felix Cheung commented on SPARK-21727: -- hmm.. I think that's what the error message is saying {code}

[jira] [Commented] (SPARK-12157) Support numpy types as return values of Python UDFs

2017-09-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151336#comment-16151336 ] Felix Cheung commented on SPARK-12157: -- any more thought on this? I think we should at least

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2017-09-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151589#comment-16151589 ] Felix Cheung commented on SPARK-21727: -- any taker of this change? > Operating on an ArrayType in a

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2017-09-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151588#comment-16151588 ] Felix Cheung commented on SPARK-21727: -- That is true. I think the documentation is unclear in this

[jira] [Commented] (SPARK-17275) Flaky test: org.apache.spark.deploy.RPackageUtilsSuite.jars that don't exist are skipped and print warning

2017-10-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196380#comment-16196380 ] Felix Cheung commented on SPARK-17275: -- perhaps we should close this? it's been a year... > Flaky

[jira] [Updated] (SPARK-22202) Release tgz content differences for python and R

2017-10-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22202: - Description: As a follow up to SPARK-22167, currently we are running different profiles/steps

[jira] [Created] (SPARK-22202) Release tgz content differences for python and R

2017-10-04 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-22202: Summary: Release tgz content differences for python and R Key: SPARK-22202 URL: https://issues.apache.org/jira/browse/SPARK-22202 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19700) Design an API for pluggable scheduler implementations

2017-10-11 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200659#comment-16200659 ] Felix Cheung commented on SPARK-19700: -- Not that I'm aware of - I agree it is very important to take

[jira] [Commented] (SPARK-22281) Handle R method breaking signature changes

2017-10-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16214751#comment-16214751 ] Felix Cheung commented on SPARK-22281: -- ok, I have a solution for both. it turns out the fix for glm

[jira] [Commented] (SPARK-21208) Ability to "setLocalProperty" from sc, in sparkR

2017-10-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216438#comment-16216438 ] Felix Cheung commented on SPARK-21208: -- hi - any taker on this? > Ability to "setLocalProperty"

[jira] [Commented] (SPARK-21616) SparkR 2.3.0 migration guide, release note

2017-10-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216439#comment-16216439 ] Felix Cheung commented on SPARK-21616: -- SPARK-17902 > SparkR 2.3.0 migration guide, release note >

[jira] [Updated] (SPARK-22442) Schema generated by Product Encoder doesn't match case class field name when using non-standard characters

2017-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22442: - Target Version/s: 2.2.1, 2.3.0 > Schema generated by Product Encoder doesn't match case class

[jira] [Updated] (SPARK-22054) Allow release managers to inject their keys

2017-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22054: - Affects Version/s: (was: 2.2.1) > Allow release managers to inject their keys >

[jira] [Updated] (SPARK-20536) Extend ColumnName to create StructFields with explicit nullable

2017-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20536: - Affects Version/s: (was: 2.3.0) 2.2.0 > Extend ColumnName to create

[jira] [Updated] (SPARK-20536) Extend ColumnName to create StructFields with explicit nullable

2017-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20536: - Affects Version/s: (was: 2.2.1) 2.3.0 > Extend ColumnName to create

[jira] [Updated] (SPARK-21427) Describe mapGroupsWithState and flatMapGroupsWithState for stateful aggregation in Structured Streaming

2017-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21427: - Affects Version/s: (was: 2.2.1) 2.2.0 > Describe mapGroupsWithState

[jira] [Updated] (SPARK-22402) Allow fetcher URIs to be downloaded to specific locations relative to Mesos Sandbox

2017-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22402: - Affects Version/s: (was: 2.2.1) 2.3.0 2.2.2 >

[jira] [Updated] (SPARK-22055) Port release scripts

2017-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22055: - Affects Version/s: (was: 2.2.1) > Port release scripts > > >

[jira] [Updated] (SPARK-22202) Release tgz content differences for python and R

2017-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22202: - Affects Version/s: (was: 2.2.1) 2.2.0 > Release tgz content

[jira] [Updated] (SPARK-22469) Accuracy problem in comparison with string and numeric

2017-11-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22469: - Target Version/s: 2.2.2 Description: {code:sql} select '1.5' > 0.5; // Result is NULL

[jira] [Commented] (SPARK-22522) Convert to apache-release to publish Maven artifacts to Nexus/repository.apache.org

2017-11-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253918#comment-16253918 ] Felix Cheung commented on SPARK-22522: -- sorry about the confusion - to clarify, the repo is the same

[jira] [Comment Edited] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2017-11-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253918#comment-16253918 ] Felix Cheung edited comment on SPARK-22522 at 11/15/17 6:33 PM: sorry

[jira] [Comment Edited] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2017-11-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253918#comment-16253918 ] Felix Cheung edited comment on SPARK-22522 at 11/15/17 6:33 PM: sorry

[jira] [Updated] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2017-11-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22522: - Description: see http://www.apache.org/dev/publishing-maven-artifacts.html to publish to

[jira] [Updated] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2017-11-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22522: - Summary: Convert to apache-release to publish Maven artifacts (was: Convert to apache-release

[jira] [Created] (SPARK-22522) Convert to apache-release to publish Maven artifacts to Nexus/repository.apache.org

2017-11-14 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-22522: Summary: Convert to apache-release to publish Maven artifacts to Nexus/repository.apache.org Key: SPARK-22522 URL: https://issues.apache.org/jira/browse/SPARK-22522

[jira] [Updated] (SPARK-22522) Convert to apache-release to publish Maven artifacts to Nexus/repository.apache.org

2017-11-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22522: - Description: see http://www.apache.org/dev/publishing-maven-artifacts.html ...at the very least

[jira] [Updated] (SPARK-22522) Convert to apache-release to publish Maven artifacts to Nexus/repository.apache.org

2017-11-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22522: - Description: see http://www.apache.org/dev/publishing-maven-artifacts.html ...at the very least

[jira] [Commented] (SPARK-22522) Convert to apache-release to publish Maven artifacts to Nexus/repository.apache.org

2017-11-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252937#comment-16252937 ] Felix Cheung commented on SPARK-22522: -- and repository.apache.org is been the same place we are

[jira] [Updated] (SPARK-22522) Convert to apache-release to publish Maven artifacts to Nexus/repository.apache.org

2017-11-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22522: - Description: see http://www.apache.org/dev/publishing-maven-artifacts.html ...at the very least

[jira] [Commented] (SPARK-22522) Convert to apache-release to publish Maven artifacts to Nexus/repository.apache.org

2017-11-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252934#comment-16252934 ] Felix Cheung commented on SPARK-22522: -- the profile and associated config is inherited from the

[jira] [Updated] (SPARK-22549) 64KB JVM bytecode limit problem with concat_ws

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22549: - Target Version/s: 2.3.0, 2.2.2 > 64KB JVM bytecode limit problem with concat_ws >

[jira] [Updated] (SPARK-22498) 64KB JVM bytecode limit problem with concat

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22498: - Target Version/s: 2.3.0, 2.2.2 > 64KB JVM bytecode limit problem with concat >

[jira] [Updated] (SPARK-22538) SQLTransformer.transform(inputDataFrame) uncaches inputDataFrame

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22538: - Target Version/s: 2.3.0, 2.2.2 > SQLTransformer.transform(inputDataFrame) uncaches

[jira] [Updated] (SPARK-22508) 64KB JVM bytecode limit problem with GenerateUnsafeRowJoiner.create()

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22508: - Target Version/s: 2.3.0, 2.2.2 > 64KB JVM bytecode limit problem with

[jira] [Updated] (SPARK-22544) FileStreamSource should use its own hadoop conf to call globPathIfNecessary

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22544: - Target Version/s: 2.3.0, 2.2.2 > FileStreamSource should use its own hadoop conf to call

[jira] [Updated] (SPARK-22377) Maven nightly snapshot jenkins jobs are broken on multiple workers due to lsof

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22377: - Fix Version/s: 2.2.2 > Maven nightly snapshot jenkins jobs are broken on multiple workers due to

[jira] [Updated] (SPARK-22471) SQLListener consumes much memory causing OutOfMemoryError

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22471: - Target Version/s: 2.2.1 (was: 2.2.2) Fix Version/s: (was: 2.2.2)

[jira] [Updated] (SPARK-22490) PySpark doc has misleading string for SparkSession.builder

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22490: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0, 2.2.2) Fix Version/s: (was: 2.2.2)

[jira] [Updated] (SPARK-22511) Update maven central repo address

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22511: - Affects Version/s: (was: 2.2.2) Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0, 2.2.2)

[jira] [Updated] (SPARK-22377) Maven nightly snapshot jenkins jobs are broken on multiple workers due to lsof

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22377: - Target Version/s: 2.2.1 (was: 2.2.2) Fix Version/s: (was: 2.2.2)

[jira] [Updated] (SPARK-22469) Accuracy problem in comparison with string and numeric

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22469: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0, 2.2.2) Fix Version/s: (was: 2.2.2)

[jira] [Updated] (SPARK-22549) 64KB JVM bytecode limit problem with concat_ws

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22549: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0, 2.2.2) > 64KB JVM bytecode limit problem with

[jira] [Updated] (SPARK-22538) SQLTransformer.transform(inputDataFrame) uncaches inputDataFrame

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22538: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0, 2.2.2) > SQLTransformer.transform(inputDataFrame)

[jira] [Updated] (SPARK-22498) 64KB JVM bytecode limit problem with concat

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22498: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0, 2.2.2) > 64KB JVM bytecode limit problem with

[jira] [Updated] (SPARK-22550) 64KB JVM bytecode limit problem with elt

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22550: - Target Version/s: 2.2.1, 2.3.0 > 64KB JVM bytecode limit problem with elt >

[jira] [Updated] (SPARK-22508) 64KB JVM bytecode limit problem with GenerateUnsafeRowJoiner.create()

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22508: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0, 2.2.2) > 64KB JVM bytecode limit problem with

[jira] [Updated] (SPARK-22544) FileStreamSource should use its own hadoop conf to call globPathIfNecessary

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22544: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0, 2.2.2) Fix Version/s: (was: 2.2.2)

[jira] [Updated] (SPARK-17920) HiveWriterContainer passes null configuration to serde.initialize, causing NullPointerException in AvroSerde when using avro.schema.url

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-17920: - Target Version/s: 2.3.0, 2.2.2 > HiveWriterContainer passes null configuration to

[jira] [Updated] (SPARK-22548) Incorrect nested AND expression pushed down to JDBC data source

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22548: - Target Version/s: 2.3.0, 2.2.2 > Incorrect nested AND expression pushed down to JDBC data source

[jira] [Updated] (SPARK-22500) 64KB JVM bytecode limit problem with cast

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22500: - Target Version/s: 2.2.0, 2.3.0 > 64KB JVM bytecode limit problem with cast >

[jira] [Updated] (SPARK-22591) GenerateOrdering shouldn't change ctx.INPUT_ROW

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22591: - Target Version/s: 2.3.0, 2.2.2 > GenerateOrdering shouldn't change ctx.INPUT_ROW >

[jira] [Updated] (SPARK-22595) flaky test: CastSuite.SPARK-22500: cast for struct should not generate codes beyond 64KB

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22595: - Target Version/s: 2.3.0, 2.2.2 > flaky test: CastSuite.SPARK-22500: cast for struct should not

[jira] [Updated] (SPARK-22501) 64KB JVM bytecode limit problem with in

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22501: - Target Version/s: 2.3.0, 2.2.2 > 64KB JVM bytecode limit problem with in >

[jira] [Updated] (SPARK-22540) HighlyCompressedMapStatus's avgSize is incorrect

2017-11-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22540: - Target Version/s: 2.3.0, 2.2.2 > HighlyCompressedMapStatus's avgSize is incorrect >

<    5   6   7   8   9   10   11   12   13   14   >