[jira] [Resolved] (SPARK-25572) SparkR tests failed on CRAN on Java 10

[ https://issues.apache.org/jira/browse/SPARK-25572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-25572. -- Resolution: Fixed Fix Version/s: 2.5.0 2.4.1 Target

[jira] [Updated] (SPARK-25572) SparkR tests failed on CRAN on Java 10

[ https://issues.apache.org/jira/browse/SPARK-25572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-25572: - Summary: SparkR tests failed on CRAN on Java 10 (was: SparkR to skip tests because Java 10

[jira] [Created] (SPARK-25572) SparkR to skip tests because Java 10

Felix Cheung created SPARK-25572: Summary: SparkR to skip tests because Java 10 Key: SPARK-25572 URL: https://issues.apache.org/jira/browse/SPARK-25572 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630410#comment-16630410 ] Felix Cheung commented on SPARK-21291: -- Wait. I don’t think saveAsTable is

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628721#comment-16628721 ] Felix Cheung commented on SPARK-21291: -- The PR did not have bucketBy?

Re: spark.lapply

It looks like the native R process is terminated from buffer overflow. Do you know how much data is involved? From: Junior Alvarez Sent: Wednesday, September 26, 2018 7:33 AM To: user@spark.apache.org Subject: spark.lapply Hi! I’m using spark.lapply() in spark

Re: SPIP: support decimals with negative scale in decimal operation

DISCUSS thread is good to have... From: Marco Gaido Sent: Friday, September 21, 2018 3:31 AM To: Wenchen Fan Cc: dev Subject: Re: SPIP: support decimals with negative scale in decimal operation Hi Wenchen, Thank you for the clarification. I agree that this is mo

Re: 2.4.0 Blockers, Critical, etc

I think the point is we actually need to do these validation before completing the release... From: Wenchen Fan Sent: Friday, September 21, 2018 12:02 AM To: Sean Owen Cc: Spark dev list Subject: Re: 2.4.0 Blockers, Critical, etc Sean thanks for checking them!

Re: Mentors wanted for Apache Dubbo (incubating)

Very cool project. Would love to act as mentor (went through incubator with a project myself) but I’m not on IPMC. Would try to help in other ways. Keep it up! On Thu, Sep 20, 2018 at 7:34 PM Huxing Zhang wrote: > Hi community, > > The Apache Dubbo project now has two active mentors, and is lo

Re: [Feedback Requested] SPARK-25299: Using Distributed Storage for Persisting Shuffle Data

Hi +baibing3 +huangtao6 Came across your presentation on Alluxio - including shuffling - would you be interested in this? From: Matt Cheah Sent: Tuesday, September 4, 2018 2:54 PM To: Yuanjian Li Cc: Spark dev list Subject: Re: [Feedback Requested] SPARK-25299:

Re: [DISCUSS] PySpark Window UDF

Definitely! numba numbers are amazing From: Wes McKinney Sent: Saturday, September 8, 2018 7:46 AM To: Li Jin Cc: dev@spark.apache.org Subject: Re: [DISCUSS] PySpark Window UDF hi Li, These results are very cool. I'm excited to see you continuing to push this ef

[jira] [Resolved] (SPARK-23648) extend hint syntax to support any expression for R

[ https://issues.apache.org/jira/browse/SPARK-23648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23648. -- Resolution: Fixed Assignee: Huaxin Gao Fix Version/s: 2.5.0 > extend h

Re: [VOTE] SPARK 2.4.0 (RC1)

If we could work on this quickly - it might get on to future RCs. From: Stavros Kontopoulos Sent: Monday, September 17, 2018 2:35 PM To: Yinan Li Cc: Xiao Li; eerla...@redhat.com; van...@cloudera.com.invalid; Sean Owen; Wenchen Fan; dev Subject: Re: [VOTE] SPAR

[jira] [Commented] (SPARK-24572) "eager execution" for R shell, IDE

[ https://issues.apache.org/jira/browse/SPARK-24572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617120#comment-16617120 ] Felix Cheung commented on SPARK-24572: -- thanks! very close -  showDF doe

[jira] [Comment Edited] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617118#comment-16617118 ] Felix Cheung edited comment on SPARK-21291 at 9/17/18 6:0

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617118#comment-16617118 ] Felix Cheung commented on SPARK-21291: -- I think it should be like this

Re: Should python-2 be supported in Spark 3.0?

I don’t think we should remove any API even in a major release without deprecating it first... From: Mark Hamstra Sent: Sunday, September 16, 2018 12:26 PM To: Erik Erlandson Cc: u...@spark.apache.org; dev Subject: Re: Should python-2 be supported in Spark 3.0?

Re: Should python-2 be supported in Spark 3.0?

I don’t think we should remove any API even in a major release without deprecating it first... From: Mark Hamstra Sent: Sunday, September 16, 2018 12:26 PM To: Erik Erlandson Cc: user@spark.apache.org; dev Subject: Re: Should python-2 be supported in Spark 3.0?

[jira] [Commented] (SPARK-21291) R bucketBy partitionBy API

[ https://issues.apache.org/jira/browse/SPARK-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613994#comment-16613994 ] Felix Cheung commented on SPARK-21291: -- No, you wouldn’t return a writer in

[jira] [Commented] (SPARK-23200) Reset configuration when restarting from checkpoints

[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610089#comment-16610089 ] Felix Cheung commented on SPARK-23200: -- probably need someone to rebuild on

[jira] [Commented] (SPARK-22632) Fix the behavior of timestamp values for R's DataFrame to respect session timezone

[ https://issues.apache.org/jira/browse/SPARK-22632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610083#comment-16610083 ] Felix Cheung commented on SPARK-22632: -- mismatch between R and JVM time zone c

Re: Branch 2.4 is cut

I’m a bit concern about what Arun is summarizing? We are building on DSv2 and already have to rewrite for bunch of changes in master/2.4, increasing in cost for dev work and release management. If we are saying more changes are coming in 3.0, do we have more info on what value the current chang

[jira] [Resolved] (SPARK-25117) Add EXEPT ALL and INTERSECT ALL support in R.

[ https://issues.apache.org/jira/browse/SPARK-25117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-25117. -- Resolution: Fixed Assignee: Dilip Biswal > Add EXEPT ALL and INTERSECT ALL support i

[jira] [Updated] (SPARK-25117) Add EXEPT ALL and INTERSECT ALL support in R.

[ https://issues.apache.org/jira/browse/SPARK-25117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-25117: - Fix Version/s: 2.4.0 > Add EXEPT ALL and INTERSECT ALL support i

Re: Spark JIRA tags clarification and management

+1 good idea. There are a few for organizing but some also are critical to the release process, like rel note. Would be good to clarify. From: Reynold Xin Sent: Sunday, September 2, 2018 11:50 PM To: Hyukjin Kwon Cc: dev Subject: Re: Spark JIRA tags clarification

Re: Jenkins automatic disabling service - who and why?

I think Jenkins asked again after there was an upgrade - seems to happen when the PR was opened for a few months. From: Hyukjin Kwon Sent: Monday, September 3, 2018 7:05 AM To: Sean Owen Cc: dev Subject: Re: Jenkins automatic disabling service - who and why? >

[jira] [Resolved] (SPARK-25007) Add array_intersect / array_except /array_union / array_shuffle to SparkR

[ https://issues.apache.org/jira/browse/SPARK-25007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-25007. -- Resolution: Fixed Fix Version/s: 2.4.0 > Add array_intersect / array_exc

[jira] [Assigned] (SPARK-25007) Add array_intersect / array_except /array_union / array_shuffle to SparkR

[ https://issues.apache.org/jira/browse/SPARK-25007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-25007: Assignee: Huaxin Gao > Add array_intersect / array_except /array_union / array_shuf

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599380#comment-16599380 ] Felix Cheung commented on SPARK-24434: -- .. and let's make sure any discu

[jira] [Assigned] (SPARK-25275) require memberhip in wheel to run 'su' (in dockerfiles)

[ https://issues.apache.org/jira/browse/SPARK-25275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-25275: Assignee: Erik Erlandson > require memberhip in wheel to run 'su' (i

[jira] [Assigned] (SPARK-24433) Add Spark R support

[ https://issues.apache.org/jira/browse/SPARK-24433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-24433: Assignee: Ilan Filonenko > Add Spark R supp

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598300#comment-16598300 ] Felix Cheung commented on SPARK-24434: -- so [~onursatici] is there a reason

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598297#comment-16598297 ] Felix Cheung commented on SPARK-24434: -- [~skonto] - hi what's happening?

Re: [DISCUSS] move away from python doctests

+1 on what Li said. And +1 on getting more coverage in unit tests - however often times we omit python unit tests deliberately if the python “wrapper” is trivial. This is what I’ve learned over the years from the previous pyspark maintainers. Admittedly gaps are there. ___

Re: SPIP: Executor Plugin (SPARK-24918)

+1 From: Mridul Muralidharan Sent: Wednesday, August 29, 2018 1:27:27 PM To: dev@spark.apache.org Subject: Re: SPIP: Executor Plugin (SPARK-24918) +1 I left a couple of comments in NiharS's PR, but this is very useful to have in spark ! Regards, Mridul On Fri, Au

Re: [DISCUSS] SparkR support on k8s back-end for Spark 2.4

I fully support merging that SparkR support on k8s. If Ilan and other are willing to manually validate the RC I’m happy to voucher for it (I’m not 100% sure i have capacity to test it that way but certainly will try) Also +1 on revisiting Jenkins builds. tbh I’m not sure we depend on them too

[jira] [Commented] (SPARK-24918) Executor Plugin API

[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582009#comment-16582009 ] Felix Cheung commented on SPARK-24918: -- I'd tend to agree with opt in -

Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln instead of [NOTEID]/note.json

Perhaps one concern is users having characters in note name that are invalid for file name/file path? From: Mohit Jaggi Sent: Sunday, August 12, 2018 6:02 PM To: users@zeppelin.apache.org Cc: dev Subject: Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln in

Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln instead of [NOTEID]/note.json

Perhaps one concern is users having characters in note name that are invalid for file name/file path? From: Mohit Jaggi Sent: Sunday, August 12, 2018 6:02 PM To: us...@zeppelin.apache.org Cc: dev Subject: Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln in

Re: [R] discuss: removing lint-r checks for old branches

SGTM for old branches. I recall we need to upgrade to newer lintr since it is missing some tests. Also these seems like real test failures? Are these only happening in 2.1 and 2.2? From: shane knapp Sent: Friday, August 10, 2018 4:04 PM To: Sean Owen Cc: Shiva

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569109#comment-16569109 ] Felix Cheung commented on SPARK-24924: --   I tend to agree that we shoul

[jira] [Commented] (SPARK-23683) FileCommitProtocol.instantiate to require 3-arg constructor for dynamic partition overwrite

[ https://issues.apache.org/jira/browse/SPARK-23683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558009#comment-16558009 ] Felix Cheung commented on SPARK-23683: -- should this be ported back to branch

Re: Cleaning Spark releases from mirrors, and the flakiness of HiveExternalCatalogVersionsSuite

+1 this has been problematic. Also, this list needs to be updated every time we make a new release? Plus can we cache them on Jenkins, maybe we can avoid downloading the same thing from Apache archive every test run. From: Marco Gaido Sent: Monday, July 16, 20

Re: [VOTE] SPIP: Standardize SQL logical plans

+1 From: Bruce Robbins Sent: Wednesday, July 18, 2018 3:02 PM To: Ryan Blue Cc: Spark Dev List Subject: Re: [VOTE] SPIP: Standardize SQL logical plans +1 (non-binding) On Tue, Jul 17, 2018 at 10:59 AM, Ryan Blue mailto:b...@apache.org>> wrote: Hi everyone, >F

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541191#comment-16541191 ] Felix Cheung commented on SPARK-20202: -- How like will there be a hive rel

[jira] [Commented] (SPARK-24781) Using a reference from Dataset in Filter/Sort might not work.

[ https://issues.apache.org/jira/browse/SPARK-24781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539627#comment-16539627 ] Felix Cheung commented on SPARK-24781: -- [~jerryshao] > Using a referen

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539622#comment-16539622 ] Felix Cheung commented on SPARK-14220: -- this shouldn't block 2.3.2 right?

[jira] [Assigned] (SPARK-23461) vignettes should include model predictions for some ML models

[ https://issues.apache.org/jira/browse/SPARK-23461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-23461: Assignee: Huaxin Gao > vignettes should include model predictions for some ML mod

[jira] [Resolved] (SPARK-23461) vignettes should include model predictions for some ML models

[ https://issues.apache.org/jira/browse/SPARK-23461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23461. -- Resolution: Fixed Fix Version/s: 2.4.0 > vignettes should include model predicti

[jira] [Resolved] (SPARK-23529) Specify hostpath volume and mount the volume in Spark driver and executor pods in Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-23529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23529. -- Resolution: Fixed Assignee: Andrew Korzhuev (was: Anirudh Ramanathan) Fix

Re: [CRAN-pretest-archived] CRAN submission SparkR 2.2.2

I recall this might be a problem running Spark on java 9 From: Shivaram Venkataraman Sent: Monday, July 9, 2018 2:17 PM To: dev; Felix Cheung; Tom Graves Subject: Fwd: [CRAN-pretest-archived] CRAN submission SparkR 2.2.2 The upcoming 2.2.2 release was submitted

[jira] [Commented] (SPARK-24674) Spark on Kubernetes BLAS performance

[ https://issues.apache.org/jira/browse/SPARK-24674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535983#comment-16535983 ] Felix Cheung commented on SPARK-24674: -- kinda - there has been discussion

[jira] [Commented] (SPARK-24152) SparkR CRAN feasibility check server problem

[ https://issues.apache.org/jira/browse/SPARK-24152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16535345#comment-16535345 ] Felix Cheung commented on SPARK-24152: -- I'm pretty sure it's a

[jira] [Resolved] (SPARK-24535) Fix java version parsing in SparkR on Windows

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-24535. -- Resolution: Fixed Fix Version/s: 2.4.0 2.3.2 Target

[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR on Windows

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534502#comment-16534502 ] Felix Cheung commented on SPARK-24535: -- [~shivaram] I've merged it, but

[jira] [Comment Edited] (SPARK-24535) Fix java version parsing in SparkR on Windows

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530963#comment-16530963 ] Felix Cheung edited comment on SPARK-24535 at 7/3/18 7:5

[jira] [Updated] (SPARK-24535) Fix java version parsing in SparkR on Windows

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-24535: - Summary: Fix java version parsing in SparkR on Windows (was: Fix java version parsing in

[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530963#comment-16530963 ] Felix Cheung commented on SPARK-24535: -- ok, I submitted a modified package to

Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released

Hmm... this is built automatically from the git tag when the release is official. We will need to investigate.. From: Krishna Pandey Sent: Thursday, June 28, 2018 3:26:46 PM To: dev@zeppelin.apache.org Subject: Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released Aweso

[jira] [Updated] (SPARK-24535) Fix java version parsing in SparkR

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-24535: - Target Version/s: 2.3.2 > Fix java version parsing in Spa

[jira] [Updated] (SPARK-24535) Fix java version parsing in SparkR

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-24535: - Priority: Blocker (was: Major) > Fix java version parsing in Spa

[jira] [Assigned] (SPARK-24535) Fix java version parsing in SparkR

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-24535: Assignee: Felix Cheung > Fix java version parsing in Spa

Re: [VOTE] Spark 2.1.3 (RC2)

If I recall we stop releasing Hadoop 2.3 or 2.4 in newer releases (2.2+?) - that might be why they are not the release script. From: Marcelo Vanzin Sent: Thursday, June 28, 2018 11:12:45 AM To: Sean Owen Cc: Marcelo Vanzin; dev Subject: Re: [VOTE] Spark 2.1.3 (R

Re: [VOTE] Spark 2.1.3 (RC2)

Exactly... From: Marcelo Vanzin Sent: Thursday, June 28, 2018 9:16:08 AM To: Tom Graves Cc: Felix Cheung; dev Subject: Re: [VOTE] Spark 2.1.3 (RC2) Yeah, we should be more careful with that in general. Like we state that "Spark runs on Java 8+"... O

Re: Time for 2.3.2?

Yap will do From: Marcelo Vanzin Sent: Thursday, June 28, 2018 9:04:41 AM To: Felix Cheung Cc: Spark dev list Subject: Re: Time for 2.3.2? Could you mark that bug as blocker and set the target version, in that case? On Thu, Jun 28, 2018 at 8:46 AM, Felix Cheung

Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released

Congrats and thanks for putting together the release From: Miquel Angel Andreu Febrer Sent: Wednesday, June 27, 2018 11:02:20 PM To: dev@zeppelin.apache.org Cc: us...@zeppelin.apache.org Subject: Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released Great news, It has b

Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released

Congrats and thanks for putting together the release From: Miquel Angel Andreu Febrer Sent: Wednesday, June 27, 2018 11:02:20 PM To: d...@zeppelin.apache.org Cc: users@zeppelin.apache.org Subject: Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released Great news, It has

Re: Time for 2.3.2?

+1 I’d like to fix SPARK-24535 first though From: Stavros Kontopoulos Sent: Thursday, June 28, 2018 3:50:34 AM To: Marco Gaido Cc: Takeshi Yamamuro; Xingbo Jiang; Wenchen Fan; Spark dev list; Saisai Shao; van...@cloudera.com.invalid Subject: Re: Time for 2.3.2?

Re: [VOTE] Spark 2.1.3 (RC2)

.nabble.com/VOTE-Spark-2-1-2-RC2-tt22540.html#a22555 Since it isn’t a regression I’d say +1 from me. From: Tom Graves Sent: Thursday, June 28, 2018 6:56:16 AM To: Marcelo Vanzin; Felix Cheung Cc: dev Subject: Re: [VOTE] Spark 2.1.3 (RC2) If this is just supporting

Re: [VOTE] Spark 2.1.3 (RC2)

Yes, this is broken with newer version of R. We check explicitly for warning for the R check which should fail the test run. From: Marcelo Vanzin Sent: Wednesday, June 27, 2018 6:55 PM To: Felix Cheung Cc: Marcelo Vanzin; Tom Graves; dev Subject: Re: [VOTE

Re: [VOTE] Spark 2.1.3 (RC2)

(I don’t want to block the release(s) per se...) We need to backport SPARK-22281 (to branch-2.1 and branch-2.2) This is fixed in 2.3 back in Nov 2017 https://github.com/apache/spark/commit/2ca5aae47a25dc6bc9e333fb592025ff14824501#diff-e1e1d3d40573127e9ee0480caf1283d6 Perhaps we don't get Jenkin

Re: Spark 2.3.1 not working on Java 10

I'm not sure we have completed support for Java 10 From: Rahul Agrawal Sent: Thursday, June 21, 2018 7:22:42 AM To: user@spark.apache.org Subject: Spark 2.3.1 not working on Java 10 Dear Team, I have installed Java 10, Scala 2.12.6 and spark 2.3.1 in my desktop

[jira] [Created] (SPARK-24572) "eager execution" for R shell, IDE

Felix Cheung created SPARK-24572: Summary: "eager execution" for R shell, IDE Key: SPARK-24572 URL: https://issues.apache.org/jira/browse/SPARK-24572 Project: Spark Issue

[jira] [Commented] (SPARK-23435) R tests should support latest testthat

[ https://issues.apache.org/jira/browse/SPARK-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513416#comment-16513416 ] Felix Cheung commented on SPARK-23435: -- sorry, I did try but couldn't

[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513372#comment-16513372 ] Felix Cheung commented on SPARK-24535: -- is this only failing on windows? I wo

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513363#comment-16513363 ] Felix Cheung commented on SPARK-24359: -- [~shivaram] sure - do you mean 2.3

Re: [CRAN-pretest-archived] CRAN submission SparkR 2.3.1

For #1 is system requirements not honored? For #2 it looks like Oracle JDK? From: Shivaram Venkataraman Sent: Tuesday, June 12, 2018 3:17:52 PM To: dev Cc: Felix Cheung Subject: Fwd: [CRAN-pretest-archived] CRAN submission SparkR 2.3.1 Corresponding to the

[jira] [Commented] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507476#comment-16507476 ] Felix Cheung commented on SPARK-13184: -- what's the next step?

[jira] [Comment Edited] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506521#comment-16506521 ] Felix Cheung edited comment on SPARK-24359 at 6/8/18 8:1

[jira] [Comment Edited] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506521#comment-16506521 ] Felix Cheung edited comment on SPARK-24359 at 6/8/18 8:1

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506521#comment-16506521 ] Felix Cheung commented on SPARK-24359: -- Thanks Joseph - correct one possibilit

Re: Scala 2.12 support

+1 Spoke to Dean as well and mentioned the problem with 2.11.12 https://github.com/scala/bug/issues/10913 _ From: Sean Owen Sent: Wednesday, June 6, 2018 12:23 PM Subject: Re: Scala 2.12 support To: Holden Karau Cc: Dean Wampler , Reynold Xin , dev If it means n

Re: Time for 2.2.2 release

+1 and thanks! From: Tom Graves Sent: Wednesday, June 6, 2018 7:54:43 AM To: Dev Subject: Time for 2.2.2 release Hello all, I think its time for another 2.2 release. I took a look at Jira and I don't see anything explicitly targeted for 2.2.2 that is not yet co

[jira] [Resolved] (SPARK-24403) reuse r worker

[ https://issues.apache.org/jira/browse/SPARK-24403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-24403. -- Resolution: Duplicate > reuse r worker > -- > > Key:

[jira] [Closed] (SPARK-24403) reuse r worker

[ https://issues.apache.org/jira/browse/SPARK-24403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung closed SPARK-24403. > reuse r worker > -- > > Key: SPARK-24403 >

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16500634#comment-16500634 ] Felix Cheung commented on SPARK-24359: -- +1 on the `spark-website` model for fa

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499528#comment-16499528 ] Felix Cheung commented on SPARK-23206: -- [~irashid] sorry I thought I had rep

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498901#comment-16498901 ] Felix Cheung commented on SPARK-24434: -- I see. Why not then pass along the

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498225#comment-16498225 ] Felix Cheung commented on SPARK-24434: -- java properties is basically the "

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498213#comment-16498213 ] Felix Cheung commented on SPARK-20202: -- Prefer newer Hive also >

[jira] [Resolved] (SPARK-24326) Add local:// scheme support for the app jar in mesos cluster mode

[ https://issues.apache.org/jira/browse/SPARK-24326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-24326. -- Resolution: Fixed Assignee: Stavros Kontopoulos Fix Version/s: 2.4.0 >

Re: [VOTE] SPIP ML Pipelines in R

+1 With my concerns in the SPIP discussion. From: Hossein Sent: Wednesday, May 30, 2018 2:03:03 PM To: dev@spark.apache.org Subject: [VOTE] SPIP ML Pipelines in R Hi, I started discussion thread

Re: Revisiting Online serving of Spark models?

: Re: Revisiting Online serving of Spark models? To: Maximiliano Felice Cc: Felix Cheung , Holden Karau , Joseph Bradley , Leif Walsh , dev Would love to join but am in Seattle, thoughts on how to make this work? Regards Sent from my iPhone On May 29, 2018, at 10:35 AM, Maximiliano Felice

[jira] [Resolved] (SPARK-24331) Add arrays_overlap / array_repeat / map_entries

[ https://issues.apache.org/jira/browse/SPARK-24331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-24331. -- Resolution: Fixed Fix Version/s: 2.4.0 Target Version/s: 2.4.0 >

[jira] [Assigned] (SPARK-24331) Add arrays_overlap / array_repeat / map_entries

[ https://issues.apache.org/jira/browse/SPARK-24331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-24331: Assignee: Marek Novotny > Add arrays_overlap / array_repeat / map_entr

[jira] [Comment Edited] (SPARK-24403) reuse r worker

[ https://issues.apache.org/jira/browse/SPARK-24403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494707#comment-16494707 ] Felix Cheung edited comment on SPARK-24403 at 5/30/18 5:3

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494708#comment-16494708 ] Felix Cheung commented on SPARK-23650: -- sorry, I really don't have time

[jira] [Commented] (SPARK-24403) reuse r worker

[ https://issues.apache.org/jira/browse/SPARK-24403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494707#comment-16494707 ] Felix Cheung commented on SPARK-24403: -- Reuse worker (daemon process) is actu

Re: Revisiting Online serving of Spark models?

You had me at blue bottle! _ From: Holden Karau Sent: Tuesday, May 29, 2018 9:47 AM Subject: Re: Revisiting Online serving of Spark models? To: Felix Cheung Cc: Saikat Kanjilal , Maximiliano Felice , Joseph Bradley , Leif Walsh , dev I'm down for that, we

Re: Revisiting Online serving of Spark models?

Bump. From: Felix Cheung Sent: Saturday, May 26, 2018 1:05:29 PM To: Saikat Kanjilal; Maximiliano Felice; Joseph Bradley Cc: Leif Walsh; Holden Karau; dev Subject: Re: Revisiting Online serving of Spark models? Hi! How about we meet the community and discuss on

<    2   3   4   5   6   7   8   9   10   11   >