RE: SparkR read.df Option type doesn't match

2015-11-27 Thread Felix Cheung
Yes - please see the code example on the SparkR API doc: http://spark.apache.org/docs/latest/api/R/read.df.html Suggestion or contribution to improve the doc is welcome! > Date: Thu, 26 Nov 2015 15:08:31 -0700 > From: s...@phemi.com > To: dev@spark.apache.org > Subject: Re: SparkR read.df

RE: Are we running SparkR tests in Jenkins?

2016-01-17 Thread Felix Cheung
I think that breaks sparkR, the commandline script, and Jenkins, in which run-test.sh is calling sparkR. I'll work on this - since this also affects my PR #10652... Date: Fri, 15 Jan 2016 15:33:13 -0800 Subject: Re: Are we running SparkR tests in Jenkins? From: zjf...@gmail.com To:

RE: Fwd: Writing to jdbc database from SparkR (1.5.2)

2016-02-07 Thread Felix Cheung
I mean not exposed from the SparkR API. Calling it from R without a SparkR API would require either a serializer change or a JVM wrapper function. On Sun, Feb 7, 2016 at 4:47 AM -0800, "Felix Cheung" <felixcheun...@hotmail.com> wrote: That does but it's a bit hard to c

RE: Fwd: Writing to jdbc database from SparkR (1.5.2)

2016-02-07 Thread Felix Cheung
Correct :) _ From: Sun, Rui <rui@intel.com> Sent: Sunday, February 7, 2016 5:19 AM Subject: RE: Fwd: Writing to jdbc database from SparkR (1.5.2) To: <dev@spark.apache.org>, Felix Cheung <felixcheun...@hotmail.com>, Andrew

Re: [VOTE] Release Apache Spark 2.0.0 (RC5)

2016-07-22 Thread Felix Cheung
+1 Tested on Ubuntu, ran a bunch of SparkR tests, found a broken link in doc but not a blocker. _ From: Michael Armbrust > Sent: Friday, July 22, 2016 3:18 PM Subject: Re: [VOTE] Release Apache Spark 2.0.0 (RC5)

Re: Spark R - Loading Third Party R Library in YARN Executors

2016-08-17 Thread Felix Cheung
When you call library(), that is the library loading function in native R. As of now it does not support HDFS but there are several packages out there that might help. Another approach is to have a prefetch/installation mechanism to call HDFS command to download the R package from HDFS onto

Re: PSA: Java 8 unidoc build

2017-02-07 Thread Felix Cheung
+1 for all the great work going in for this, HyukjinKwon, and +1 on what Sean says about "Jenkins builds with Java 8" and we should catch these nasty javadoc8 issue quickly. I think that would be the great first step to move away from java 7 _ From: Reynold Xin

Re: welcoming Burak and Holden as committers

2017-01-24 Thread Felix Cheung
Congrats and welcome!! From: Reynold Xin Sent: Tuesday, January 24, 2017 10:13:16 AM To: dev@spark.apache.org Cc: Burak Yavuz; Holden Karau Subject: welcoming Burak and Holden as committers Hi all, Burak and Holden have recently been

Re: welcoming Takuya Ueshin as a new Apache Spark committer

2017-02-13 Thread Felix Cheung
Congratulations! From: Xuefu Zhang Sent: Monday, February 13, 2017 11:29:12 AM To: Xiao Li Cc: Holden Karau; Reynold Xin; dev@spark.apache.org Subject: Re: welcoming Takuya Ueshin as a new Apache Spark committer Congratulations, Takuya! --Xuefu

Re: Feedback on MLlib roadmap process proposal

2017-01-19 Thread Felix Cheung
Hi Seth Re: "The most important thing we can do, given that MLlib currently has a very limited committer review bandwidth, is to make clear issues that, if worked on, will definitely get reviewed. " We are adopting a Shepherd model, as described in the JIRA Joseph has, in which, when

Re: PSA: JIRA resolutions and meanings

2016-10-08 Thread Felix Cheung
+1 on this proposal and everyone can contribute to updates and discussions on JIRAs Will be great if this could be put on the Spark wiki. On Sat, Oct 8, 2016 at 9:05 AM -0700, "Ted Yu" > wrote: Makes sense. I trust Hyukjin, Holden and

Re: Suggestion in README.md for guiding pull requests/JIRAs (probably about linking CONTRIBUTING.md or wiki)

2016-10-09 Thread Felix Cheung
Should we just link to https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark On Sun, Oct 9, 2016 at 10:09 AM -0700, "Hyukjin Kwon" > wrote: Thanks for confirming this, Sean. I filed this in

Re: [discuss] Spark 2.x release cadence

2016-09-27 Thread Felix Cheung
+1 on longer release cycle at schedule and more maintenance releases. _ From: Mark Hamstra > Sent: Tuesday, September 27, 2016 2:01 PM Subject: Re: [discuss] Spark 2.x release cadence To: Reynold Xin

Re: [VOTE] Release Apache Spark 2.0.1 (RC4)

2016-10-01 Thread Felix Cheung
+1 Tested and didn't find any blocker - found a few minor R doc issues to follow up. _ From: Reynold Xin > Sent: Wednesday, September 28, 2016 7:15 PM Subject: [VOTE] Release Apache Spark 2.0.1 (RC4) To:

Re: welcoming Xiao Li as a committer

2016-10-04 Thread Felix Cheung
Congrats and welcome, Xiao! _ From: Reynold Xin > Sent: Monday, October 3, 2016 10:47 PM Subject: welcoming Xiao Li as a committer To: Xiao Li >,

Re: Straw poll: dropping support for things like Scala 2.10

2016-10-27 Thread Felix Cheung
+1 on Matei's. _ From: Davies Liu > Sent: Thursday, October 27, 2016 9:58 AM Subject: Re: Straw poll: dropping support for things like Scala 2.10 To: Matei Zaharia

Re: SparkR issue with array types in gapply()

2016-10-27 Thread Felix Cheung
This is a R native data.frame behavior. While arr is a character vector of length = 2, > arr [1] "rows= 50" "cols= 2" > length(arr) [1] 2 when it is set as R data.frame the character vector is splitted into 2 rows > data.frame(key, strings = arr, stringsAsFactors = F) key strings 1 a rows=

Re: Question about SPARK-11374 (skip.header.line.count)

2016-12-10 Thread Felix Cheung
+1 I think it's useful to always have a pure SQL way and skip header for plain text / csv that lots of companies have. From: Dongjoon Hyun Sent: Friday, December 9, 2016 9:42:58 AM To: Dongjin Lee; dev@spark.apache.org Subject: Re: Question

Re: Spark GraphFrame ConnectedComponents

2017-01-05 Thread Felix Cheung
. From: Ankur Srivastava <ankur.srivast...@gmail.com> Sent: Thursday, January 5, 2017 3:45:59 PM To: Felix Cheung; dev@spark.apache.org Cc: u...@spark.apache.org Subject: Re: Spark GraphFrame ConnectedComponents Adding DEV mailing list to see if this is a defect with ConnectedComponent or if th

Re: ml word2vec finSynonyms return type

2016-12-30 Thread Felix Cheung
Could you link to the JIRA here? What you suggest makes sense to me. Though we might want to maintain compatibility and add a new method instead of changing the return type of the existing one. _ From: Asher Krim > Sent:

Re: [ML] [GraphFrames] : Bayesian Network framework

2016-12-30 Thread Felix Cheung
GraphFrames has a Belief Propagation example Have you checked it out? graphframes.github.io/api/scala/index.html#org.graphframes.examples.BeliefPropagation$ From:

Spark checkpointing

2017-01-07 Thread Felix Cheung
From: Steve Loughran <ste...@hortonworks.com> Sent: Friday, January 6, 2017 9:57:05 AM To: Ankur Srivastava Cc: Felix Cheung; u...@spark.apache.org Subject: Re: Spark GraphFrame ConnectedComponents On 5 Jan 2017, at 21:10, Ankur Srivastava <ankur.srivast...@gmail.com<m

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-18 Thread Felix Cheung
nnot fit in memory. On Sat, Dec 17, 2016 at 10:23 AM, Joseph Bradley <jos...@databricks.com<mailto:jos...@databricks.com>> wrote: +1 On Fri, Dec 16, 2016 at 3:21 PM, Herman van Hövell tot Westerflier <hvanhov...@databricks.com<mailto:hvanhov...@databricks.com>> wrote: +1 O

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Felix Cheung
For R we have a license field in the DESCRIPTION, and this is standard practice (and requirement) for R packages. https://cran.r-project.org/doc/manuals/R-exts.html#Licensing From: Sean Owen Sent: Friday, December 16, 2016 9:57:15 AM To:

Re: Outstanding Spark 2.1.1 issues

2017-03-20 Thread Felix Cheung
I've been scrubbing R and think we are tracking 2 issues https://issues.apache.org/jira/browse/SPARK-19237 https://issues.apache.org/jira/browse/SPARK-19925 From: holden.ka...@gmail.com on behalf of Holden Karau

Re: [VOTE] Apache Spark 2.1.1 (RC2)

2017-04-02 Thread Felix Cheung
-1 sorry, found an issue with SparkR CRAN check. Opened SPARK-20197 and working on fix. From: holden.ka...@gmail.com on behalf of Holden Karau Sent: Friday, March 31, 2017 6:25:20 PM To: Xiao Li Cc: Michael

Re: [VOTE] Apache Spark 2.1.1 (RC3)

2017-04-20 Thread Felix Cheung
Tested on both Linux and Windows, as package. Found StackOverflowError with ALS on Windows https://issues.apache.org/jira/browse/SPARK-20402 This is part of the R CRAN check to build the vignettes. Very simple, quick and consistent repo on Windows. The exact same code works fine on Linux.

Re: [SparkR] - options around setting up SparkSession / SparkContext

2017-04-22 Thread Felix Cheung
? _ From: Vin J <winjos...@gmail.com<mailto:winjos...@gmail.com>> Sent: Saturday, April 22, 2017 12:33 AM Subject: Re: [SparkR] - options around setting up SparkSession / SparkContext To: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> C

Re: Should we consider a Spark 2.1.1 release?

2017-03-13 Thread Felix Cheung
+1 there are a lot of good fixes in overall and we need a release for Python and R packages. From: Holden Karau <hol...@pigscanfly.ca> Sent: Monday, March 13, 2017 12:06:47 PM To: Felix Cheung; Shivaram Venkataraman; dev@spark.apache.org Subject: Sho

Re: [VOTE] Apache Spark 2.2.0 (RC6)

2017-07-05 Thread Felix Cheung
+1 (non binding) Tested R, R package on Ubuntu and Windows, CRAN checks, manual tests with steaming & udf. _ From: Denny Lee > Sent: Monday, July 3, 2017 9:30 PM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC6) To:

Re: [SparkR] - options around setting up SparkSession / SparkContext

2017-04-21 Thread Felix Cheung
How would you handle this in Scala? If you are adding a wrapper func like getSparkSession for Scala, and have your users call it, can't you do that same in SparkR? After all, while true you don't need a SparkSession object to call the R API, someone still needs to call sparkR.session() to

Re: [VOTE] Apache Spark 2.1.1 (RC4)

2017-04-28 Thread Felix Cheung
+1 Tested R on linux and windows Previous issue with building vignettes on windows with stackoverflow in ALS still reproduce but as confirmed the issue was in 2.1.0 so this isn't a regression (and hope for the best on CRAN..) https://issues.apache.org/jira/browse/SPARK-20402

Re: Welcoming Hyukjin Kwon and Sameer Agarwal as committers

2017-08-07 Thread Felix Cheung
Congrats!! From: Kevin Kim (Sangwoo) Sent: Monday, August 7, 2017 7:30:01 PM To: Hyukjin Kwon; dev Cc: Bryan Cutler; Mridul Muralidharan; Matei Zaharia; Holden Karau Subject: Re: Welcoming Hyukjin Kwon and Sameer Agarwal as committers Thanks

Re: SBT / PR builder builds failing on "include an external JAR in SparkR"

2017-06-12 Thread Felix Cheung
Facepalm I broken them - I was making changes to test files and of course Jenkins was only running only R tests since I was only changing R files, and everything passed there. Fix is Seq(sparkHome, "R", "pkg", "inst", "tests", To Seq(sparkHome, "R", "pkg", "tests", "fulltests", And 2

Re: [build system] rolling back R to working version

2017-06-20 Thread Felix Cheung
Thanks Shane! From: shane knapp Sent: Tuesday, June 20, 2017 9:23:57 PM To: dev Subject: Re: [build system] rolling back R to working version this is done... i backported R to 3.1.1 and reinstalled all the R packages so we're starting w/a

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-06 Thread Felix Cheung
All tasks on the R QA umbrella are completed SPARK-20512 We can close this. _ From: Sean Owen > Sent: Tuesday, June 6, 2017 1:16 AM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC4) To: Michael Armbrust

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-09 Thread Felix Cheung
Hmm, that's odd. This test would be in Jenkins too - let me double check _ From: Nick Pentreath > Sent: Friday, June 9, 2017 1:12 AM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC4) To: dev

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-13 Thread Felix Cheung
<so...@cloudera.com<mailto:so...@cloudera.com>>, Nick Pentreath <nick.pentre...@gmail.com<mailto:nick.pentre...@gmail.com>>, Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> For the test failure on R, I checked: Per https://githu

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-14 Thread Felix Cheung
Thanks! Will try to setup RHEL/CentOS to test it out _ From: Nick Pentreath <nick.pentre...@gmail.com<mailto:nick.pentre...@gmail.com>> Sent: Tuesday, June 13, 2017 11:38 PM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC4) To: Felix Cheung <felixcheun

Re: [VOTE] Apache Spark 2.2.0 (RC4)

2017-06-15 Thread Felix Cheung
Sounds good. Think we checked and should be good to go. Appreciated. From: Michael Armbrust <mich...@databricks.com> Sent: Wednesday, June 14, 2017 4:51:48 PM To: Hyukjin Kwon Cc: Felix Cheung; Nick Pentreath; dev; Sean Owen Subject: Re: [VOTE] Apache Spark

Re: Spark 2.2.0 or Spark 2.3.0?

2017-05-02 Thread Felix Cheung
Yes 2.2.0 From: kant kodali Sent: Monday, May 1, 2017 10:43:44 PM To: dev Subject: Spark 2.2.0 or Spark 2.3.0? Hi All, If I understand the Spark standard release process correctly. It looks like the official release is going to be sometime

Re: [VOTE] Spark 2.1.2 (RC1)

2017-09-14 Thread Felix Cheung
+1 tested SparkR package on Windows, r-hub, Ubuntu. _ From: Sean Owen > Sent: Thursday, September 14, 2017 3:12 PM Subject: Re: [VOTE] Spark 2.1.2 (RC1) To: Holden Karau >,

Re: [VOTE] Spark 2.1.2 (RC1)

2017-09-15 Thread Felix Cheung
Yes ;) From: Xiao Li <gatorsm...@gmail.com> Sent: Friday, September 15, 2017 2:22:03 PM To: Holden Karau Cc: Ryan Blue; Denny Lee; Felix Cheung; Sean Owen; dev@spark.apache.org Subject: Re: [VOTE] Spark 2.1.2 (RC1) Sorry, this release candidate is

Re: Nightly builds for master branch failed

2017-10-04 Thread Felix Cheung
Hmm, sounds like some sort of corruption of the maven directory on the Jenkins box... From: Liwei Lin Sent: Wednesday, October 4, 2017 6:52:54 PM To: Spark dev list Subject: Nightly builds for master branch failed

Re: Disabling Closed -> Reopened transition for non-committers

2017-10-04 Thread Felix Cheung
To be sure, this is only for JIRA and not for github PR, right? If then +1 but I think the access control on JIRA does not necessarily match the committer list, and is manually maintained, last I hear. From: Sean Owen Sent: Wednesday,

Re: [VOTE] Spark 2.1.2 (RC4)

2017-10-04 Thread Felix Cheung
+1 Tested SparkR package manually on multiple platforms and checked different Hadoop release jar. And previously tested the last RC on different R releases (see the last RC vote thread) I found some differences in bin release jars created by the different options when running the

Re: [VOTE] Spark 2.1.2 (RC4)

2017-10-06 Thread Felix Cheung
Thanks Nick, Hyukjin. Yes this seems to be a longer standing issue on RHEL with respect to forking. From: Nick Pentreath Sent: Friday, October 6, 2017 6:16:53 AM To: Hyukjin Kwon Cc: dev Subject: Re: [VOTE] Spark 2.1.2 (RC4) Ah yes - I

Re: Nightly builds for master branch failed

2017-10-05 Thread Felix Cheung
Thanks Shane! From: shane knapp <skn...@berkeley.edu> Sent: Thursday, October 5, 2017 9:14:54 AM To: Felix Cheung Cc: Liwei Lin; Spark dev list Subject: Re: Nightly builds for master branch failed yep, it was a corrupted jar on amp-jenkins-worker-01. i g

Re: Putting Kafka 0.8 behind an (opt-in) profile

2017-09-05 Thread Felix Cheung
+1 From: Cody Koeninger Sent: Tuesday, September 5, 2017 8:12:07 AM To: Sean Owen Cc: dev Subject: Re: Putting Kafka 0.8 behind an (opt-in) profile +1 to going ahead and giving a deprecation warning now On Tue, Sep 5, 2017 at 6:39 AM, Sean

Re: 2.1.2 maintenance release?

2017-09-08 Thread Felix Cheung
+1 on both 2.1.2 and 2.2.1 And would try to help and/or wrangle the release if needed. (Note: trying to backport a few changes to branch-2.1 right now) From: Sean Owen Sent: Friday, September 8, 2017 12:05:28 AM To: Holden Karau; dev

Re: Welcoming Saisai (Jerry) Shao as a committer

2017-08-29 Thread Felix Cheung
Congrats! From: Wenchen Fan Sent: Tuesday, August 29, 2017 9:21:38 AM To: Kevin Yu Cc: Meisam Fathi; dev Subject: Re: Welcoming Saisai (Jerry) Shao as a committer Congratulations, Saisai! On 29 Aug 2017, at 10:38 PM, Kevin Yu

Re: 2.1.2 maintenance release?

2017-09-11 Thread Felix Cheung
maintenance release? To: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>>, Holden Karau <hol...@pigscanfly.ca<mailto:hol...@pigscanfly.ca>>, Sean Owen <so...@cloudera.com<mailto:so...@cloudera.com>>, dev <dev@spark.apache.org<mail

Re: [VOTE] Spark 2.1.2 (RC2)

2017-09-29 Thread Felix Cheung
-1 (Sorry) spark-2.1.2-bin-hadoop2.7.tgz is missing the R directory, not sure why yet. Tested on multiple platform as source package, (against 2.1.1 jar) seemed fine except this WARNING on R-devel * checking for code/documentation mismatches ... WARNING Codoc mismatches from documentation

Re: [VOTE][SPIP] SPARK-21190: Vectorized UDFs in Python

2017-09-01 Thread Felix Cheung
+1 on this and like the suggestion of type in string form. Would it be correct to assume there will be data type check, for example the returned pandas data frame column data types match what are specified. We have seen quite a bit of issues/confusions with that in R. Would it make sense to

Re: Updates on migration guides

2017-08-31 Thread Felix Cheung
+1 think we do migration guide changes for ML and R in separate JIRA/PR/commit but we definition should have it updated before the release. From: linguin@gmail.com Sent: Wednesday, August 30, 2017 8:27:17 AM To: Dongjoon Hyun Cc: Xiao

Re: Cutting the RC for Spark 2.2.1 release

2017-11-13 Thread Felix Cheung
). There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release. _ From: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> Sen

[VOTE] Spark 2.2.1 (RC1)

2017-11-14 Thread Felix Cheung
Please vote on releasing the following candidate as Apache Spark version 2.2.1. The vote is open until Monday November 20, 2017 at 23:00 UTC and passes if a majority of at least 3 PMC +1 votes are cast. [ ] +1 Release this package as Apache Spark 2.2.1 [ ] -1 Do not release this package because

Re: [VOTE] Spark 2.2.1 (RC1)

2017-11-15 Thread Felix Cheung
.org/jira/browse/SPARK-22469 >> >> Maybe also include this regression of 2.2? It works in 2.1 >> >> Thanks, >> >> Xiao >> >> >> >> 2017-11-14 22:25 GMT-08:00 Felix Cheung <felixche...@apache.org>: >> >>> Please vote on rel

Re: [VOTE] Spark 2.2.1 (RC2)

2017-11-28 Thread Felix Cheung
the latest Debian, so +1 for this release. > > (I committed the change to set -Xss4m for tests consistently, but this > shouldn't block a release.) > > > On Sat, Nov 25, 2017 at 12:47 PM Felix Cheung <felixche...@apache.org> > wrote: > >> Ah sorry digging through the

Re: [VOTE] Spark 2.2.1 (RC2)

2017-11-25 Thread Felix Cheung
d status 1 > > tar: Error is not recoverable: exiting now > > *** RUN ABORTED *** > > java.io.IOException: Cannot run program "./bin/spark-submit" (in > directory "/tmp/test-spark/spark-2.0.2"): error=2, No such file or directory > > On Sat, Nov 25

Re: [VOTE] Spark 2.2.1 (RC2)

2017-11-25 Thread Felix Cheung
, Nov 25, 2017 at 10:36 AM Felix Cheung <felixche...@apache.org> wrote: > Thanks Sean. > > For the second one, it looks like the HiveExternalCatalogVersionsSuite is > trying to download the release tgz from the official Apache mirror, which > won’t work unless the release

[VOTE] Spark 2.2.1 (RC2)

2017-11-24 Thread Felix Cheung
Please vote on releasing the following candidate as Apache Spark version 2.2.1. The vote is open until Friday December 1, 2017 at 8:00:00 am UTC and passes if a majority of at least 3 PMC +1 votes are cast. [ ] +1 Release this package as Apache Spark 2.2.1 [ ] -1 Do not release this package

[RESULT][VOTE] Spark 2.2.1 (RC2)

2017-12-01 Thread Felix Cheung
This vote passes. Thanks everyone for testing this release. +1: Sean Owen (binding) Herman van Hövell tot Westerflier (binding) Wenchen Fan (binding) Shivaram Venkataraman (binding) Felix Cheung Henry Robinson Hyukjin Kwon Dongjoon Hyun Kazuaki Ishizaki Holden Karau Weichen Xu 0

Re: [RESULT][VOTE] Spark 2.2.1 (RC2)

2017-12-14 Thread Felix Cheung
; access? We definitely need to give you all necessary access if you're the > release manager! > > > On Thu, Dec 14, 2017 at 6:32 AM Felix Cheung <felixche...@apache.org> > wrote: > >> And I don’t have access to publish python. >> >> On Wed, De

Re: [RESULT][VOTE] Spark 2.2.1 (RC2)

2017-12-14 Thread Felix Cheung
pretty ready. >> We should announce the release officially too then. >> >> On Wed, Dec 6, 2017 at 5:00 PM Felix Cheung <felixche...@apache.org> >> wrote: >> >>> I saw the svn move on Monday so I’m working on the website updates. >>> >>> I will lo

Re: [VOTE] Spark 2.2.1 (RC1)

2017-11-17 Thread Felix Cheung
ain; see > https://issues.apache.org/jira/browse/SPARK-16845?focusedCommentId=16018840=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16018840 > > > On Wed, Nov 15, 2017 at 12:25 AM Felix Cheung <felixche...@apache.org> > wrote: > > Please vote

Re: Cutting the RC for Spark 2.2.1 release

2017-11-14 Thread Felix Cheung
nterface at repository.apache.org either. ____ From: Felix Cheung <felixcheun...@hotmail.com> Sent: Monday, November 13, 2017 11:23:44 AM To: Sean Owen Cc: Holden Karau; dev@spark.apache.org Subject: Re: Cutting the RC for Spark 2.2.1 release Ouch ;) yes that works and RC1 is t

Re: Cutting the RC for Spark 2.2.1 release

2017-11-13 Thread Felix Cheung
Anything to build with maven on a clean machine. It couldn’t connect to maven central repo. From: Holden Karau <hol...@pigscanfly.ca> Sent: Monday, November 13, 2017 10:38:03 AM To: Felix Cheung Cc: dev@spark.apache.org Subject: Re: Cutting the RC for Spark

Re: Cutting the RC for Spark 2.2.1 release

2017-11-13 Thread Felix Cheung
Ouch ;) yes that works and RC1 is tagged. From: Sean Owen <so...@cloudera.com> Sent: Monday, November 13, 2017 10:54:48 AM To: Felix Cheung Cc: Holden Karau; dev@spark.apache.org Subject: Re: Cutting the RC for Spark 2.2.1 release It's repo.maven.apache.or

Re: Cutting the RC for Spark 2.2.1 release

2017-11-13 Thread Felix Cheung
VNCENTRAL-1369 Stay tuned. ________ From: Felix Cheung <felixcheun...@hotmail.com> Sent: Monday, November 13, 2017 12:00:41 AM To: dev@spark.apache.org Subject: Re: Cutting the RC for Spark 2.2.1 release Quick update: We merged 6 fixes Friday and 7 fixes today (than

Re: Cutting the RC for Spark 2.2.1 release

2017-11-13 Thread Felix Cheung
: Monday, November 13, 2017 10:48 AM Subject: Re: Cutting the RC for Spark 2.2.1 release To: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> Cc: Holden Karau <hol...@pigscanfly.ca<mailto:hol...@pigscanfly.ca>>, <dev@spark.apache.org<mailto:d

Re: [CANCEL] Spark 2.2.1 (RC1)

2017-11-19 Thread Felix Cheung
This vote is cancelled due to no vote. I’m going to test or track down a few issues (please see link below for those targeting this release) and roll RC2 in a few days if we could make progress. On Tue, Nov 14, 2017 at 10:25 PM Felix Cheung <felixche...@apache.org> wrote: > Pl

Re: Kicking off the process around Spark 2.2.1

2017-11-02 Thread Felix Cheung
For the 2.2.1, we are still working through a few bugs. Hopefully it won't be long. From: Kevin Grealish <kevin...@microsoft.com> Sent: Thursday, November 2, 2017 9:51:56 AM To: Felix Cheung; Sean Owen; Holden Karau Cc: dev@spark.apache.org Subject: RE: K

Re: Kicking off the process around Spark 2.2.1

2017-11-02 Thread Felix Cheung
; Sent: Thursday, November 2, 2017 10:38:48 AM To: Felix Cheung; Kevin Grealish; Sean Owen Cc: dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 If it’s desired I’d be happy to start on 2.3 once 2.2.1 is finished. On Thu, Nov 2, 2017 at 10:24 AM Felix Cheung <f

Re: Jenkins upgrade/Test Parallelization & Containerization

2017-11-07 Thread Felix Cheung
We actually have some immediate needs for custom config for some upcoming integration tests. I don't know if such changes are possible in ASF Jenkins but the work is in progress in RISELab Jenkins :) From: holden.ka...@gmail.com on

Re: Kicking off the process around Spark 2.2.1

2017-11-08 Thread Felix Cheung
canfly.ca> Sent: Thursday, November 2, 2017 12:47:13 PM To: Reynold Xin Cc: Felix Cheung; Sean Owen; dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 I agree, except in this case we probably want some of the fixes that are going into the maintenance release t

Cutting the RC for Spark 2.2.1 release

2017-11-08 Thread Felix Cheung
Hi! As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release. If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

Re: Cutting the RC for Spark 2.2.1 release

2017-11-08 Thread Felix Cheung
Thanks Dongjoon! I will track that. From: Dongjoon Hyun <dongjoon.h...@gmail.com> Sent: Wednesday, November 8, 2017 7:41:20 PM To: Holden Karau Cc: Felix Cheung; dev@spark.apache.org Subject: Re: Cutting the RC for Spark 2.2.1 release It's great,

Re: [RESULT][VOTE] Spark 2.2.1 (RC2)

2017-12-06 Thread Felix Cheung
ome question about getting a hand in finishing the release process, > including copying artifacts in svn. Was there anything else you're waiting > on someone to do? > > > On Fri, Dec 1, 2017 at 2:10 AM Felix Cheung <felixche...@apache.org> > wrote: > >> This vote pas

Re: CRAN SparkR package removed?

2017-10-25 Thread Felix Cheung
Yes - unfortunately something was found after it was published and made available publicly. We have a JIRA on this and are working on the best course of action. _ From: Holden Karau > Sent: Wednesday, October 25,

Re: Revisiting Online serving of Spark models?

2018-05-10 Thread Felix Cheung
Huge +1 on this! From: holden.ka...@gmail.com on behalf of Holden Karau Sent: Thursday, May 10, 2018 9:39:26 AM To: Joseph Bradley Cc: dev Subject: Re: Revisiting Online serving of Spark models? On Thu, May 10,

Re: Running lint-java during PR builds?

2018-05-21 Thread Felix Cheung
One concern is with the volume of test runs on Travis. In ASF projects Travis could get significantly backed up since - if I recall - all of ASF shares one queue. At the number of PRs Spark has this could be a big issue. From: Marcelo Vanzin

Re: Revisiting Online serving of Spark models?

2018-05-21 Thread Felix Cheung
+1 on meeting up! From: Holden Karau <hol...@pigscanfly.ca> Sent: Monday, May 21, 2018 2:52:20 PM To: Joseph Bradley Cc: Felix Cheung; dev Subject: Re: Revisiting Online serving of Spark models? (Oh also the write API has already been extended to take f

Re: SparkR was removed from CRAN on 2018-05-01

2018-05-25 Thread Felix Cheung
This is the fix https://github.com/apache/spark/commit/f27a035daf705766d3445e5c6a99867c11c552b0#diff-e1e1d3d40573127e9ee0480caf1283d6 I don't have the email though. From: Hossein Sent: Friday, May 25, 2018 10:58:42 AM To: dev@spark.apache.org

Re: Revisiting Online serving of Spark models?

2018-05-20 Thread Felix Cheung
be interested in getting together to discuss/get some feedback? _ From: Felix Cheung <felixcheun...@hotmail.com> Sent: Thursday, May 10, 2018 10:10 AM Subject: Re: Revisiting Online serving of Spark models? To: Holden Karau <hol...@pigscanfly.ca>, Joseph

Re: Integrating ML/DL frameworks with Spark

2018-05-20 Thread Felix Cheung
Very cool. We would be very interested in this. What is the plan forward to make progress in each of the three areas? From: Bryan Cutler Sent: Monday, May 14, 2018 11:37:20 PM To: Xiangrui Meng Cc: Reynold Xin; dev Subject: Re: Integrating

Re: Scala 2.12 support

2018-06-07 Thread Felix Cheung
+1 Spoke to Dean as well and mentioned the problem with 2.11.12 https://github.com/scala/bug/issues/10913 _ From: Sean Owen Sent: Wednesday, June 6, 2018 12:23 PM Subject: Re: Scala 2.12 support To: Holden Karau Cc: Dean Wampler , Reynold Xin , dev If it means

Re: Time for 2.2.2 release

2018-06-07 Thread Felix Cheung
+1 and thanks! From: Tom Graves Sent: Wednesday, June 6, 2018 7:54:43 AM To: Dev Subject: Time for 2.2.2 release Hello all, I think its time for another 2.2 release. I took a look at Jira and I don't see anything explicitly targeted for 2.2.2 that is not yet

Re: [CRAN-pretest-archived] CRAN submission SparkR 2.3.1

2018-06-12 Thread Felix Cheung
For #1 is system requirements not honored? For #2 it looks like Oracle JDK? From: Shivaram Venkataraman Sent: Tuesday, June 12, 2018 3:17:52 PM To: dev Cc: Felix Cheung Subject: Fwd: [CRAN-pretest-archived] CRAN submission SparkR 2.3.1 Corresponding

Re: Revisiting Online serving of Spark models?

2018-05-26 Thread Felix Cheung
2018 7:47:07 AM To: Maximiliano Felice Cc: Leif Walsh; Felix Cheung; Holden Karau; Joseph Bradley; dev Subject: Re: Revisiting Online serving of Spark models? I’m in the same exact boat as Maximiliano and have use cases as well for model serving and would love to join this discussion. Sent f

Re: Revisiting Online serving of Spark models?

2018-05-30 Thread Felix Cheung
: Re: Revisiting Online serving of Spark models? To: Maximiliano Felice Cc: Felix Cheung , Holden Karau , Joseph Bradley , Leif Walsh , dev Would love to join but am in Seattle, thoughts on how to make this work? Regards Sent from my iPhone On May 29, 2018, at 10:35 AM, Maximiliano Felice

Re: [VOTE] SPIP ML Pipelines in R

2018-05-31 Thread Felix Cheung
+1 With my concerns in the SPIP discussion. From: Hossein Sent: Wednesday, May 30, 2018 2:03:03 PM To: dev@spark.apache.org Subject: [VOTE] SPIP ML Pipelines in R Hi, I started discussion

Re: Revisiting Online serving of Spark models?

2018-05-29 Thread Felix Cheung
Bump. From: Felix Cheung Sent: Saturday, May 26, 2018 1:05:29 PM To: Saikat Kanjilal; Maximiliano Felice; Joseph Bradley Cc: Leif Walsh; Holden Karau; dev Subject: Re: Revisiting Online serving of Spark models? Hi! How about we meet the community and discuss

Re: Revisiting Online serving of Spark models?

2018-05-29 Thread Felix Cheung
You had me at blue bottle! _ From: Holden Karau Sent: Tuesday, May 29, 2018 9:47 AM Subject: Re: Revisiting Online serving of Spark models? To: Felix Cheung Cc: Saikat Kanjilal , Maximiliano Felice , Joseph Bradley , Leif Walsh , dev I'm down for that, we could

Re: [VOTE] Spark 2.1.3 (RC2)

2018-06-27 Thread Felix Cheung
(I don’t want to block the release(s) per se...) We need to backport SPARK-22281 (to branch-2.1 and branch-2.2) This is fixed in 2.3 back in Nov 2017 https://github.com/apache/spark/commit/2ca5aae47a25dc6bc9e333fb592025ff14824501#diff-e1e1d3d40573127e9ee0480caf1283d6 Perhaps we don't get

Re: [VOTE] Spark 2.1.3 (RC2)

2018-06-27 Thread Felix Cheung
Yes, this is broken with newer version of R. We check explicitly for warning for the R check which should fail the test run. From: Marcelo Vanzin Sent: Wednesday, June 27, 2018 6:55 PM To: Felix Cheung Cc: Marcelo Vanzin; Tom Graves; dev Subject: Re: [VOTE

Re: SparkR test failures in PR builder

2018-05-03 Thread Felix Cheung
This is resolved. Please see https://issues.apache.org/jira/browse/SPARK-24152 From: Kazuaki Ishizaki Sent: Wednesday, May 2, 2018 4:51:11 PM To: dev Cc: Joseph Bradley; Hossein Falaki Subject: Re: SparkR test failures in PR builder I am

Re: Kicking off the process around Spark 2.2.1

2017-10-26 Thread Felix Cheung
2017 4:39:15 AM To: Holden Karau Cc: Felix Cheung; dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah. Note also that in theory the code freeze for

Re: Kubernetes backend and docker images

2018-01-06 Thread Felix Cheung
+1 Thanks for taking on this. That was my feedback on one of the long comment thread as well, I think we should have one docker image instead of 3 (also pending in the fork are python and R variant, we should consider having one that we official release instead of 9, for example)

Re: Integration testing and Scheduler Backends

2018-01-08 Thread Felix Cheung
How would (2) be uncommon elsewhere? On Mon, Jan 8, 2018 at 10:16 PM Anirudh Ramanathan wrote: > This is with regard to the Kubernetes Scheduler Backend and scaling the > process to accept contributions. Given we're moving past upstreaming > changes from our fork, and into

  1   2   >