Thanks for letting me know! I will look into it and ask CRAN admin for help.
Hyukjin Kwon wrote > Looks it's happening again. Liang-Chi, do you mind if I ask it again? > > FYI, R 3.4 is officially deprecated as of > https://github.com/apache/spark/pull/23012 > We could upgrade R version to 3.4.x in Jenkins, which deals with the > malformed(?) responses after 3.0 release. > Then, we could get rid of this problem..! > > 2018년 11월 12일 (월) 오후 1:47, Hyukjin Kwon < > gurwls223@ > >님이 작성: > >> I made a PR to officially drop R prior to version 3.4 ( >> https://github.com/apache/spark/pull/23012). >> The tests will probably fail for now since it produces warnings for using >> R 3.1.x. >> >> 2018년 11월 11일 (일) 오전 3:00, Felix Cheung < > felixcheung_m@ > >님이 작성: >> >>> It’s a great point about min R version. From what I see, mostly because >>> of fixes and packages support, most users of R are fairly up to date? So >>> perhaps 3.4 as min version is reasonable esp. for Spark 3. >>> >>> Are we getting traction with CRAN sysadmin? It seems like this has been >>> broken a few times. >>> >>> >>> ------------------------------ >>> *From:* Liang-Chi Hsieh < > viirya@ > > >>> *Sent:* Saturday, November 10, 2018 2:32 AM >>> *To:* > dev@.apache >>> *Subject:* Re: [discuss] SparkR CRAN feasibility check server problem >>> >>> >>> Yeah, thanks Hyukjin Kwon for bringing this up for discussion. >>> >>> I don't know how higher versions of R are widely used across R >>> community. >>> If >>> R version 3.1.x was not very commonly used, I think we can discuss to >>> upgrade minimum R version in next Spark version. >>> >>> If we ended up with not upgrading, we can discuss with CRAN sysadmin to >>> fix >>> it by the service side automatically that prevents malformed R packages >>> info. So we don't need to fix it manually every time. >>> >>> >>> >>> Hyukjin Kwon wrote >>> >> Can upgrading R able to fix the issue. Is this perhaps not >>> necessarily >>> > malform but some new format for new versions perhaps? >>> > That's my guess. I am not totally sure about it tho. >>> > >>> >> Anyway we should consider upgrading R version if that fixes the >>> problem. >>> > Yea, we should. If we should, it should be more them R 3.4. Maybe it's >>> > good >>> > time to start to talk about minimum R version. 3.1.x is too old. It's >>> > released 4.5 years ago. >>> > R 3.4.0 is released 1.5 years ago. Considering the timing for Spark >>> 3.0, >>> > deprecating lower versions, bumping up R to 3.4 might be reasonable >>> > option. >>> > >>> > Adding Shane as well. >>> > >>> > If we ended up with not upgrading it, I will forward this email to >>> CRAN >>> > sysadmin to discuss further anyway. >>> > >>> > >>> > >>> > 2018년 11월 2일 (금) 오후 12:51, Felix Cheung < >>> >>> > felixcheung@ >>> >>> > >님이 작성: >>> > >>> >> Thanks for being this up and much appreciate with keeping on top of >>> this >>> >> at all times. >>> >> >>> >> Can upgrading R able to fix the issue. Is this perhaps not >>> necessarily >>> >> malform but some new format for new versions perhaps? Anyway we >>> should >>> >> consider upgrading R version if that fixes the problem. >>> >> >>> >> As an option we could also disable the repo check in Jenkins but I >>> can >>> >> see >>> >> that could also be problematic. >>> >> >>> >> >>> >> On Thu, Nov 1, 2018 at 7:35 PM Hyukjin Kwon < >>> >>> > gurwls223@ >>> >>> > > wrote: >>> >> >>> >>> Hi all, >>> >>> >>> >>> I want to raise the CRAN failure issue because it started to block >>> Spark >>> >>> PRs time to time. Since the number >>> >>> of PRs grows hugely in Spark community, this is critical to not >>> block >>> >>> other PRs. >>> >>> >>> >>> There has been a problem at CRAN (See >>> >>> https://github.com/apache/spark/pull/20005 for analysis). >>> >>> To cut it short, the root cause is malformed package info from >>> >>> https://cran.r-project.org/src/contrib/PACKAGES >>> >>> from server side, and this had to be fixed by requesting it to CRAN >>> >>> sysaadmin's help. >>> >>> >>> >>> https://issues.apache.org/jira/browse/SPARK-24152 <- newly open. I >>> am >>> >>> pretty sure it's the same issue >>> >>> https://issues.apache.org/jira/browse/SPARK-25923 <- reopen/resolved >>> 2 >>> >>> times >>> >>> https://issues.apache.org/jira/browse/SPARK-22812 >>> >>> >>> >>> This happened 5 times for roughly about 10 months, causing blocking >>> >>> almost all PRs in Apache Spark. >>> >>> Historically, it blocked whole PRs for few days once, and whole >>> Spark >>> >>> community had to stop working. >>> >>> >>> >>> I assume this has been not a super big big issue so far for other >>> >>> projects or other people because apparently >>> >>> higher version of R has some logics to handle this malformed >>> documents >>> >>> (at least I verified R 3.4.0 works fine). >>> >>> >>> >>> For our side, Jenkins has low R version (R 3.1.1 if that's not >>> updated >>> >>> from what I have seen before), >>> >>> which is unable to parse the malformed server's response. >>> >>> >>> >>> So, I want to talk about how we are going to handle this. Possible >>> >>> solutions are: >>> >>> >>> >>> 1. We should start a talk with CRAN sysadmin to permanently prevent >>> this >>> >>> issue >>> >>> 2. We upgrade R to 3.4.0 in Jenkins (however we will not be able to >>> test >>> >>> low R versions) >>> >>> 3. ... >>> >>> >>> >>> If if we fine, I would like to suggest to forward this email to CRAN >>> >>> sysadmin to discuss further about this. >>> >>> >>> >>> Adding Liang-Chi Felix and Shivaram who I already talked about this >>> few >>> >>> times before. >>> >>> >>> >>> Thanks all. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: > dev-unsubscribe@.apache >>> >>> -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
