> Can upgrading R able to fix the issue. Is this perhaps not necessarily malform but some new format for new versions perhaps? That's my guess. I am not totally sure about it tho.
> Anyway we should consider upgrading R version if that fixes the problem. Yea, we should. If we should, it should be more them R 3.4. Maybe it's good time to start to talk about minimum R version. 3.1.x is too old. It's released 4.5 years ago. R 3.4.0 is released 1.5 years ago. Considering the timing for Spark 3.0, deprecating lower versions, bumping up R to 3.4 might be reasonable option. Adding Shane as well. If we ended up with not upgrading it, I will forward this email to CRAN sysadmin to discuss further anyway. 2018년 11월 2일 (금) 오후 12:51, Felix Cheung <felixche...@apache.org>님이 작성: > Thanks for being this up and much appreciate with keeping on top of this > at all times. > > Can upgrading R able to fix the issue. Is this perhaps not necessarily > malform but some new format for new versions perhaps? Anyway we should > consider upgrading R version if that fixes the problem. > > As an option we could also disable the repo check in Jenkins but I can see > that could also be problematic. > > > On Thu, Nov 1, 2018 at 7:35 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: > >> Hi all, >> >> I want to raise the CRAN failure issue because it started to block Spark >> PRs time to time. Since the number >> of PRs grows hugely in Spark community, this is critical to not block >> other PRs. >> >> There has been a problem at CRAN (See >> https://github.com/apache/spark/pull/20005 for analysis). >> To cut it short, the root cause is malformed package info from >> https://cran.r-project.org/src/contrib/PACKAGES >> from server side, and this had to be fixed by requesting it to CRAN >> sysaadmin's help. >> >> https://issues.apache.org/jira/browse/SPARK-24152 <- newly open. I am >> pretty sure it's the same issue >> https://issues.apache.org/jira/browse/SPARK-25923 <- reopen/resolved 2 >> times >> https://issues.apache.org/jira/browse/SPARK-22812 >> >> This happened 5 times for roughly about 10 months, causing blocking >> almost all PRs in Apache Spark. >> Historically, it blocked whole PRs for few days once, and whole Spark >> community had to stop working. >> >> I assume this has been not a super big big issue so far for other >> projects or other people because apparently >> higher version of R has some logics to handle this malformed documents >> (at least I verified R 3.4.0 works fine). >> >> For our side, Jenkins has low R version (R 3.1.1 if that's not updated >> from what I have seen before), >> which is unable to parse the malformed server's response. >> >> So, I want to talk about how we are going to handle this. Possible >> solutions are: >> >> 1. We should start a talk with CRAN sysadmin to permanently prevent this >> issue >> 2. We upgrade R to 3.4.0 in Jenkins (however we will not be able to test >> low R versions) >> 3. ... >> >> If if we fine, I would like to suggest to forward this email to CRAN >> sysadmin to discuss further about this. >> >> Adding Liang-Chi Felix and Shivaram who I already talked about this few >> times before. >> >> Thanks all. >> >> >> >>
