Re: [R] discuss: removing lint-r checks for old branches

2018-08-11 Thread shane knapp
they do seem like real failures on branches 2.0 and 2.1. regarding infrastructure, centos and ubuntu have lintr pinned to 1.0.1.9000, and installed via: devtools::install_github('jimhester/lintr@5431140') builds on branches 2.2+ (and master) are passing R lint checks on both OSes as well. this

[Structured Streaming SPARK-23966] Why non-atomic rename is problem in State Store ?

2018-08-11 Thread chandan prakash
Hi All, I was going through this pull request about new CheckpointFileManager abstraction in structured streaming coming in 2.4 : https://issues.apache.org/jira/browse/SPARK-23966 https://github.com/apache/spark/pull/21048 I went through the code in detail and found it will indtroduce a very nice

Re: [R] discuss: removing lint-r checks for old branches

2018-08-11 Thread Felix Cheung
SGTM for old branches. I recall we need to upgrade to newer lintr since it is missing some tests. Also these seems like real test failures? Are these only happening in 2.1 and 2.2? From: shane knapp Sent: Friday, August 10, 2018 4:04 PM To: Sean Owen Cc:

Re: code freeze and branch cut for Apache Spark 2.4

2018-08-11 Thread Petar Zečević
Hi, I made some changes to SPARK-24020 (https://github.com/apache/spark/pull/21109) and implemented spill-over to disk. I believe there are no objections to the implementation left and that this can now be merged. Please take a look. Thanks, Petar Zečević Wenchen Fan @ 1970-01-01 01:00