Re: CRAN package SparkR
Made a quick fix: https://github.com/apache/spark/pull/33887 I would very appreciate if you guys double check and test against my change for doubly sure .. adding @Shivaram Venkataraman too FYI 2021년 9월 1일 (수) 오전 11:56, Felix Cheung 님이 작성: > I think a few lines to add the prompt might be enough. This checks for > interactive() > > > https://github.com/apache/spark/blob/c6a2021fec5bab9069fbfba33f75d4415ea76e99/R/pkg/R/sparkR.R#L658 > > > On Tue, Aug 31, 2021 at 5:55 PM Hyukjin Kwon wrote: > >> Oh I missed this. Yes, can we simply get the user' confirmation when we >> install.spark? >> IIRC, the auto installation is only triggered by interactive shell so >> getting user's confirmation should be fine. >> >> 2021년 6월 18일 (금) 오전 2:54, Felix Cheung 님이 작성: >> >>> Any suggestion or comment on this? They are going to remove the package >>> by 6-28 >>> >>> Seems to me if we have a switch to opt in to install (and not by default >>> on), or prompt the user in interactive session, should be good as user >>> confirmation. >>> >>> >>> >>> On Sun, Jun 13, 2021 at 11:25 PM Felix Cheung >>> wrote: >>> It looks like they would not allow caching the Spark Distribution. I’m not sure what can be done about this. If I recall, the package should remove this during test. Or maybe spark.install() ie optional (hence getting user confirmation?) -- Forwarded message - Date: Sun, Jun 13, 2021 at 10:19 PM Subject: CRAN package SparkR To: Felix Cheung CC: Dear maintainer, Checking this apparently creates the default directory as per #' @param localDir a local directory where Spark is installed. The directory con tains #' version-specific folders of Spark packages. Default is path t o #' the cache directory: #' \itemize{ #' \item Mac OS X: \file{~/Library/Caches/spark} #' \item Unix: \env{$XDG_CACHE_HOME} if defined, otherwise \file{~/.cache/spark} #' \item Windows: \file{\%LOCALAPPDATA\%\\Apache\\Spark\\Cache}. #' } However, the CRAN Policy says - Packages should not write in the user’s home filespace (including clipboards), nor anywhere else on the file system apart from the R session’s temporary directory (or during installation in the location pointed to by TMPDIR: and such usage should be cleaned up). Installing into the system’s R installation (e.g., scripts to its bin directory) is not allowed. Limited exceptions may be allowed in interactive sessions if the package obtains confirmation from the user. For R version 4.0 or later (hence a version dependency is required or only conditional use is possible), packages may store user-specific data, configuration and cache files in their respective user directories obtained from tools::R_user_dir(), provided that by default sizes are kept as small as possible and the contents are actively managed (including removing outdated material). Can you pls fix as necessary? Please fix before 2021-06-28 to safely retain your package on CRAN. Best -k >>>
Re: CRAN package SparkR
I think a few lines to add the prompt might be enough. This checks for interactive() https://github.com/apache/spark/blob/c6a2021fec5bab9069fbfba33f75d4415ea76e99/R/pkg/R/sparkR.R#L658 On Tue, Aug 31, 2021 at 5:55 PM Hyukjin Kwon wrote: > Oh I missed this. Yes, can we simply get the user' confirmation when we > install.spark? > IIRC, the auto installation is only triggered by interactive shell so > getting user's confirmation should be fine. > > 2021년 6월 18일 (금) 오전 2:54, Felix Cheung 님이 작성: > >> Any suggestion or comment on this? They are going to remove the package >> by 6-28 >> >> Seems to me if we have a switch to opt in to install (and not by default >> on), or prompt the user in interactive session, should be good as user >> confirmation. >> >> >> >> On Sun, Jun 13, 2021 at 11:25 PM Felix Cheung >> wrote: >> >>> It looks like they would not allow caching the Spark >>> Distribution. >>> >>> I’m not sure what can be done about this. >>> >>> If I recall, the package should remove this during test. Or maybe >>> spark.install() ie optional (hence getting user confirmation?) >>> >>> >>> -- Forwarded message - >>> Date: Sun, Jun 13, 2021 at 10:19 PM >>> Subject: CRAN package SparkR >>> To: Felix Cheung >>> CC: >>> >>> >>> Dear maintainer, >>> >>> Checking this apparently creates the default directory as per >>> >>> #' @param localDir a local directory where Spark is installed. The >>> directory con >>> tains >>> #' version-specific folders of Spark packages. Default >>> is path t >>> o >>> #' the cache directory: >>> #' \itemize{ >>> #' \item Mac OS X: \file{~/Library/Caches/spark} >>> #' \item Unix: \env{$XDG_CACHE_HOME} if defined, >>> otherwise \file{~/.cache/spark} >>> #' \item Windows: >>> \file{\%LOCALAPPDATA\%\\Apache\\Spark\\Cache}. >>> #' } >>> >>> However, the CRAN Policy says >>> >>> - Packages should not write in the user’s home filespace (including >>> clipboards), nor anywhere else on the file system apart from the R >>> session’s temporary directory (or during installation in the >>> location pointed to by TMPDIR: and such usage should be cleaned >>> up). Installing into the system’s R installation (e.g., scripts to >>> its bin directory) is not allowed. >>> >>> Limited exceptions may be allowed in interactive sessions if the >>> package obtains confirmation from the user. >>> >>> For R version 4.0 or later (hence a version dependency is required >>> or only conditional use is possible), packages may store >>> user-specific data, configuration and cache files in their >>> respective user directories obtained from tools::R_user_dir(), >>> provided that by default sizes are kept as small as possible and the >>> contents are actively managed (including removing outdated >>> material). >>> >>> Can you pls fix as necessary? >>> >>> Please fix before 2021-06-28 to safely retain your package on CRAN. >>> >>> Best >>> -k >>> >>
Re: CRAN package SparkR
Oh I missed this. Yes, can we simply get the user' confirmation when we install.spark? IIRC, the auto installation is only triggered by interactive shell so getting user's confirmation should be fine. 2021년 6월 18일 (금) 오전 2:54, Felix Cheung 님이 작성: > Any suggestion or comment on this? They are going to remove the package by > 6-28 > > Seems to me if we have a switch to opt in to install (and not by default > on), or prompt the user in interactive session, should be good as user > confirmation. > > > > On Sun, Jun 13, 2021 at 11:25 PM Felix Cheung > wrote: > >> It looks like they would not allow caching the Spark >> Distribution. >> >> I’m not sure what can be done about this. >> >> If I recall, the package should remove this during test. Or maybe >> spark.install() ie optional (hence getting user confirmation?) >> >> >> -- Forwarded message - >> Date: Sun, Jun 13, 2021 at 10:19 PM >> Subject: CRAN package SparkR >> To: Felix Cheung >> CC: >> >> >> Dear maintainer, >> >> Checking this apparently creates the default directory as per >> >> #' @param localDir a local directory where Spark is installed. The >> directory con >> tains >> #' version-specific folders of Spark packages. Default is >> path t >> o >> #' the cache directory: >> #' \itemize{ >> #' \item Mac OS X: \file{~/Library/Caches/spark} >> #' \item Unix: \env{$XDG_CACHE_HOME} if defined, >> otherwise \file{~/.cache/spark} >> #' \item Windows: >> \file{\%LOCALAPPDATA\%\\Apache\\Spark\\Cache}. >> #' } >> >> However, the CRAN Policy says >> >> - Packages should not write in the user’s home filespace (including >> clipboards), nor anywhere else on the file system apart from the R >> session’s temporary directory (or during installation in the >> location pointed to by TMPDIR: and such usage should be cleaned >> up). Installing into the system’s R installation (e.g., scripts to >> its bin directory) is not allowed. >> >> Limited exceptions may be allowed in interactive sessions if the >> package obtains confirmation from the user. >> >> For R version 4.0 or later (hence a version dependency is required >> or only conditional use is possible), packages may store >> user-specific data, configuration and cache files in their >> respective user directories obtained from tools::R_user_dir(), >> provided that by default sizes are kept as small as possible and the >> contents are actively managed (including removing outdated >> material). >> >> Can you pls fix as necessary? >> >> Please fix before 2021-06-28 to safely retain your package on CRAN. >> >> Best >> -k >> >
Re: CRAN package SparkR
Any suggestion or comment on this? They are going to remove the package by 6-28 Seems to me if we have a switch to opt in to install (and not by default on), or prompt the user in interactive session, should be good as user confirmation. On Sun, Jun 13, 2021 at 11:25 PM Felix Cheung wrote: > It looks like they would not allow caching the Spark > Distribution. > > I’m not sure what can be done about this. > > If I recall, the package should remove this during test. Or maybe > spark.install() ie optional (hence getting user confirmation?) > > > -- Forwarded message - > Date: Sun, Jun 13, 2021 at 10:19 PM > Subject: CRAN package SparkR > To: Felix Cheung > CC: > > > Dear maintainer, > > Checking this apparently creates the default directory as per > > #' @param localDir a local directory where Spark is installed. The > directory con > tains > #' version-specific folders of Spark packages. Default is > path t > o > #' the cache directory: > #' \itemize{ > #' \item Mac OS X: \file{~/Library/Caches/spark} > #' \item Unix: \env{$XDG_CACHE_HOME} if defined, > otherwise \file{~/.cache/spark} > #' \item Windows: > \file{\%LOCALAPPDATA\%\\Apache\\Spark\\Cache}. > #' } > > However, the CRAN Policy says > > - Packages should not write in the user’s home filespace (including > clipboards), nor anywhere else on the file system apart from the R > session’s temporary directory (or during installation in the > location pointed to by TMPDIR: and such usage should be cleaned > up). Installing into the system’s R installation (e.g., scripts to > its bin directory) is not allowed. > > Limited exceptions may be allowed in interactive sessions if the > package obtains confirmation from the user. > > For R version 4.0 or later (hence a version dependency is required > or only conditional use is possible), packages may store > user-specific data, configuration and cache files in their > respective user directories obtained from tools::R_user_dir(), > provided that by default sizes are kept as small as possible and the > contents are actively managed (including removing outdated > material). > > Can you pls fix as necessary? > > Please fix before 2021-06-28 to safely retain your package on CRAN. > > Best > -k >