Re: [R-pkg-devel] CRAN submission struggle

[Ivan Krylov]

On Sun, 7 Jan 2024 10:52:44 +0200
Christiaan Pieterse  wrote:

> I have edited my package to have two examples. One uses a small
> self-generated dataset and another uses a big dataset. For the big
> dataset example, I put \donttest{} around it, should this be fine?

The small example is definitely fine: it exercises the code and does so
fast.

The big example wrapped in \donttest{} could be fine, I'm not sure.
I've seen CRAN packages that wrap the long parts of their examples in
\donttest{} while still making sure that R CMD check --run-donttest
exercises most of their code.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Christiaan Pieterse]

Good day

I have edited my package to have two examples. One uses a small
self-generated dataset and another uses a big dataset. For the big dataset
example, I put \donttest{} around it, should this be fine?

I only get one note for the new package, available at
https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep.

Kind regards
Christiaan

On Sat, 6 Jan 2024 at 14:59, Ivan Krylov  wrote:

> On Sat, 6 Jan 2024 15:16:01 +0300
> Ivan Krylov  wrote:
>
> > Congratulations! I also get the single expected NOTE in my checks.
>
> Apologies for the double e-mail, but I've read the code now, and
> wrapping the example of your only function in \dontrun{} will most
> likely not be allowed.
>
> Is it really the case that you cannot remove a single row from the
> example dataset without making the example crash? It may help to write
> a function that would remove rows one by one, make sure that the
> example still runs, and keep doing that until not a single row can be
> removed. The complexity is terrible (something like O(n^k)), but let it
> run for a while, and maybe it'll reduce the dataset enough to fit in
> the example time limit.
>
> --
> Best regards,
> Ivan
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Ivan Krylov]

On Sat, 6 Jan 2024 15:16:01 +0300
Ivan Krylov  wrote:

> Congratulations! I also get the single expected NOTE in my checks.

Apologies for the double e-mail, but I've read the code now, and
wrapping the example of your only function in \dontrun{} will most
likely not be allowed.

Is it really the case that you cannot remove a single row from the
example dataset without making the example crash? It may help to write
a function that would remove rows one by one, make sure that the
example still runs, and keep doing that until not a single row can be
removed. The complexity is terrible (something like O(n^k)), but let it
run for a while, and maybe it'll reduce the dataset enough to fit in
the example time limit.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Ivan Krylov]

On Sat, 6 Jan 2024 13:50:49 +0200
Christiaan Pieterse  wrote:

> Is there a way to confirm that this package is ready for submission?
> I submitted it to https://win-builder.r-project.org/ and
> https://mac.r-project.org/macbuilder/submit.html and
> https://builder.r-hub.io/. All of these seem to only show the
> expected new submission note.

Congratulations! I also get the single expected NOTE in my checks.

This may be your last chance to rename the package from iopspackage to,
say, "IOPS". If you go over the package and the CRAN policy at
 one last time
and deem the package compliant, it should be ready for submission.

The CRAN reviewer may find additional problems and ask you to fix them,
but you are likely most of the way there.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Christiaan Pieterse]

Good day

I think I finally have a final package that only yields the expected 'New
submission' note.
The package is available at:
https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep.

Is there a way to confirm that this package is ready for submission?
I submitted it to https://win-builder.r-project.org/ and
https://mac.r-project.org/macbuilder/submit.html and
https://builder.r-hub.io/. All of these seem to only show the expected new
submission note.

I see the CRAN package submission web form is unavailable until the 8th, so
any confirmation that my package should be fine for submission would be
appreciated.

Thank you
Christiaan

On Tue, 2 Jan 2024 at 14:59, Uwe Ligges 
wrote:

>
>
> On 29.12.2023 09:13, Greg Hunt wrote:
> > Christaan,
> > The elapsed time note is because CRAN expects that examples will be
> > configured to run single threaded and some package that you use, or a
> > package used by a package that you use is multi-threading by default and
> > using more CPU time than clock time. If you cannot figure out how to
> > reconfigure the multi-threaded package, a number of people have found
> that
> > the simplest thing to do is disable running the example (which reduces
> the
> > effective test coverage provided by the example).
>
>
> No and no:
>
> 1.
>user system elapsed
> IOPS 10.06   3.35   35.04
>
> suggests that either the machines this ran on is heavily loaded (so that
> elapsed >> user) or something is waiting like some internet access. [For
> multithreading it would be user > elapsed.]
>
> 2. The solution is not to exclude examples ebtirely, as we need runtime
> checks.
>
> For internet access: Set a timeout and let the exampe fail gracefully in
> case web access takes more than, e.g., 2. sec.
>
> In general: Please reduce each example to less than 5 sec.
>
> So use small toy examples. If you really want to add rel world examples
> that may take longer, then add them in addition to toy examples and wrap
> in \donttest{}.
>
> Best,
> Uwe Ligges
>
>
>
>
>
>
>
>
> >
> > I haven’t encountered the miktex exception file before but i suspect its
> a
> > side effect of a miktex error. Packages should not leave files behind in
> > the temp directory. If you expect a miktex error you need to remove the
> > file. If you don’t you need to track down and fix or work around the bug.
> > The build process is really a quality check on your package.
> >
> > Greg
> >
> > On Fri, 29 Dec 2023 at 3:01 am, Christiaan Pieterse <
> > pietie.cjp.1...@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> Thank you for showing the difference in the ExampleTradeData. I've fixed
> >> this by adding a .Gitignore file and a "data-raw" folder to load the
> >> ExampleTradeData. I hope I did this correctly. When I check the package
> (
> >> https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep) in
> >> RStudio.
> >> I only get 3 notes (see below), and if I run it in PositCloud, it
> crashes
> >> or yields the same 1 ERROR and 2 NOTES result as before. Why might this
> be?
> >> Is it a problem or is it fine if I continue working in RStudio since I
> >> cannot increase the specs in PositCloud because I'm working on a
> research
> >> group account?
> >>
> >> Here are the 3 notes I receive in RStudio:
> >>
> >> The first is the expected New Submission Note.
> >>
> >> The second is the runtime that is too long:
> >> * checking examples ... [43s] NOTE
> >> Examples with CPU (user + system) or elapsed time > 5s
> >>user system elapsed
> >> IOPS 10.06   3.35   35.04
> >> How can I reduce this time? I'm not sure how to reduce the size of my
> >> ExampleTradeData without the check giving errors when running the
> example.
> >>
> >> The third note I am unsure what it means:
> >> * checking for detritus in the temp directory ... NOTE
> >> Found the following files/directories:
> >>'lastMiKTeXException'
> >>
> >> Kind regards
> >> Christiaan
> >>
> >> On Thu, 28 Dec 2023 at 15:55, Ivan Krylov 
> wrote:
> >>
> >>> Hi Christiaan,
> >>>
> >>> В Thu, 28 Dec 2023 14:57:55 +0200
> >>> Christiaan Pieterse  пишет:
> >>>
>  Still, I couldn't figure out why I ran into this problem, so I
>  created a test file called "Test Example.R" (available at the same
>  GitHub repository:
>  https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep).
> >>>
> >>> I see you're always adding or updating files to the GitHub repo by
> >>> means of uploading. While that's certainly one way to use GitHub, it's
> >>> combines the least convenient aspects of two approaches to using
> GitHub.
> >>>
> >>> With GitHub purely in the browser, GitHub is just a website where you
> >>> keep and edit code, running nothing else on the local computer. Code
> >>> can be run in Codespaces or using GitHub Actions. Microsoft will want
> >>> to be paid money to run code on their computers.
> >>>
> >>> With GitHub as a Git remote, there is a local checkout [*] that's kept
> >>> in sync with GitHub by means of commits [**] and 

Re: [R-pkg-devel] CRAN submission struggle

[Uwe Ligges]

On 29.12.2023 09:13, Greg Hunt wrote:

Christaan,
The elapsed time note is because CRAN expects that examples will be
configured to run single threaded and some package that you use, or a
package used by a package that you use is multi-threading by default and
using more CPU time than clock time. If you cannot figure out how to
reconfigure the multi-threaded package, a number of people have found that
the simplest thing to do is disable running the example (which reduces the
effective test coverage provided by the example).



No and no:

1.
  user system elapsed
IOPS 10.06   3.35   35.04

suggests that either the machines this ran on is heavily loaded (so that 
elapsed >> user) or something is waiting like some internet access. [For 
multithreading it would be user > elapsed.]


2. The solution is not to exclude examples ebtirely, as we need runtime 
checks.


For internet access: Set a timeout and let the exampe fail gracefully in 
case web access takes more than, e.g., 2. sec.


In general: Please reduce each example to less than 5 sec.

So use small toy examples. If you really want to add rel world examples 
that may take longer, then add them in addition to toy examples and wrap 
in \donttest{}.


Best,
Uwe Ligges










I haven’t encountered the miktex exception file before but i suspect its a
side effect of a miktex error. Packages should not leave files behind in
the temp directory. If you expect a miktex error you need to remove the
file. If you don’t you need to track down and fix or work around the bug.
The build process is really a quality check on your package.

Greg

On Fri, 29 Dec 2023 at 3:01 am, Christiaan Pieterse <
pietie.cjp.1...@gmail.com> wrote:


Hi,

Thank you for showing the difference in the ExampleTradeData. I've fixed
this by adding a .Gitignore file and a "data-raw" folder to load the
ExampleTradeData. I hope I did this correctly. When I check the package (
https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep) in
RStudio.
I only get 3 notes (see below), and if I run it in PositCloud, it crashes
or yields the same 1 ERROR and 2 NOTES result as before. Why might this be?
Is it a problem or is it fine if I continue working in RStudio since I
cannot increase the specs in PositCloud because I'm working on a research
group account?

Here are the 3 notes I receive in RStudio:

The first is the expected New Submission Note.

The second is the runtime that is too long:
* checking examples ... [43s] NOTE
Examples with CPU (user + system) or elapsed time > 5s
   user system elapsed
IOPS 10.06   3.35   35.04
How can I reduce this time? I'm not sure how to reduce the size of my
ExampleTradeData without the check giving errors when running the example.

The third note I am unsure what it means:
* checking for detritus in the temp directory ... NOTE
Found the following files/directories:
   'lastMiKTeXException'

Kind regards
Christiaan

On Thu, 28 Dec 2023 at 15:55, Ivan Krylov  wrote:


Hi Christiaan,

В Thu, 28 Dec 2023 14:57:55 +0200
Christiaan Pieterse  пишет:


Still, I couldn't figure out why I ran into this problem, so I
created a test file called "Test Example.R" (available at the same
GitHub repository:
https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep).


I see you're always adding or updating files to the GitHub repo by
means of uploading. While that's certainly one way to use GitHub, it's
combines the least convenient aspects of two approaches to using GitHub.

With GitHub purely in the browser, GitHub is just a website where you
keep and edit code, running nothing else on the local computer. Code
can be run in Codespaces or using GitHub Actions. Microsoft will want
to be paid money to run code on their computers.

With GitHub as a Git remote, there is a local checkout [*] that's kept
in sync with GitHub by means of commits [**] and pushes [***], letting
you create meaningful, describable snapshots of changes in your code
spanning multiple files at the same time.

Right now, it probably feels like Dropbox but worse.


This file creates the function in the global environment (note that
this is the same function code as available in the package
"R/iopspackage2.0.R" file), and then runs this function with the same
example as in the package (If you want to try this yourself, just
load the data/ExampleTradeData.rda in before running the Test Example
file). This test file yields no errors when I run it and produces the
correct results. When I then proceed to build and check the package,
it yields the same example error as before. I do not understand why
or what could cause this issue.


The difference is in the ExampleTradeData variable, which "Test
Example.R" doesn't define.

With data(ExampleTradeData), the script works.

With ExampleTradeData <-



read.csv(system.file("extdata","ExampleTradeData.csv",package="iopspackage")),

the script fails exactly the same way as example(IOPS) does.


I'm not sure if I should send out another email to the 

Re: [R-pkg-devel] CRAN submission struggle

[Uwe Ligges]

On 28.12.2023 17:00, Christiaan Pieterse wrote:

Hi,

Thank you for showing the difference in the ExampleTradeData. I've fixed
this by adding a .Gitignore file and a "data-raw" folder to load the
ExampleTradeData. I hope I did this correctly. When I check the package (
https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep) in RStudio.
I only get 3 notes (see below), and if I run it in PositCloud, it crashes
or yields the same 1 ERROR and 2 NOTES result as before. Why might this be?
Is it a problem or is it fine if I continue working in RStudio since I
cannot increase the specs in PositCloud because I'm working on a research
group account?

Here are the 3 notes I receive in RStudio:

The first is the expected New Submission Note.

The second is the runtime that is too long:
* checking examples ... [43s] NOTE
Examples with CPU (user + system) or elapsed time > 5s
   user system elapsed
IOPS 10.06   3.35   35.04
How can I reduce this time? I'm not sure how to reduce the size of my
ExampleTradeData without the check giving errors when running the example.


Use a subset of the data or less iterations?
If this fails for you, then we need code to reproduce...



The third note I am unsure what it means:
* checking for detritus in the temp directory ... NOTE
Found the following files/directories:
   'lastMiKTeXException'


This can typically be ignored.

Best,
Uwe Ligges



Kind regards
Christiaan

On Thu, 28 Dec 2023 at 15:55, Ivan Krylov  wrote:


Hi Christiaan,

В Thu, 28 Dec 2023 14:57:55 +0200
Christiaan Pieterse  пишет:


Still, I couldn't figure out why I ran into this problem, so I
created a test file called "Test Example.R" (available at the same
GitHub repository:
https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep).


I see you're always adding or updating files to the GitHub repo by
means of uploading. While that's certainly one way to use GitHub, it's
combines the least convenient aspects of two approaches to using GitHub.

With GitHub purely in the browser, GitHub is just a website where you
keep and edit code, running nothing else on the local computer. Code
can be run in Codespaces or using GitHub Actions. Microsoft will want
to be paid money to run code on their computers.

With GitHub as a Git remote, there is a local checkout [*] that's kept
in sync with GitHub by means of commits [**] and pushes [***], letting
you create meaningful, describable snapshots of changes in your code
spanning multiple files at the same time.

Right now, it probably feels like Dropbox but worse.


This file creates the function in the global environment (note that
this is the same function code as available in the package
"R/iopspackage2.0.R" file), and then runs this function with the same
example as in the package (If you want to try this yourself, just
load the data/ExampleTradeData.rda in before running the Test Example
file). This test file yields no errors when I run it and produces the
correct results. When I then proceed to build and check the package,
it yields the same example error as before. I do not understand why
or what could cause this issue.


The difference is in the ExampleTradeData variable, which "Test
Example.R" doesn't define.

With data(ExampleTradeData), the script works.

With ExampleTradeData <-

read.csv(system.file("extdata","ExampleTradeData.csv",package="iopspackage")),
the script fails exactly the same way as example(IOPS) does.


I'm not sure if I should send out another email to the developers to
see if someone else spots something I'm not seeing.


It may help to keep Cc: r-package-devel@r-project.org in the e-mails
for the search engines to index the potential solutions in the mailing
list archives.

--
Best regards,
Ivan

[*]
https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository

[**]

https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository

[***]
https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes



[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Ivan Krylov]

On Thu, 28 Dec 2023 18:00:37 +0200
Christiaan Pieterse  wrote:

> I only get 3 notes (see below), and if I run it in PositCloud, it
> crashes or yields the same 1 ERROR and 2 NOTES result as before. Why
> might this be? 

Does the PositCloud check crash with "Killed" (most likely out of RAM)
or with a different error message?

> Is it a problem or is it fine if I continue working in RStudio since
> I cannot increase the specs in PositCloud because I'm working on a
> research group account?

If your local R CMD check works, it should be fine. Rough
specifications for the machines running CRAN checks can be found at
. We ought to
test our packages on the weakest hardware that could plausibly be used
to run our code, but that's not always easy to do. I know I don't
always dig out my old Intel Atom ultraportable to run the checks myself.

> The second is the runtime that is too long:
> * checking examples ... [43s] NOTE
> Examples with CPU (user + system) or elapsed time > 5s
>   user system elapsed
> IOPS 10.06   3.35   35.04

Similar NOTEs can be seen about the use of multi-threading, but here
the "elapsed" (real, as measured by a clock) time exceeds the "user"
(CPU time spent inside applications) + "system" (CPU time spent inside
the operating system kernel) time, so the code uses less than 100% of
one CPU core on average, which fits comfortably in the 200% allowed by
the CRAN policy for examples and tests.

Unfortunately, 35 seconds is still too much.

> How can I reduce this time? I'm not sure how to reduce the size of my
> ExampleTradeData without the check giving errors when running the
> example.

How does the algorithm work? I've seen it fail due to Proximities being
a 0x0 matrix. Can you work backwards from
economiccomplexity::proximity() returning a 0x0 matrix to derive the
requirements that IOPS places on the dataset? It may help to experiment
with sample.int() to subset the rows and see which combinations work.
Perhaps you can reduce the dataset to two countries and two products?

Have you tried profiling? You can profile your code for both speed and
memory use, and replacing less performant idioms with those using less
CPU time and memory may solve both the CPU time problem and the
OOM-crash problem:
https://cran.r-project.org/doc/manuals/R-exts.html#Tidying-and-profiling-R-code
 
> The third note I am unsure what it means:
> * checking for detritus in the temp directory ... NOTE
> Found the following files/directories:
>   'lastMiKTeXException'

Have you installed the inconsolata MiKTeX package?
https://cran.r-project.org/doc/manuals/R-admin.html#LaTeX-on-Windows
Try running R CMD Rd2pdf on your package directory: maybe MiKTeX will
pop up an interactive dialog to let you install any remaining missing
dependencies. If not, there should be a lastMiKTeXException file for
you to read.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Greg Hunt]

Christaan,
The elapsed time note is because CRAN expects that examples will be
configured to run single threaded and some package that you use, or a
package used by a package that you use is multi-threading by default and
using more CPU time than clock time. If you cannot figure out how to
reconfigure the multi-threaded package, a number of people have found that
the simplest thing to do is disable running the example (which reduces the
effective test coverage provided by the example).

I haven’t encountered the miktex exception file before but i suspect its a
side effect of a miktex error. Packages should not leave files behind in
the temp directory. If you expect a miktex error you need to remove the
file. If you don’t you need to track down and fix or work around the bug.
The build process is really a quality check on your package.

Greg

On Fri, 29 Dec 2023 at 3:01 am, Christiaan Pieterse <
pietie.cjp.1...@gmail.com> wrote:

> Hi,
>
> Thank you for showing the difference in the ExampleTradeData. I've fixed
> this by adding a .Gitignore file and a "data-raw" folder to load the
> ExampleTradeData. I hope I did this correctly. When I check the package (
> https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep) in
> RStudio.
> I only get 3 notes (see below), and if I run it in PositCloud, it crashes
> or yields the same 1 ERROR and 2 NOTES result as before. Why might this be?
> Is it a problem or is it fine if I continue working in RStudio since I
> cannot increase the specs in PositCloud because I'm working on a research
> group account?
>
> Here are the 3 notes I receive in RStudio:
>
> The first is the expected New Submission Note.
>
> The second is the runtime that is too long:
> * checking examples ... [43s] NOTE
> Examples with CPU (user + system) or elapsed time > 5s
>   user system elapsed
> IOPS 10.06   3.35   35.04
> How can I reduce this time? I'm not sure how to reduce the size of my
> ExampleTradeData without the check giving errors when running the example.
>
> The third note I am unsure what it means:
> * checking for detritus in the temp directory ... NOTE
> Found the following files/directories:
>   'lastMiKTeXException'
>
> Kind regards
> Christiaan
>
> On Thu, 28 Dec 2023 at 15:55, Ivan Krylov  wrote:
>
> > Hi Christiaan,
> >
> > В Thu, 28 Dec 2023 14:57:55 +0200
> > Christiaan Pieterse  пишет:
> >
> > > Still, I couldn't figure out why I ran into this problem, so I
> > > created a test file called "Test Example.R" (available at the same
> > > GitHub repository:
> > > https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep).
> >
> > I see you're always adding or updating files to the GitHub repo by
> > means of uploading. While that's certainly one way to use GitHub, it's
> > combines the least convenient aspects of two approaches to using GitHub.
> >
> > With GitHub purely in the browser, GitHub is just a website where you
> > keep and edit code, running nothing else on the local computer. Code
> > can be run in Codespaces or using GitHub Actions. Microsoft will want
> > to be paid money to run code on their computers.
> >
> > With GitHub as a Git remote, there is a local checkout [*] that's kept
> > in sync with GitHub by means of commits [**] and pushes [***], letting
> > you create meaningful, describable snapshots of changes in your code
> > spanning multiple files at the same time.
> >
> > Right now, it probably feels like Dropbox but worse.
> >
> > > This file creates the function in the global environment (note that
> > > this is the same function code as available in the package
> > > "R/iopspackage2.0.R" file), and then runs this function with the same
> > > example as in the package (If you want to try this yourself, just
> > > load the data/ExampleTradeData.rda in before running the Test Example
> > > file). This test file yields no errors when I run it and produces the
> > > correct results. When I then proceed to build and check the package,
> > > it yields the same example error as before. I do not understand why
> > > or what could cause this issue.
> >
> > The difference is in the ExampleTradeData variable, which "Test
> > Example.R" doesn't define.
> >
> > With data(ExampleTradeData), the script works.
> >
> > With ExampleTradeData <-
> >
> >
> read.csv(system.file("extdata","ExampleTradeData.csv",package="iopspackage")),
> > the script fails exactly the same way as example(IOPS) does.
> >
> > > I'm not sure if I should send out another email to the developers to
> > > see if someone else spots something I'm not seeing.
> >
> > It may help to keep Cc: r-package-devel@r-project.org in the e-mails
> > for the search engines to index the potential solutions in the mailing
> > list archives.
> >
> > --
> > Best regards,
> > Ivan
> >
> > [*]
> > https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository
> >
> > [**]
> >
> >
> https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository
> >
> > [***]
> > 

Re: [R-pkg-devel] CRAN submission struggle

[Christiaan Pieterse]

Hi,

Thank you for showing the difference in the ExampleTradeData. I've fixed
this by adding a .Gitignore file and a "data-raw" folder to load the
ExampleTradeData. I hope I did this correctly. When I check the package (
https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep) in RStudio.
I only get 3 notes (see below), and if I run it in PositCloud, it crashes
or yields the same 1 ERROR and 2 NOTES result as before. Why might this be?
Is it a problem or is it fine if I continue working in RStudio since I
cannot increase the specs in PositCloud because I'm working on a research
group account?

Here are the 3 notes I receive in RStudio:

The first is the expected New Submission Note.

The second is the runtime that is too long:
* checking examples ... [43s] NOTE
Examples with CPU (user + system) or elapsed time > 5s
  user system elapsed
IOPS 10.06   3.35   35.04
How can I reduce this time? I'm not sure how to reduce the size of my
ExampleTradeData without the check giving errors when running the example.

The third note I am unsure what it means:
* checking for detritus in the temp directory ... NOTE
Found the following files/directories:
  'lastMiKTeXException'

Kind regards
Christiaan

On Thu, 28 Dec 2023 at 15:55, Ivan Krylov  wrote:

> Hi Christiaan,
>
> В Thu, 28 Dec 2023 14:57:55 +0200
> Christiaan Pieterse  пишет:
>
> > Still, I couldn't figure out why I ran into this problem, so I
> > created a test file called "Test Example.R" (available at the same
> > GitHub repository:
> > https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep).
>
> I see you're always adding or updating files to the GitHub repo by
> means of uploading. While that's certainly one way to use GitHub, it's
> combines the least convenient aspects of two approaches to using GitHub.
>
> With GitHub purely in the browser, GitHub is just a website where you
> keep and edit code, running nothing else on the local computer. Code
> can be run in Codespaces or using GitHub Actions. Microsoft will want
> to be paid money to run code on their computers.
>
> With GitHub as a Git remote, there is a local checkout [*] that's kept
> in sync with GitHub by means of commits [**] and pushes [***], letting
> you create meaningful, describable snapshots of changes in your code
> spanning multiple files at the same time.
>
> Right now, it probably feels like Dropbox but worse.
>
> > This file creates the function in the global environment (note that
> > this is the same function code as available in the package
> > "R/iopspackage2.0.R" file), and then runs this function with the same
> > example as in the package (If you want to try this yourself, just
> > load the data/ExampleTradeData.rda in before running the Test Example
> > file). This test file yields no errors when I run it and produces the
> > correct results. When I then proceed to build and check the package,
> > it yields the same example error as before. I do not understand why
> > or what could cause this issue.
>
> The difference is in the ExampleTradeData variable, which "Test
> Example.R" doesn't define.
>
> With data(ExampleTradeData), the script works.
>
> With ExampleTradeData <-
>
> read.csv(system.file("extdata","ExampleTradeData.csv",package="iopspackage")),
> the script fails exactly the same way as example(IOPS) does.
>
> > I'm not sure if I should send out another email to the developers to
> > see if someone else spots something I'm not seeing.
>
> It may help to keep Cc: r-package-devel@r-project.org in the e-mails
> for the search engines to index the potential solutions in the mailing
> list archives.
>
> --
> Best regards,
> Ivan
>
> [*]
> https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository
>
> [**]
>
> https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository
>
> [***]
> https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Ivan Krylov]

On Sun, 17 Dec 2023 21:48:51 +0200
Christiaan Pieterse  wrote:

> Warning in complexity_measures(Mbin, method = "reflections",
> iterations = iterCompl) :
>   'iterations' was changed to 'iterations + 1' to work with an even
> number of iterations
> Killed

If this is happening on Linux, this could mean the current example (or
some other, completely unrelated process running at the same time)
allocating too much memory and summoning the OOM-killer.

In the current HEAD of the CRAN-prep branch, the man/*.Rd files are
empty, which prevents the package from being installed, so I couldn't
reproduce the problem. Are there any local changes you need to commit
and push to GitHub? If you're not comfortable keeping the package
source in sync with GitHub, we can consider other options.

> * checking HTML version of manual ... NOTE
> Skipping checking HTML validation: no command 'tidy' found

This is a problem with the system running R CMD check, not your
package. In order to check the validity of generated HTML
documentation, R needs to use the program called HTML-Tidy, which is
not installed on this computer.

Where are you getting these results from? Win-Builder
? macOS builder
?

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Christiaan Pieterse]

Hi Ivan

I have decided to clean up the existing package using a branch, as you
suggested. The branch I've created is available at:
https://github.com/WoutersResearchGroup/R-IO-PS/tree/CRAN-prep.

I've cleaned, built and checked this branch package but I get one error and
2 notes (one is the new submission note).
The error is related to the example I'm running. I suspect it might be due
to the way I am trying to import the ExampleTradeData. Here is the error
and note:

* checking examples ... ERROR
Running examples in ‘iopspackage-Ex.R’ failed
The error most likely occurred in:

> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: IOPS
> ### Title: IOPS
> ### Aliases: IOPS
>
> ### ** Examples
>
> # Load the example data
> ExampleTradeData <-
read.csv(system.file("extdata","ExampleTradeData.csv",package="iopspackage"))
>
> # Create a temporary directory
> temp_dir <- tempfile()
> dir.create(temp_dir)
>
> # Set the working directory to the temporary directory
> old_dir <- setwd(temp_dir)
>
> # Then use it in your function
> IOPS(
+   CountryCode = 710,
+   tradeData = ExampleTradeData,
+   ComplexMethod = "reflections",
+   iterCompl = 5,
+   GVCMapping = NULL,
+   tradedigit = 6
+ )
Selected country: ZAF
Export data 6-digit summation complete
Trade data successfully imported
M-matrices calculated
RCA calculated
Starting calculation of complexity measures using the 'reflections' method
Warning in complexity_measures(Mbin, method = "reflections", iterations =
iterCompl) :
  'iterations' was changed to 'iterations + 1' to work with an even number
of iterations
Killed

* checking HTML version of manual ... NOTE
Skipping checking HTML validation: no command 'tidy' found

Note that I also updated the iopspackage2.0.R as you suggested, and I did
re-run roxygen before building and checking (using roxygen2::roxygenize()).
All the updated folders are in the "CRAN-prep" branch (link provided above).

Regards
Christiaan

On Sun, 17 Dec 2023 at 16:06, Ivan Krylov  wrote:

> On Sun, 17 Dec 2023 15:29:34 +0200
> Christiaan Pieterse  wrote:
>
> > But, I've uploaded the newly created package as discussed in my first
> > email, available at:
> > https://github.com/ChristiaanPieterse/iopspackage2.1.0
>
> Are you sure it wouldn't be better to clean up the existing package
> instead of creating unrelated forks? If you're afraid of breaking
> something that works, do the work on a separate branch
> 
> until it both works as well as your current repo currently does *and*
> passes R CMD check --as-cran.
>
> > * checking CRAN incoming feasibility ... [25s] NOTE
> > Maintainer: 'C.J Pieterse '
> > New submission
>
> "New submission" is to be expected.
>
> > Unknown, possibly misspelled, fields in DESCRIPTION:
> >   'Exports'
>
> "Writing R Extensions" doesn't define a field named "Exports". The
> exports are declared in the NAMESPACE file. Since you're using
> roxygen2, use its @export tag to export your functions and remove the
> Exports: field.
>
> > * checking whether package 'iopspackage' can be installed ... [27s]
> > WARNING Found the following significant warnings:
> >   Warning: package 'Rcpp' was built under R version 4.3.2
>
> This is probably not a problem with your package (but may be a problem
> with the way the machine running R CMD check is set up).
>
> > * checking dependencies in R code ... NOTE
> > Namespaces in Imports field not imported from:
> >   'openxlsx' 'roxygen2' 'tibble'
> >   All declared Imports should be used.
>
> > * checking R code for possible problems ... [12s] NOTE
> > IOPS: no visible global function definition for 'createWorkbook'
> > IOPS: no visible global function definition for 'addWorksheet'
> > IOPS: no visible global function definition for 'writeData'
> > IOPS: no visible global function definition for 'saveWorkbook'
> > Undefined global functions or variables:
> >   addWorksheet createWorkbook saveWorkbook writeData
>
> Are you sure you should be importing roxygen2? You only run
> roxygenise() before running R CMD build in order to generate
> man/*.Rd and NAMESPACE; I don't think it's used after that.
>
> If you don't use functions from tibble, there's no need to import it
> or depend on it either. I also don't see you directly using Rcpp, but
> there's no warning about it for some reason.
>
> Use the @importFrom tags to import individual functions that you
> actually use (i.e. createWorkbook and friends). See
> 
> for more information on importing.
>
> Also, remove all library() calls from R/iopspackage2.R. Packages live
> in namespaces, not in the global environment; your package should rely
> upon the dependency information in DESCRIPTION and NAMESPACE (the
> latter generated by roxygen2) for its dependencies.
>
> > > data(ExampleTradeData)
> > Warning in data(ExampleTradeData) :
> >   data set 'ExampleTradeData' not found
>
> 

Re: [R-pkg-devel] CRAN submission struggle

[Ivan Krylov]

On Sun, 17 Dec 2023 15:29:34 +0200
Christiaan Pieterse  wrote:

> But, I've uploaded the newly created package as discussed in my first
> email, available at:
> https://github.com/ChristiaanPieterse/iopspackage2.1.0

Are you sure it wouldn't be better to clean up the existing package
instead of creating unrelated forks? If you're afraid of breaking
something that works, do the work on a separate branch

until it both works as well as your current repo currently does *and*
passes R CMD check --as-cran.

> * checking CRAN incoming feasibility ... [25s] NOTE
> Maintainer: 'C.J Pieterse '
> New submission

"New submission" is to be expected.

> Unknown, possibly misspelled, fields in DESCRIPTION:
>   'Exports'

"Writing R Extensions" doesn't define a field named "Exports". The
exports are declared in the NAMESPACE file. Since you're using
roxygen2, use its @export tag to export your functions and remove the
Exports: field.

> * checking whether package 'iopspackage' can be installed ... [27s]
> WARNING Found the following significant warnings:
>   Warning: package 'Rcpp' was built under R version 4.3.2

This is probably not a problem with your package (but may be a problem
with the way the machine running R CMD check is set up).

> * checking dependencies in R code ... NOTE
> Namespaces in Imports field not imported from:
>   'openxlsx' 'roxygen2' 'tibble'
>   All declared Imports should be used.

> * checking R code for possible problems ... [12s] NOTE
> IOPS: no visible global function definition for 'createWorkbook'
> IOPS: no visible global function definition for 'addWorksheet'
> IOPS: no visible global function definition for 'writeData'
> IOPS: no visible global function definition for 'saveWorkbook'
> Undefined global functions or variables:
>   addWorksheet createWorkbook saveWorkbook writeData

Are you sure you should be importing roxygen2? You only run
roxygenise() before running R CMD build in order to generate
man/*.Rd and NAMESPACE; I don't think it's used after that.

If you don't use functions from tibble, there's no need to import it
or depend on it either. I also don't see you directly using Rcpp, but
there's no warning about it for some reason.

Use the @importFrom tags to import individual functions that you
actually use (i.e. createWorkbook and friends). See

for more information on importing.

Also, remove all library() calls from R/iopspackage2.R. Packages live
in namespaces, not in the global environment; your package should rely
upon the dependency information in DESCRIPTION and NAMESPACE (the
latter generated by roxygen2) for its dependencies.

> > data(ExampleTradeData)  
> Warning in data(ExampleTradeData) :
>   data set 'ExampleTradeData' not found

There's no 'data' directory and no file named ExampleTradeData.* in it.
Data for use by the function data() should be prepared as described in
.
If you want to use files under inst/extdata, you have to read them
manually:

ETD <- read.csv(system.file(
 file.path('extdata','ExampleTradeData.csv'),
 package = 'iopspackage'
))

> * checking for detritus in the temp directory ... NOTE
> Found the following files/directories:
>   'lastMiKTeXException'

Is this on R-hub? This usually happens on R-hub and doesn't indicate a
problem with your package.

> #' temp_dir <- tempdir()
> #' 
> #' # Set the working directory to a temporary directory
> #' setwd(temp_dir)
<...>
> #' # Clean up the temporary directory
> #' unlink(temp_dir, recursive = TRUE)

Please make it a subdirectory of the session temporary directory:

temp_dir <- tempfile()
dir.create(temp_dir)
...

Removing the session temporary directory is not as bad as directly
overwriting user data (it's all going away after the process is shut
down), but quite a lot of parts of R and other packages rely on
tempdir() being a directory that exists, not to mention that there
could be other temporary files in use by other packages.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Christiaan Pieterse]

Hi, thank you for all the responses.

I have uploaded the previous package to GitHub, available at:
https://github.com/WoutersResearchGroup/R-IO-PS . Note that this works when
installed via GitHub using devtools, but is a bit messy. Hence I decided to
start over for CRAN release. If you can help me to get this uploaded to
cran it would be awesome!

But, I've uploaded the newly created package as discussed in my first
email, available at: https://github.com/ChristiaanPieterse/iopspackage2.1.0
This doesn't have a readme. See the readme at the first link above.

Here are the Errors, Warnings and Notes from the check:

* checking CRAN incoming feasibility ... [25s] NOTE
Maintainer: 'C.J Pieterse '
New submission
Unknown, possibly misspelled, fields in DESCRIPTION:
  'Exports'

* checking whether package 'iopspackage' can be installed ... [27s] WARNING
Found the following significant warnings:
  Warning: package 'Rcpp' was built under R version 4.3.2

* checking dependencies in R code ... NOTE
Namespaces in Imports field not imported from:
  'openxlsx' 'roxygen2' 'tibble'
  All declared Imports should be used.

* checking R code for possible problems ... [12s] NOTE
IOPS: no visible global function definition for 'createWorkbook'
IOPS: no visible global function definition for 'addWorksheet'
IOPS: no visible global function definition for 'writeData'
IOPS: no visible global function definition for 'saveWorkbook'
Undefined global functions or variables:
  addWorksheet createWorkbook saveWorkbook writeData

* checking examples ... ERROR
Running examples in 'iopspackage-Ex.R' failed
The error most likely occurred in:
> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: IOPS
> ### Title: IOPS
> ### Aliases: IOPS
>
> ### ** Examples
>
>
> # Load the example data
> data(ExampleTradeData)
Warning in data(ExampleTradeData) :
  data set 'ExampleTradeData' not found
>
> oldwd <- getwd()
> temp_dir <- tempdir()
>
> # Set the working directory to a temporary directory
> setwd(temp_dir)
>
> # Then use it in your function
> IOPS(
+   CountryCode = 710,
+   tradeData = ExampleTradeData,
+   ComplexMethod = "reflections",
+   iterCompl = 5,
+   GVCMapping = NULL,
+   tradedigit = 6
+ )
Selected country: ZAF
Error: object 'ExampleTradeData' not found
Execution halted

* checking for detritus in the temp directory ... NOTE
Found the following files/directories:
  'lastMiKTeXException'

I've struggled for a while with the error on my example that doesn't want
to run. I'd like to know if I've uploaded the ExampleTradeDate correctly as
seen in the inst/extdata file. I was also told previously to create a
temporary directory for the function output and delete the temporary
directory after the example, but I'm unsure if I did it correctly.

Thank you for the assistance
Christiaan

On Sat, 16 Dec 2023 at 20:07, Ivan Krylov  wrote:

> On Sat, 16 Dec 2023 19:41:16 +0200
> Christiaan Pieterse  wrote:
>
> > This .R file contained the Roxygen2 comments. (I was very unsure what
> > comments to include in this so it might be wrong, I'm unsure)
>
> If you like roxygen2 and would like to keep using it, you're welcome to
> keep the roxygen2 comments in your R files. Just don't forget to re-run
> roxygenise() every time you update them.
>
> >3. Included a DESCRIPTION file is the 'iopspackage' folder. (Once
> > again I was very unsure what to include in this file so it might be
> > wrong).
>
> Does it help to follow the guide at
> ?
> Start with the mandatory fields Package, Version, License,
> Description, Title and Authors@R (to generate Author: and Maintainer:
> from).
>
> >8. I checked the tar file using *R CMD check --as-cran
> >"iopspackage_2.1.0.tar.gz". *This yielded errors, warnings and
> > notes which I don't know how to solve and suspect are due to me
> > setting the file up wrong.
>
> Can you show us the log from the check? It should be fine to copy &
> paste the entries that don't say OK (i.e. NOTEs, WARNINGs, and ERRORs).
>
> Most of what you'll need to fix is described in "Writing R Extensions"
> (link above) and the CRAN policy at
> .
>
> > I've been told before not to include my package as an attachment, so
> > can someone please help me with the submission process?
>
> Can you publish the code anywhere? Ideally, this place should provide
> instant access to the latest version of every source code file inside
> your package. The most popular option nowadays is GitHub, but it does
> not have to be GitHub. Two GDPR-friendly alternatives are Codeberg and
> SourceHut. If you don't like Git (which does take effort to learn),
> there's R-Forge and Chiselapp.com. If you don't want to learn software
> version control right now, any free web/file hosting will suffice as
> long as you keep the files updated and accessible.
>
> > It is a 10mb file
>
> I think we've discussed this 

Re: [R-pkg-devel] CRAN submission struggle

[Ivan Krylov]

On Sat, 16 Dec 2023 19:41:16 +0200
Christiaan Pieterse  wrote:

> This .R file contained the Roxygen2 comments. (I was very unsure what
> comments to include in this so it might be wrong, I'm unsure)

If you like roxygen2 and would like to keep using it, you're welcome to
keep the roxygen2 comments in your R files. Just don't forget to re-run
roxygenise() every time you update them.

>3. Included a DESCRIPTION file is the 'iopspackage' folder. (Once
> again I was very unsure what to include in this file so it might be
> wrong).

Does it help to follow the guide at
?
Start with the mandatory fields Package, Version, License,
Description, Title and Authors@R (to generate Author: and Maintainer:
from).

>8. I checked the tar file using *R CMD check --as-cran
>"iopspackage_2.1.0.tar.gz". *This yielded errors, warnings and
> notes which I don't know how to solve and suspect are due to me
> setting the file up wrong.

Can you show us the log from the check? It should be fine to copy &
paste the entries that don't say OK (i.e. NOTEs, WARNINGs, and ERRORs).

Most of what you'll need to fix is described in "Writing R Extensions"
(link above) and the CRAN policy at
.

> I've been told before not to include my package as an attachment, so
> can someone please help me with the submission process?

Can you publish the code anywhere? Ideally, this place should provide
instant access to the latest version of every source code file inside
your package. The most popular option nowadays is GitHub, but it does
not have to be GitHub. Two GDPR-friendly alternatives are Codeberg and
SourceHut. If you don't like Git (which does take effort to learn),
there's R-Forge and Chiselapp.com. If you don't want to learn software
version control right now, any free web/file hosting will suffice as
long as you keep the files updated and accessible.

> It is a 10mb file

I think we've discussed this before. A 10-megabyte package mostly
consisting of example data is not a good fit for CRAN. It's possible to
use free Web hosting services to distribute data packages (see the
'drat' package and the function tools::writePACKAGES) separate from the
CRAN package that should mainly contain the code.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Roy Mendelssohn - NOAA Federal via R-package-devel]

And a second good starting place would be:

https://r-pkgs.org

HTH,

-Roy

> On Dec 16, 2023, at 9:48 AM, Roy Mendelssohn - NOAA Federal 
>  wrote:
> 
> 
> 
>> On Dec 16, 2023, at 9:41 AM, Christiaan Pieterse  
>> wrote:
>> 
>> I checked the tar file using *R CMD check --as-cran
>>  "iopspackage_2.1.0.tar.gz". *This yielded errors, warnings and notes
>>  which I don't know how to solve and suspect are due to me setting the file
>>  up wrong.
> 
> A good start might be to provide the "errors, warnings and notes"
> 
> Thanks,
> 
> -Roy
> **
> "The contents of this message do not reflect any position of the U.S. 
> Government or NOAA."
> **
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: roy.mendelss...@noaa.gov www: https://www.pfeg.noaa.gov/
> 
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected" 
> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
> 

**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Dirk Eddelbuettel]

Christiaan,

You say "errors" but you don't say which.

You say you have a package, but don't provide a source reference.

This makes it awfully hard to say or do anything. In case you are on github
or gitlab or ... it would simply be easiest to share a reference to the
repository.  Emailing 10mb blobs to every list subscriber is not ideal.

Dirk

PS Fortunes has that covered too

   > fortunes::fortune("mind read")

   There are actual error messages, and until you show them, we can not help as 
the mind
   reading machine is currently off for repairs.
  -- Dirk Eddelbuettel (after reports about errors with R CMD check)
 R-help (July 2010)

   > 
-- 
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] CRAN submission struggle

[Roy Mendelssohn - NOAA Federal via R-package-devel]

> On Dec 16, 2023, at 9:41 AM, Christiaan Pieterse  
> wrote:
> 
>  I checked the tar file using *R CMD check --as-cran
>   "iopspackage_2.1.0.tar.gz". *This yielded errors, warnings and notes
>   which I don't know how to solve and suspect are due to me setting the file
>   up wrong.

A good start might be to provide the "errors, warnings and notes"

Thanks,

-Roy
**
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected" 
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel